Interface interaction method and device of VR system, equipment and storage medium

Abstract

The embodiment of the application discloses a VR system interface interaction method and device, equipment and storage medium, and belongs to the technical field of human-computer interaction. The method comprises the following steps: displaying an application list window and a 3D system taskbar in a VR system interface, displaying 2D application icons of installed applications in the application list window, displaying the 3D system taskbar in a 3D form, and displaying 3D application icons in the 3D system taskbar; in response to a triggering operation on an editing control, entering a taskbar editing mode, wherein the 3D application icons in the 3D system taskbar are in an editable state in the taskbar editing mode; and in response to an interactive operation on a first 2D application icon in the application list window, adding a first 3D application icon in the 3D system taskbar, wherein the first 2D application icon and the first 3D application icon correspond to the same application.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction technology, and in particular to a method, apparatus, device and storage medium for interface interaction in a VR system. Background Technology

[0002] Virtual Reality (VR) technology refers to the technology of using computers to generate a virtual world that can directly exert visual, auditory, and tactile sensations on users and allow them to interactively observe and operate it.

[0003] In related technologies, the system taskbar of a VR system interface often displays several quick application icons in the form of 2D application icons. These 2D application icons are fixed and cannot be changed, resulting in low efficiency of the system taskbar and a lack of realistic control, thus reducing the user experience. Summary of the Invention

[0004] This application provides a VR system interface interaction method, apparatus, device, and storage medium, which can improve the realism of user operations in VR space. The technical solution is as follows:

[0005] On one hand, embodiments of this application provide a user interface interaction method for a VR system, the method comprising:

[0006] The VR system interface displays an application list window and a 3D system taskbar. The application list window displays 2D application icons of installed applications, and the 3D system taskbar is displayed in 3D form and displays 3D application icons. The application list window and the 3D system taskbar are located in the first field of view of the VR space, and the virtual manipulation objects in the VR space can touch the objects displayed in the first field of view.

[0007] In response to a trigger operation on the editing control, the system enters the taskbar editing mode, in which the 3D application icon in the 3D system taskbar is in an editable state;

[0008] In response to an interactive operation on the first 2D application icon in the application list window, a first 3D application icon is added to the 3D system taskbar, wherein the first 2D application icon and the first 3D application icon correspond to the same application.

[0009] In response to a trigger operation on the first 2D application icon in the application list window, or on the first 3D application icon in the 3D system taskbar, a VR application interface is displayed within a second field of view in the VR space, wherein the second field of view is farther than the first field of view.

[0010] On the other hand, embodiments of this application provide an interface interaction device for a VR system, the device comprising:

[0011] The first display module is used to display an application list window and a 3D system taskbar in the VR system interface. The application list window displays 2D application icons of installed applications. The 3D system taskbar is displayed in 3D form and displays 3D application icons. The application list window and the 3D system taskbar are located within the first field of view of the VR space, and the virtual manipulation objects in the VR space can touch the objects displayed within the first field of view.

[0012] The editing module is used to enter the taskbar editing mode in response to the trigger operation of the editing control. In the taskbar editing mode, the 3D application icon in the 3D system taskbar is in an editable state.

[0013] An icon adding module is used to add a first 3D application icon to the 3D system taskbar in response to an interactive operation on the first 2D application icon in the application list window. The first 2D application icon and the first 3D application icon correspond to the same application.

[0014] The second display module is used to display the VR application interface in a second field of view in the VR space in response to a trigger operation on the first 2D application icon in the application list window, or on the first 3D application icon in the 3D system taskbar. The second field of view is farther than the first field of view.

[0015] On the other hand, embodiments of this application provide a VR device, which includes a processor and a memory. The memory stores at least one instruction, which is loaded and executed by the processor to implement the interface interaction method of the VR system as described above.

[0016] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the interface interaction method of the VR system as described above.

[0017] On the other hand, embodiments of this application provide a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the interface interaction method of the VR system provided above.

[0018] In this embodiment, in addition to displaying the 2D application icons of installed applications in the application list window, a 3D system taskbar is also provided, displaying 3D application icons. The VR device displays the application list window and the 3D system taskbar within the first field of view accessible to the virtual control object in the VR space. Furthermore, by setting editing controls, the 3D system taskbar can be edited. In response to interactive operations on the first 2D application icon in the application list window, a corresponding first 3D application icon is added to the 3D system taskbar. In response to triggering operations on either the first 2D application icon in the application list window or the first 3D application icon in the 3D system taskbar, the VR application interface is displayed within the second field of view in the VR space. Clearly, by setting the 3D system taskbar and 3D application icons within the first field of view, the realism of control in the VR space is improved; by displaying the VR application interface within the second field of view, the application interface display effect is optimized; and by setting an editable 3D system taskbar, the user's personalized settings for the 3D system taskbar are satisfied, improving the user experience. Attached Figure Description

[0019] Figure 1 A schematic diagram of a VR system provided in an exemplary embodiment of this application is shown;

[0020] Figure 2 This is a flowchart of a user interface interaction method for a VR system provided in an exemplary embodiment of this application;

[0021] Figure 3 This is a schematic diagram of a VR space provided in an exemplary embodiment of this application;

[0022] Figure 4 This is a schematic diagram illustrating the addition of a first 3D application icon according to an exemplary embodiment of this application;

[0023] Figure 5 This is a flowchart of a VR system interface interaction method provided in another exemplary embodiment of this application;

[0024] Figure 6 This is a schematic diagram illustrating the addition of a first 3D application icon provided in another exemplary embodiment of this application;

[0025] Figure 7 This is a schematic diagram of a sliding switch for a second 3D application icon provided in an exemplary embodiment of this application;

[0026] Figure 8 This is a flowchart of a VR system interface interaction method provided in another exemplary embodiment of this application;

[0027] Figure 9This is a schematic diagram of a gesture-based interface interaction provided in an exemplary embodiment of this application;

[0028] Figure 10 This is a flowchart of a VR system interface interaction method provided in another exemplary embodiment of this application;

[0029] Figure 11 This is a flowchart of a VR system interface interaction method provided in another exemplary embodiment of this application;

[0030] Figure 12 This is a schematic diagram of a control-triggered interface interaction provided in an exemplary embodiment of this application;

[0031] Figure 13 This is a flowchart of a VR system interface interaction method provided in another exemplary embodiment of this application;

[0032] Figure 14 This is a schematic diagram illustrating the deletion of a second 3D application icon provided in an exemplary embodiment of this application;

[0033] Figure 15 This is a schematic diagram of calling up the 3D system taskbar provided in an exemplary embodiment of this application;

[0034] Figure 16 This is a structural block diagram of the interface interaction device of a VR system provided in an exemplary embodiment of this application;

[0035] Figure 17 A schematic diagram of the structure of a VR device provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0037] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0038] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0039] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0040] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0041] The solutions provided in this application relate to technologies such as artificial intelligence and virtual reality, and are specifically illustrated through the following embodiments:

[0042] Please refer to Figure 1 The illustration shows a schematic diagram of a VR system provided in an exemplary embodiment of this application, which may include: a VR device 110, a VR host 120, and an input device 130.

[0043] VR device 110 is a display device worn on a user's head to display images. Optionally, VR device 110 typically includes a wearing part and a display part. The wearing part includes temples for wearing VR device 110 on the user's head and an elastic band. The display part includes a left-eye display screen and a right-eye display screen. VR device 110 can display different images on the left-eye display screen and the right-eye display screen, thereby simulating a three-dimensional virtual environment for the user.

[0044] Optionally, the VR device 110 is equipped with motion sensors to capture the user's head movements, so that the VR host 120 can change the display screen in the VR device 110.

[0045] VR device 110 is electrically connected to VR host 120 via a flexible circuit board, hardware interface, data cable, or wireless network.

[0046] The VR host 120 is used to model a three-dimensional virtual environment, generate a three-dimensional display screen corresponding to the three-dimensional virtual environment, and generate virtual objects in the three-dimensional virtual environment. Of course, the VR host 120 can also model a two-dimensional virtual environment, generate a two-dimensional display screen corresponding to the two-dimensional virtual environment, and generate virtual objects in the two-dimensional virtual environment; or, the VR host 120 can model a three-dimensional virtual environment, generate a two-dimensional display screen corresponding to the three-dimensional virtual environment according to the user's viewpoint, and generate a two-dimensional projection screen of virtual objects in the three-dimensional virtual environment, etc. This embodiment does not limit this.

[0047] Optionally, the VR host 120 can be integrated inside the VR device 110, or it can be integrated into other devices different from the VR device 110. This embodiment does not limit this. These other devices can be desktop computers or servers, etc., and this embodiment does not limit this either.

[0048] The VR host 120 receives input signals from the input device 130 and generates the display image of the VR device 110 based on the input signals. The VR host 120 is typically implemented by electronic devices such as a processor, memory, and image virtual reality host mounted on a circuit board. Optionally, the VR host 120 also includes an image acquisition device for capturing the user's head movements and changing the display image in the VR device 110 according to the user's head movements.

[0049] VR host 120 is connected to input device 130 via cable, Bluetooth or wireless network.

[0050] The input device 130 may be at least one of the following input peripherals: motion-sensing gloves, motion-sensing handles, remote controls, treadmills, mice, keyboards, and eye-focusing devices.

[0051] Optionally, the input device 130 can also be integrated into the VR device 110. Optionally, the VR device 110 is provided with a camera component for capturing the user's hand gestures and generating corresponding command signals based on the hand gestures, so that the VR host 120 generates the display screen of the VR device 110 according to the command signals.

[0052] It should be noted that, in this embodiment of the application, the VR host 120 and the input device 130 are both integrated inside the VR device 110 as an example.

[0053] Please refer to Figure 2 This document illustrates a flowchart of a user interface interaction method for a VR system provided in an exemplary embodiment of this application. This embodiment uses the method applied to a VR device as an example for illustration. The method includes the following steps:

[0054] Step 201: Display an application list window and a 3D system taskbar in the VR system interface. The application list window displays 2D application icons of installed applications. The 3D system taskbar is displayed in 3D form and displays 3D application icons. The application list window and the 3D system taskbar are located in the first field of view of the VR space, and the virtual manipulation objects in the VR space can touch the objects displayed in the first field of view.

[0055] Optionally, VR space is a three-dimensional virtual environment that can display both three-dimensional and two-dimensional images.

[0056] Optionally, the application list window displays 2D application icons of installed applications. In response to the selection of a 2D application icon in the application list window, the VR device launches the application corresponding to that 2D application icon.

[0057] Optionally, when an application is running, the application list window is hidden. If the user needs to run another application, the current application needs to be exited, and the VR device will then redisplay the application list window. In response to the user's selection of the 2D application icon of another application in the application list window, the VR device will run the other application.

[0058] Optionally, the 3D system taskbar is displayed in 3D format and includes 3D application icons. The 3D system taskbar can also be referred to as a dock. In response to a selection of a 3D application icon in the 3D system taskbar, the VR device launches the application corresponding to that icon.

[0059] Optionally, when an application is running, in response to the user's selection of the 3D application icon corresponding to the non-running application in the 3D system taskbar, the VR device switches the currently running application to the background and runs the application corresponding to the 3D application icon.

[0060] Optionally, 3D application icons differ from 2D application icons. 3D application icons are displayed in a three-dimensional form in VR space, and based on the volume of the 3D application icon, the virtual control object can grasp the 3D application icon in VR space.

[0061] In one possible implementation, to distinguish between application icons displayed in 2D and 3D, the VR device displays the application list window and the 3D system taskbar in separate sections.

[0062] Optionally, to enhance the realism of operations in VR space, the VR device divides the VR space based on the user's location and distance from the user, into a first field of view and a second field of view. The first field of view can be called the near field of view, while the second field of view can include the mid field of view and the far field of view. Furthermore, the user can touch objects displayed in the first field of view by controlling virtual objects in the VR space.

[0063] In one possible implementation, considering that the 3D system taskbar usually displays 3D application icons corresponding to applications that users use frequently, in order to improve the convenience of the 3D system taskbar, the VR device displays the 3D system taskbar within the first field of view, thereby making it easier for users to select 3D application icons.

[0064] In one possible implementation, in order to fully display the 2D application icons in the application list window to the user and facilitate the user's selection of the 2D application icons in the application list window, the VR device displays the application list window at the boundary of the first field of view and displays the application list window directly above the 3D system taskbar, thereby both facilitating the user's selection of 2D application icons and ensuring that the 2D application icons in the application list window are fully displayed.

[0065] Indicative, such as Figure 3 As shown, the VR device uses the user's location as a reference, setting the area within 0.7 meters of the user as the near field of view, also known as the first field of view, the area between 0.7 meters and 2 meters of the user as the mid field of view, and the area more than 2 meters away from the user as the far field of view, also known as the second field of view. Thus, the VR device displays the 3D system taskbar 301 0.7 meters directly in front of the user within the first field of view, and displays the application list window 302 tilted above the 3D system taskbar 301.

[0066] Step 202: In response to the trigger operation of the editing control, enter the taskbar editing mode. In the taskbar editing mode, the 3D application icons in the 3D system taskbar are in an editable state.

[0067] In one possible implementation, since the 3D system taskbar typically displays 3D application icons corresponding to applications that users use frequently, VR devices are equipped with editing controls in the VR space to facilitate users' personalization of the 3D application icons displayed in the 3D system taskbar.

[0068] Optionally, the editing controls can be displayed between the 3D system taskbar and the application list window, or they can be displayed on the side of the 3D system taskbar. This application embodiment does not limit this.

[0069] In one possible implementation, in response to a triggering operation of the editing control, the VR device enters the taskbar editing mode. In the taskbar editing mode, the 3D application icons in the 3D system taskbar are editable, allowing users to add or remove 3D application icons according to their needs.

[0070] Indicative, such as Figure 4 As shown, in response to the triggering operation of the editing control 401, the VR device enters the taskbar editing mode, and the 3D application icons in the 3D system taskbar 402 are in an editable state.

[0071] Step 203: In response to the interaction with the first 2D application icon in the application list window, add a first 3D application icon to the 3D system taskbar. The first 2D application icon and the first 3D application icon correspond to the same application.

[0072] In one possible implementation, since the application list window displays 2D application icons for all installed applications, users need to perform a search operation in the application list window before launching the application. Therefore, in order to reduce the user's need to search for 2D application icons in the application list window before launching the application, the user can add the 3D application icon of the application corresponding to the 2D application icon to the 3D system taskbar, so as to directly launch the corresponding application based on the selection operation of the 3D application icon.

[0073] In one possible implementation, upon receiving an interactive operation from a user on a first 2D application icon in the application list window, the VR device displays a first 3D application icon based on the application corresponding to the first 2D application icon, and adds the first 3D application icon to the 3D system taskbar.

[0074] Optionally, the interactive operation can be a pre-set gesture operation, or a user's click operation, drag operation, etc. based on a virtual control object. This application embodiment does not limit this.

[0075] Indicative, such as Figure 4As shown, in response to the interactive operation of the first 2D application icon 404 in the application list window 403, the VR device adds the corresponding first 3D application icon 405 to the 3D system taskbar 402.

[0076] Step 204: In response to a trigger operation on the first 2D application icon in the application list window, or on the first 3D application icon in the 3D system taskbar, display the VR application interface within the second field of view in the VR space. The second field of view is farther than the first field of view.

[0077] In one possible implementation, upon receiving a trigger operation on the first 2D application icon in the application list window, the VR device launches the application corresponding to the first 2D application icon and displays the VR application interface within the second field of view of the VR space.

[0078] In one possible implementation, upon receiving a trigger operation on the first 3D application icon in the 3D system taskbar, the VR device launches the application corresponding to the first 3D application icon and displays the VR application interface within the second field of view of the VR space.

[0079] Optionally, the second field of view may include a mid-field field of view and a long-field field of view.

[0080] In summary, in this embodiment, in addition to displaying the 2D application icons of installed applications in the application list window, a 3D system taskbar is also provided, displaying 3D application icons. The VR device displays the application list window and the 3D system taskbar within the first field of view accessible to the virtual control object in the VR space. Furthermore, by setting editing controls, the 3D system taskbar can be edited. In response to interactive operations on the first 2D application icon in the application list window, a corresponding first 3D application icon is added to the 3D system taskbar. In response to triggering operations on either the first 2D application icon in the application list window or the first 3D application icon in the 3D system taskbar, the VR application interface is displayed within the second field of view in the VR space. Clearly, by setting the 3D system taskbar and 3D application icons within the first field of view, the realism of operations in the VR space is improved; by displaying the VR application interface within the second field of view, the application interface display effect is optimized; and by setting an editable 3D system taskbar, the user's personalized settings for the 3D system taskbar are satisfied, improving the user experience.

[0081] In one possible implementation, in order to display the application information corresponding to the selected 2D application icon to the user in the application list window, the VR device is provided with an icon display bar and an auxiliary panel in the application list window. The icon display bar is used to display 2D application icons, and the auxiliary panel is used to display the application information corresponding to the selected 2D application icon in the icon display bar.

[0082] Please refer to Figure 5 This document illustrates a flowchart of a VR system interface interaction method provided in another exemplary embodiment of this application. This embodiment uses the method applied to a VR device as an example for illustration. The method includes the following steps:

[0083] Step 501: Display an application list window and a 3D system taskbar in the VR system interface. The application list window displays 2D application icons of installed applications. The 3D system taskbar is displayed in 3D form and displays 3D application icons. The application list window and the 3D system taskbar are located in the first field of view of the VR space, and the virtual control objects in the VR space can touch the objects displayed in the first field of view.

[0084] Step 502: In response to the trigger operation of the editing control, enter the taskbar editing mode. In the taskbar editing mode, the 3D application icons in the 3D system taskbar are in an editable state.

[0085] The specific implementation of steps 501 and 502 can be found in steps 201 and 202, and will not be repeated here.

[0086] Step 503: In response to the selection operation of the first 2D application icon in the application list window, the first 3D application icon corresponding to the first 2D application icon is displayed in the auxiliary panel.

[0087] In one possible implementation, in order to pre-display the first 3D application icon corresponding to the first 2D application icon to the user before adding the first 3D application icon to the 3D system taskbar, the VR device sets up an icon display bar and an auxiliary panel in the application list window, thereby displaying the 2D application icons of the installed applications in the icon display bar and displaying the first 3D application icon to the user through the auxiliary panel.

[0088] In one possible implementation, to facilitate users in selecting the first 2D application icon in the icon display bar, the VR device adds an application category bar to the application list window. By setting an application category control in the application category bar, the 2D application icons in the icon display bar are displayed in categories.

[0089] In one possible implementation, upon receiving a user's selection operation on the first 2D application icon in the icon display bar of the application list window, the VR device displays the first 3D application icon corresponding to the first 2D application icon in the auxiliary panel.

[0090] Indicative, such as Figure 6 As shown, the VR device sets up an icon display bar 601, an auxiliary panel 602, and an application category bar 603 in the application list window 600. In response to the user's selection operation of the first 2D application icon 604 in the icon display bar 601, the VR device displays the first 3D application icon 605 corresponding to the first 2D application icon 604 in the auxiliary panel 602.

[0091] In one possible implementation, since displaying the first 3D application icon is a pre-operation of adding the first 3D application icon to the 3D system taskbar, the VR device displays the first 3D application icon on the auxiliary panel only when the 3D system taskbar does not contain the first 3D application icon corresponding to the first 2D application icon, in response to the user's selection operation of the first 2D application icon in the application list window.

[0092] In one possible implementation, when the 3D system taskbar contains a first 3D application icon corresponding to a first 2D application icon, in response to the user's selection operation of the first 2D application icon in the application list window, the VR device displays a blank auxiliary panel, that is, in the blank state, the auxiliary panel does not display the first application icon.

[0093] Optionally, in the blank state, the auxiliary panel can display the application information corresponding to the first 2D application icon. This application information may include the application's download time, update information, the time of the most recent launch, etc. This application embodiment does not limit this.

[0094] Step 504: In response to the operation of adding the first 3D application icon, add the first 3D application icon to the 3D system taskbar.

[0095] Furthermore, in response to the user's action of adding the first 3D application icon, the VR device adds the first 3D application icon to the 3D system taskbar.

[0096] Optionally, the addition operation can be implemented by setting up an added control or by setting up a corresponding gesture operation. This application embodiment does not limit this.

[0097] Indicative, such as Figure 6 As shown, when the user adds the first 3D application icon 605, the VR device adds the first 3D application icon 605 to the 3D system taskbar 606.

[0098] In one possible implementation, in order to fully display each 3D application icon in the 3D system taskbar, the VR device sets the 3D system taskbar to a variable length form while keeping the size of the 3D application icons constant, and sets an upper limit on the number of 3D system taskbar icons.

[0099] In one possible implementation, if the number of second 3D application icons in the 3D system taskbar has not reached the upper limit, in response to the addition operation of the first 3D application icon, the VR device extends the 3D system taskbar and adds the first 3D application icon.

[0100] Indicative, such as Figure 6 As shown, the VR device sets the maximum number of 3D system taskbars 606 to 5. When 4 second 3D application icons are displayed in the 3D system taskbar 606, in response to the addition operation of the first 3D application icon 605, the VR device extends the 3D system taskbar 606 and adds the first 3D application icon 605.

[0101] In one possible implementation, when the number of second 3D application icons in the 3D system taskbar reaches the maximum limit, the VR device sets the 3D system taskbar to a sliding state. In the sliding state, the 3D application icons displayed in the 3D system taskbar can be switched by sliding. In response to the addition operation of the first 3D application icon, the VR device switches the second 3D application icon by sliding and adds the first 3D application icon.

[0102] Indicative, such as Figure 7 As shown, the VR device sets the maximum number of 3D system taskbars 701 to 4. When 4 second 3D application icons are displayed in the 3D system taskbar 701, in response to the addition operation of the first 3D application icon 702, the VR device slides to switch the second 3D application icons in the 3D system taskbar 701 and adds the first 3D application icon 702.

[0103] In the above embodiments, by setting an icon display bar and an auxiliary panel in the application list window, the first 3D application icon can be displayed preferentially through the auxiliary panel before it is added to the 3D system taskbar, providing users with preparation time to add the first 3D application icon and increasing the user-friendly settings in the 3D system taskbar editing process.

[0104] Furthermore, the 3D system taskbar was set to an extendable mode with a maximum number of options, improving the display efficiency of the 3D system taskbar and optimizing the VR system interface.

[0105] In one possible implementation, to enhance the realism of user interaction within the VR space, the VR device allows users to edit 3D application icons in the 3D system taskbar using gesture controls.

[0106] Please refer to Figure 8 This document illustrates a flowchart of a VR system interface interaction method provided in another exemplary embodiment of this application. This embodiment uses the method applied to a VR device as an example for illustration. The method includes the following steps:

[0107] Step 801: Display an application list window and a 3D system taskbar in the VR system interface. The application list window displays 2D application icons of installed applications. The 3D system taskbar is displayed in 3D form and displays 3D application icons. The application list window and the 3D system taskbar are located in the first field of view of the VR space, and the virtual manipulation objects in the VR space can touch the objects displayed in the first field of view.

[0108] Step 802: In response to the trigger operation of the editing control, enter the taskbar editing mode. In the taskbar editing mode, the 3D application icons in the 3D system taskbar are in an editable state.

[0109] Step 803: In response to the selection operation of the first 2D application icon in the application list window, the first 3D application icon corresponding to the first 2D application icon is displayed in the auxiliary panel.

[0110] The specific implementation of steps 801 to 803 can be found in steps 501 to 503, and will not be repeated here in this embodiment.

[0111] Step 804: When the first 3D application icon is within the reach of the virtual control object, in response to the selection command, control the virtual control object to select the first 3D application icon.

[0112] Optionally, the VR device can set corresponding gesture operations for different input commands. The VR device can then detect the user's gesture operations through the camera component and generate corresponding input commands based on the detected user gesture operations to control the virtual control object to perform corresponding operations.

[0113] Optionally, the VR device provides a cube selection display frame for the first 3D application icon, which enhances the stereoscopic display effect of the first 3D application icon after it is selected.

[0114] Optionally, the VR device binds a drag event to the first 3D application icon. When the VR device receives a drag operation on the first 3D application icon from the virtual control object, the first 3D application icon can move in the VR space following the virtual control object.

[0115] Optionally, the VR device binds a collision event to the first 3D application icon. When the virtual control object receives a collision operation on the first 3D application icon, the first 3D application icon is selected by the virtual control object.

[0116] Optionally, the VR device binds a rigid body property to the first 3D application icon, so that the first 3D application icon does not deform when it is displaced.

[0117] In one possible implementation, the VR device performs collision detection on the first 3D application icon in the auxiliary panel. When a collision is detected between the first 3D application icon and the virtual control object, i.e., the first 3D application icon is within the reach of the virtual control object, the VR device controls the virtual control object to select the first 3D application icon in response to a selection command issued by the user.

[0118] In one possible implementation, in order to simulate a user's selection and grasping action of a real object in a real environment, the VR device binds the selection command to the corresponding grasping gesture. When the user's grasping gesture in the real environment is recognized, the VR device triggers the selection command, thereby controlling the virtual object to select the first 3D application icon.

[0119] Indicative, such as Figure 9 As shown, after receiving a user's selection operation on the first 2D application icon 904 in the icon display bar 902 of the application list window 901, the VR device displays the corresponding first 3D application icon 905 in the auxiliary panel 903. When the VR device detects that the first 3D application icon 905 is within the reach range of the virtual control object, it adds a stereoscopic display effect to the first 3D application icon 905. Thus, when the VR device recognizes the user's grasping gesture in the real environment, it triggers a selection command and controls the virtual control object to select the first 3D application icon 905.

[0120] Step 805: When the virtual control object moves, control the first 3D application icon to follow the virtual control object.

[0121] In one possible implementation, in order to simulate the effect of grasping and moving a real object in a real environment, when a user's movement operation is detected, the VR device controls the virtual control object to perform the corresponding movement operation, thereby controlling the first 3D application icon to move along with the virtual control object.

[0122] Indicative, such as Figure 9As shown, based on the user's movement operation, the VR device controls the virtual control object to move the first 3D application icon 905 to the 3D system taskbar 906, and further adjusts the position of the first 3D application icon 905 in the 3D system taskbar 906 based on the user's movement operation.

[0123] Step 806: If the first 3D application icon is located in the 3D system taskbar, add the first 3D application icon to the 3D system taskbar in response to the placement command.

[0124] Optionally, the VR device binds a collision event to the 3D system taskbar. If a collision is detected between the first 3D application icon and the 3D system taskbar, the first 3D application icon is determined to be located in the 3D system taskbar.

[0125] In one possible implementation, the VR device associates the placement command with adding a first 3D application icon to the 3D system taskbar, so that when the first 3D application icon is detected to be in the 3D system taskbar, the VR device adds the first 3D application icon to the 3D system taskbar in response to the placement command.

[0126] In one possible implementation, in order to simulate the placement of a real object in a real environment, when the first 3D application icon is located in the 3D system taskbar, the VR device monitors the user's gesture operations. When the user's release gesture is recognized in the real environment, the VR device triggers a placement command, controls the virtual object to perform the gesture release operation, and adds the first 3D application icon to the 3D system taskbar. Furthermore, the VR device exits the taskbar editing mode and cancels the stereoscopic display effect of the first 3D application icon.

[0127] Optionally, after controlling the virtual object to perform a gesture release operation, in order to further simulate the release effect on real objects in a real environment, the VR device controls the first 3D application icon to be added to the 3D system taskbar by simulating the process of free fall.

[0128] In one possible implementation, in order to ensure the orderly placement of each 3D application icon in the 3D system taskbar, the VR device sets an icon placement position for each 3D application icon in the 3D system taskbar. Thus, when the first 3D application icon moves to the 3D system taskbar, the VR device displays the icon placement position in the 3D system taskbar based on the icon position of the first 3D application icon. Furthermore, in response to a placement command, the VR device adds the first 3D application icon to the icon placement position in the 3D system taskbar.

[0129] In one possible implementation, when the icon position of the first 3D application icon is between the icon positions of the second 3D application icons in the 3D system taskbar, the VR device will shift the second 3D application icons that are located after the icon position of the first 3D application icon in the 3D system taskbar one position to the right, thereby displaying the icon placement position in the 3D system taskbar based on the icon position of the first 3D application icon.

[0130] Indicative, such as Figure 9 As shown, when the first 3D application icon 905 is detected to be located in the 3D system taskbar 906, the VR device displays the icon placement position in the 3D system taskbar 906 based on the icon position of the first 3D application icon 905. Furthermore, when the user's release gesture is recognized in the real environment, the VR device triggers a placement command, controls the virtual control object to perform a gesture release operation, adds the first 3D application icon 905 to the 3D system taskbar 906, exits the taskbar editing mode, and cancels the stereoscopic display effect of the first 3D application icon 905.

[0131] In the above embodiments, by setting selection and placement instructions during the process of adding the first 3D application icon to the 3D system taskbar, and generating instructions based on the user's gesture operations in the recognized real environment, the grabbing and placement effect of real objects in the real environment is simulated, thereby improving the realism of the user's operation in the VR space.

[0132] Please refer to Figure 10 The diagram illustrates a flowchart of a user interface interaction method for a VR system provided in another exemplary embodiment of this application.

[0133] Step 1001, Begin.

[0134] Step 1002: In response to the trigger operation on the editing control, enter the taskbar editing mode.

[0135] In response to a triggering operation on the editing control, the VR device enters the taskbar editing mode, making the 3D application icons in the 3D system taskbar editable.

[0136] Step 1003: In response to the selection operation of the first 2D application icon in the application list window, the first 3D application icon corresponding to the first 2D application icon is displayed in the auxiliary panel.

[0137] Step 1004: Bind drag events, collision events, and rigid body properties to the first 3D application icon.

[0138] The VR device binds drag events, collision events, and rigid body properties to the first 3D application icon displayed in the auxiliary panel.

[0139] Step 1005: Recognize the user's grabbing gesture in the real environment and generate a selection command.

[0140] VR devices recognize users' grasping gestures in the real environment and generate selection commands accordingly.

[0141] Step 1006: In response to the selection command, add a stereoscopic display effect to the first 3D application icon.

[0142] In response to the selection command, the VR device adds a stereoscopic display effect to the first 3D application icon.

[0143] Step 1007: Control the first 3D application icon to follow the movement of the virtual control object.

[0144] When the virtual object moves, the VR device controls the first 3D application icon to follow the virtual object's movement.

[0145] Step 1008: Determine whether the first 3D application icon is located in the 3D system taskbar.

[0146] The VR device determines whether the first 3D application icon is located in the 3D system taskbar. If the first 3D application icon is located in the 3D system taskbar, proceed to step 1009; if the first 3D application icon is not located in the 3D system taskbar, return to step 1007.

[0147] Step 1009: Determine whether the first 3D application icon is located on the far right of the 3D system taskbar.

[0148] The VR device determines whether the first 3D application icon is located on the far right of the 3D system taskbar. If the first 3D application icon is located on the far right of the 3D system taskbar, proceed to step 1010; if the first 3D application icon is not located on the far right of the 3D system taskbar, proceed to step 1011.

[0149] Step 1010: Display an icon placement slot on the far right of the 3D system taskbar.

[0150] When the first 3D application icon is located on the far right of the 3D system taskbar, the VR device displays an icon placement slot on the far right of the 3D system taskbar.

[0151] Step 1011: Adjust the icon position of the 3D application icon in the 3D system taskbar.

[0152] If the first 3D application icon is not located on the far right of the 3D system taskbar, the VR device adjusts the icon position of the 3D application icon in the 3D system taskbar, moves the second 3D application icon, which is located to the right of the first 3D application icon, one icon placement position to the right, and displays an icon placement position in the location of the first 3D application icon.

[0153] Step 1012: Recognize the user's release gesture in the real environment and generate a placement command.

[0154] VR devices recognize users' release gestures in the real environment and generate placement instructions accordingly.

[0155] Step 1013: Add the first 3D application icon to the icon placement position in the 3D system taskbar.

[0156] VR devices add the first 3D application icon to the icon placement position of the 3D system taskbar.

[0157] Step 1014: Exit taskbar editing mode.

[0158] Step 1015: Cancel the stereoscopic display effect of the first 3D application icon.

[0159] Furthermore, VR devices eliminate the stereoscopic display effect of the first 3D application icon.

[0160] Step 1016, End.

[0161] In one possible implementation, in order to prevent deviations in recognizing user gestures in the real environment, the VR device adds a first 3D application icon to the 3D system taskbar by setting up an add control.

[0162] Please refer to Figure 11 This document illustrates a flowchart of a VR system interface interaction method provided in another exemplary embodiment of this application. This embodiment uses the method applied to a VR device as an example for illustration. The method includes the following steps:

[0163] Step 1101: Display an application list window and a 3D system taskbar in the VR system interface. The application list window displays 2D application icons of installed applications. The 3D system taskbar is displayed in 3D form and displays 3D application icons. The application list window and the 3D system taskbar are located in the first field of view of the VR space, and the virtual manipulation objects in the VR space can touch the objects displayed in the first field of view.

[0164] Step 1102: In response to the trigger operation of the editing control, enter the taskbar editing mode. In the taskbar editing mode, the 3D application icons in the 3D system taskbar are in an editable state.

[0165] Step 1103: In response to the selection operation of the first 2D application icon in the application list window, the first 3D application icon corresponding to the first 2D application icon is displayed in the auxiliary panel.

[0166] The specific implementation of steps 1101 to 1103 can be found in steps 501 to 503, and will not be repeated here in this embodiment.

[0167] Step 1104: In response to the triggering operation of adding a control corresponding to the first 3D application icon, display the first 3D application icon at the candidate icon position in the 3D system taskbar.

[0168] In one possible implementation, the VR device has a corresponding add control for the first 3D application icon. In response to the user's control of the virtual control object to trigger the add control corresponding to the first 3D application icon, the VR device adds a candidate icon position in the 3D system taskbar, thereby displaying the first 3D application icon at the candidate icon position.

[0169] Indicative, such as Figure 12 As shown, after receiving the user's selection operation on the first 2D application icon 1204 in the icon display bar 1202 of the application list window 1201, the VR device displays the corresponding first 3D application icon 1205 in the auxiliary panel 1203. In response to the user's trigger operation of adding control 1206 to the first 3D application icon 1205 by controlling the virtual control object, the VR device displays the first 3D application icon 1205 in the candidate icon position of the 3D system taskbar 1207.

[0170] In one possible implementation, to facilitate the cancellation of the operation of adding the first 3D application icon, the VR device changes the addition of the control to deletion of the control after the addition of the control is triggered.

[0171] In one possible implementation, in response to a user-controlled virtual manipulation object triggering a delete control corresponding to the first 3D application icon, the VR device stops displaying the first 3D application icon in the 3D system taskbar and re-displays the first 3D application icon on the auxiliary panel.

[0172] Indicative, such as Figure 12 As shown, in response to the user-controlled virtual manipulation object triggering the delete control 1208 corresponding to the first 3D application icon 1205, the VR device stops displaying the first 3D application icon 1205 in the 3D system taskbar 1207 and re-displays the first 3D application icon 1205 on the auxiliary panel 1203.

[0173] Step 1105: In response to the trigger operation of the moving control corresponding to the first 3D application icon, adjust the position of the first 3D application icon in the candidate icon position in the 3D system taskbar.

[0174] In one possible implementation, to facilitate users in adjusting the position of the first 3D application icon in the 3D system taskbar, the VR device is equipped with a movement control corresponding to the first 3D application icon. In response to the user's control of the virtual control object to trigger the movement control corresponding to the first 3D application icon, the VR device adjusts the position of the first 3D application icon in the candidate icon position in the 3D system taskbar.

[0175] Optionally, the movement control can be divided into a left-moving control and a right-moving control, which are used to adjust the position of the first 3D application icon in the candidate icon in the 3D system taskbar.

[0176] In one possible implementation, if the first 3D application icon is detected to be located on the far right of the 3D system taskbar, the VR device will gray out the rightward control; if the first 3D application icon is detected to be located on the far left of the 3D system taskbar, the VR device will gray out the leftward control.

[0177] Indicative, such as Figure 12 As shown, in response to the user's control of the virtual manipulation object triggering the left-shift control 1209 corresponding to the first 3D application icon 1205, the VR device adjusts the position of the first 3D application icon 1205 in the 3D system taskbar 1207, shifting the position of the candidate icon one position to the left.

[0178] Step 1106: In response to the triggering operation of the edit exit control, add the first 3D application icon to the candidate icon position of the 3D system taskbar.

[0179] In one possible implementation, after the user determines the position of the candidate icon corresponding to the first 3D application icon, a command to exit editing can be issued to the VR device, thereby exiting the taskbar editing mode.

[0180] In one possible implementation, in response to a user-controlled virtual manipulation object triggering an edit exit control, the VR device adds a first 3D application icon to the candidate icon position in the 3D system taskbar and exits the taskbar editing mode.

[0181] Indicative, such as Figure 12 As shown, in response to the user's control of the virtual manipulation object triggering the edit exit control 1210, the VR device adds the first 3D application icon 1205 to the candidate icon position of the 3D system taskbar 1207.

[0182] In one possible implementation, after the user controls the virtual control object to trigger the delete control corresponding to the first 3D application icon, in response to the user controlling the virtual control object to trigger the edit exit control, the VR device displays the first 3D application icon on the auxiliary panel and exits the taskbar editing mode.

[0183] Indicative, such as Figure 12 As shown, in response to the user's control of the virtual control object triggering the edit exit control 1210, the VR device displays the first 3D application icon 1205 on the auxiliary panel 1203 and exits the taskbar editing mode.

[0184] In the above embodiments, by setting an add control and a move control corresponding to the first 3D application icon, and by triggering the add control, the first 3D application icon is added. By triggering the move control, the position of the first 3D application icon in the 3D system taskbar is adjusted, thus avoiding deviations in the gesture recognition process of the VR device and improving the accuracy of adding the first 3D application icon to the 3D system taskbar.

[0185] Please refer to Figure 13 The diagram illustrates a flowchart of a user interface interaction method for a VR system provided in another exemplary embodiment of this application.

[0186] Step 1301, Begin.

[0187] Step 1302: In response to the trigger operation on the editing control, enter the taskbar editing mode.

[0188] In response to a triggering operation on the editing control, the VR device enters the taskbar editing mode, making the 3D application icons in the 3D system taskbar editable.

[0189] Step 1303: In response to the selection operation of the first 2D application icon in the application list window, the first 3D application icon corresponding to the first 2D application icon is displayed in the auxiliary panel.

[0190] Step 1304: Add candidate icon positions to the 3D system taskbar.

[0191] In response to the triggering operation of adding a control corresponding to the first 3D application icon, the VR device adds a candidate icon position in the 3D system taskbar.

[0192] Step 1305: Display the first 3D application icon at the candidate icon location.

[0193] Furthermore, the VR device displays the first 3D application icon at the candidate icon location.

[0194] Step 1306: Receive the trigger operation for the left shift control.

[0195] The VR device receives a trigger operation from the user controlling a virtual control object to move the control to the left.

[0196] Step 1307: Determine whether the first 3D application icon is located on the far left of the 3D system taskbar.

[0197] The VR device determines whether the first 3D application icon is located on the far left of the 3D system taskbar. If the first 3D application icon is located on the far left of the 3D system taskbar, proceed to step 1308; if the first 3D application icon is not located on the far left of the 3D system taskbar, proceed to step 1309.

[0198] Step 1308: Gray out the left shift control.

[0199] When the first 3D application icon is located on the far left of the 3D system taskbar, the VR device will gray out the left-moving control.

[0200] Step 1309: Shift the position of the candidate icon one position to the left.

[0201] If the first 3D application icon is not located on the far left of the 3D system taskbar, the VR device will shift the position of the candidate icon one position to the left.

[0202] Step 1310: Receive the trigger operation on the right-moving control.

[0203] The VR device receives a trigger operation from the user controlling a virtual control object to move the control to the right.

[0204] Step 1311: Determine whether the first 3D application icon is located on the far right of the 3D system taskbar.

[0205] The VR device determines whether the first 3D application icon is located on the far right of the 3D system taskbar. If the first 3D application icon is located on the far right of the 3D system taskbar, proceed to step 1312; if the first 3D application icon is not located on the far right of the 3D system taskbar, proceed to step 1313.

[0206] Step 1312: Gray out the right-shift control.

[0207] When the first 3D application icon is located on the far right of the 3D system taskbar, the VR device will gray out the right-moving control.

[0208] Step 1313: Shift the candidate icon position one position to the right.

[0209] If the first 3D application icon is not located on the far right of the 3D system taskbar, the VR device will shift the position of the candidate icon one position to the right.

[0210] Step 1314: In response to the triggering operation of the edit exit control, add the first 3D application icon to the candidate icon position of the 3D system taskbar.

[0211] After the user determines the candidate icon position, in response to the user's control of the virtual control object triggering the edit exit control, the VR device adds the first 3D application icon to the candidate icon position in the 3D system taskbar.

[0212] Step 1315: Exit taskbar editing mode.

[0213] Furthermore, the VR device exits the taskbar editing mode.

[0214] Step 1316, End.

[0215] In one possible implementation, in order to further optimize the display effect of the second 3D application icon in the 3D system taskbar, the VR device can support users to delete the second 3D application icon in the 3D system taskbar in taskbar editing mode.

[0216] In one possible implementation, in response to a trigger operation on the editing control, after the VR device enters the taskbar editing mode, upon receiving a deletion operation on the second 3D application icon, the VR device removes the second 3D application icon from the 3D system taskbar.

[0217] In one possible implementation, after entering the taskbar editing mode, upon receiving a selection operation on the second 3D application icon, the VR device displays a delete control. In response to the triggering operation of the delete control, the VR device removes the second 3D application icon from the 3D system taskbar and displays the second 3D application icon on the auxiliary panel.

[0218] Indicative, such as Figure 14 As shown, after entering the taskbar editing mode, when a selection operation is received on the second 3D application icon 1402, the VR device displays a delete control 1403. In response to the trigger operation of the delete control 1403, the VR device removes the second 3D application icon 1402 from the 3D system taskbar 1401 and displays the second 3D application icon 1402 in the auxiliary panel 1404.

[0219] In one possible implementation, when displaying the VR application interface, in order to improve the display effect of the VR application interface, the VR device hides the 3D system taskbar. At the same time, in order to improve the efficiency of users switching running applications, when displaying the VR application interface, the VR device supports calling up and displaying the 3D system taskbar. Thus, when receiving a user's trigger operation on the second 3D application icon, the VR device can switch the running application and display the VR application interface corresponding to the second 3D application icon.

[0220] In one possible implementation, in response to a trigger operation on a first 2D application icon in the application list window, or on a first 3D application icon in the 3D system taskbar, the VR device displays the VR application interface within the second field of view and hides the application list window and the 3D system taskbar. Furthermore, during the display of the VR application interface, in response to a taskbar call-out operation, the VR device displays the 3D system taskbar within the first field of view.

[0221] Optionally, the taskbar call-up operation can be an up-swipe gesture operation, or a trigger operation of the taskbar call-up control, etc., and this application embodiment does not limit it in this way.

[0222] Indicative, such as Figure 15 As shown, in response to a trigger operation on the first 3D application icon 1503 in the 3D system taskbar 1502, the VR device displays the VR application interface 1504 and hides the application list window 1501 and the 3D system taskbar 1502. Furthermore, during the display of the VR application interface 1504, in response to an upward swipe gesture operation, the VR device displays the 3D system taskbar 1502 within the first field of view.

[0223] Please refer to Figure 16 This illustration shows a structural block diagram of an interface interaction device for a VR system provided in an exemplary embodiment of this application. The device includes:

[0224] The first display module 1601 is used to display an application list window and a 3D system taskbar in the VR system interface. The application list window displays 2D application icons of installed applications. The 3D system taskbar is displayed in 3D form and displays 3D application icons. The application list window and the 3D system taskbar are located within the first field of view of the VR space, and the virtual manipulation object in the VR space can touch the objects displayed within the first field of view.

[0225] The editing module 1602 is used to enter the taskbar editing mode in response to the trigger operation of the editing control. In the taskbar editing mode, the 3D application icon in the 3D system taskbar is in an editable state.

[0226] The icon adding module 1603 is used to add a first 3D application icon to the 3D system taskbar in response to an interactive operation on the first 2D application icon in the application list window. The first 2D application icon and the first 3D application icon correspond to the same application.

[0227] The second display module 1604 is used to display the VR application interface in a second field of view in the VR space in response to a trigger operation on the first 2D application icon in the application list window or on the first 3D application icon in the 3D system taskbar. The second field of view is farther than the first field of view.

[0228] Optionally, the application list window includes an icon display bar and an auxiliary panel. The icon display bar is used to display the 2D application icons, and the auxiliary panel is used to display the application information corresponding to the selected 2D application icon in the icon display bar.

[0229] The icon adding module 1603 includes:

[0230] An icon display unit is configured to display the first 3D application icon corresponding to the first 2D application icon in the auxiliary panel in response to a selection operation of the first 2D application icon in the application list window.

[0231] An icon adding unit is used to add the first 3D application icon to the 3D system taskbar in response to the operation of adding the first 3D application icon.

[0232] Optionally, the icon adding unit is used for:

[0233] When the first 3D application icon is within the reach of the virtual control object, in response to the selection command, the virtual control object is controlled to select the first 3D application icon;

[0234] When the virtual control object moves, control the first 3D application icon to follow the movement of the virtual control object;

[0235] When the first 3D application icon is located in the 3D system taskbar, in response to the placement command, the first 3D application icon is added to the 3D system taskbar.

[0236] Optionally, the icon adding unit is also used for:

[0237] Upon recognizing a grasping gesture in the real environment, the selection command is triggered, and the virtual control object is controlled to select the first 3D application icon;

[0238] The icon adding unit is also used for:

[0239] Upon recognizing a release gesture in the real environment, the placement command is triggered, and the first 3D application icon is added to the 3D system taskbar.

[0240] Optionally, the icon adding unit is also used for:

[0241] When the first 3D application icon is moved to the 3D system taskbar, an icon placement position is displayed in the 3D system taskbar based on the icon position of the first 3D application icon.

[0242] In response to the placement instruction, the first 3D application icon is added to the icon placement position in the 3D system taskbar.

[0243] Optionally, the icon adding unit is used for:

[0244] In response to the triggering operation of adding a control corresponding to the first 3D application icon, the first 3D application icon is displayed at the candidate icon position in the 3D system taskbar;

[0245] In response to a trigger operation on the edit exit control, the first 3D application icon is added to the candidate icon position in the 3D system taskbar.

[0246] Optionally, the device further includes:

[0247] The adjustment module is used to adjust the position of the candidate icon in the 3D system taskbar of the first 3D application icon in response to a trigger operation on the movement control corresponding to the first 3D application icon.

[0248] Optionally, the add control becomes a delete control after being triggered;

[0249] The device further includes:

[0250] The third display module is used to stop displaying the first 3D application icon in the 3D system taskbar in response to the triggering operation of the delete control corresponding to the first 3D application icon.

[0251] Optionally, the length of the 3D system taskbar is variable;

[0252] Before adding the first 3D application icon to the 3D system taskbar, the device further includes:

[0253] An extension module is used to extend the 3D system taskbar when the number of second 3D application icons in the 3D system taskbar does not reach the upper limit.

[0254] Optionally, the device further includes:

[0255] The settings module is used to set the 3D system taskbar to a sliding state when the number of the second 3D application icons in the 3D system taskbar reaches the upper limit. In the sliding state, the 3D application icons displayed in the 3D system taskbar can be switched by sliding.

[0256] Optionally, the icon display unit is used for:

[0257] In response to the selection operation of the first 2D application icon in the application list window, if the first 3D application icon corresponding to the first 2D application icon is not included in the 3D system taskbar, the first 3D application icon is displayed in the auxiliary panel.

[0258] The device further includes:

[0259] The fourth display module is used to respond to the selection operation of the first 2D application icon in the application list window, and when the 3D system taskbar contains the first 3D application icon corresponding to the first 2D application icon, display the auxiliary panel in a blank state, wherein the first 3D application icon is not displayed in the auxiliary panel in the blank state.

[0260] Optionally, after entering the taskbar editing mode in response to a trigger operation on the editing control, the device further includes:

[0261] A removal module is configured to remove the second 3D application icon from the 3D system taskbar in response to a deletion operation of the second 3D application icon in the 3D system taskbar.

[0262] Optionally, the device further includes:

[0263] The fifth display module is used to hide the application list window and the 3D system taskbar in response to a trigger operation on the first 2D application icon in the application list window or on the first 3D application icon in the 3D system taskbar.

[0264] The sixth display module is used to display the 3D system taskbar within the first field of view of the VR space in response to the taskbar call-out operation.

[0265] In summary, in this embodiment, in addition to displaying the 2D application icons of installed applications in the application list window, a 3D system taskbar is also provided, displaying 3D application icons. The VR device displays the application list window and the 3D system taskbar within the first field of view accessible to the virtual control object in the VR space. Furthermore, by setting editing controls, the 3D system taskbar can be edited. In response to interactive operations on the first 2D application icon in the application list window, a corresponding first 3D application icon is added to the 3D system taskbar. In response to triggering operations on either the first 2D application icon in the application list window or the first 3D application icon in the 3D system taskbar, the VR application interface is displayed within the second field of view in the VR space. Clearly, by setting the 3D system taskbar and 3D application icons within the first field of view, the realism of operations in the VR space is improved; by displaying the VR application interface within the second field of view, the application interface display effect is optimized; and by setting an editable 3D system taskbar, the user's personalized settings for the 3D system taskbar are satisfied, improving the user experience.

[0266] It should be noted that the apparatus provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their implementation process can be found in the method embodiments, which will not be repeated here.

[0267] Please refer to Figure 17 This illustration shows a schematic diagram of the structure of a VR device provided in an exemplary embodiment of this application. The VR device 1700 can be implemented as a terminal or server as described in the above embodiments. Specifically, the VR device 1700 includes a Central Processing Unit (CPU) 1701, a system memory 1704 including a random access memory 1702 and a read-only memory 1703, and a system bus 1705 connecting the system memory 1704 and the CPU 1701. The VR device 1700 also includes a basic input / output system (I / O system) 1706 to facilitate information transfer between various devices within the computer, and a mass storage device 1707 for storing the operating system 1713, application programs 1714, and other program modules 1715.

[0268] The basic input / output system 1706 includes a display 1708 for displaying information and an input device 1709 for user input, such as a mouse or keyboard. Both the display 1708 and the input device 1709 are connected to the central processing unit 1701 via an input / output controller 1710 connected to the system bus 1705. The basic input / output system 1706 may also include the input / output controller 1710 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1710 also provides output to a display screen, printer, or other types of output devices.

[0269] The mass storage device 1707 is connected to the central processing unit 1701 via a mass storage controller (not shown) connected to the system bus 1705. The mass storage device 1707 and its associated computer-readable media provide non-volatile storage for the VR device 1700. That is, the mass storage device 1707 may include computer-readable media (not shown) such as a hard disk or drive.

[0270] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include random access memory (RAM), read-only memory (ROM), flash memory or other solid-state storage technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 1704 and mass storage device 1707 described above can be collectively referred to as memory.

[0271] The memory stores one or more programs, which are configured to be executed by one or more central processing units 1701. The one or more programs contain instructions for implementing the above methods. The central processing unit 1701 executes the one or more programs to implement the interface interaction method of the VR system provided in the above method embodiments.

[0272] According to various embodiments of this application, the VR device 1700 can also be connected to a remote computer on a network such as the Internet. That is, the VR device 1700 can be connected to a network 1712 via a network interface unit 1711 connected to the system bus 1705, or the network interface unit 1711 can be used to connect to other types of networks or remote computer systems (not shown).

[0273] This application also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the interface interaction method of the VR system provided in the above embodiments.

[0274] Optionally, the computer-readable storage medium may include ROM, RAM, solid-state drives (SSDs), or optical discs, etc. The RAM may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).

[0275] This application provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the interface interaction method of the VR system described in the above embodiments.

[0276] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0277] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for user interface interaction in a VR system, characterized in that, The method includes: The VR system interface displays an application list window and a 3D system taskbar. The application list window displays 2D application icons of installed applications, and the 3D system taskbar is displayed in 3D form and displays 3D application icons. The application list window and the 3D system taskbar are located in the first field of view of the VR space, and the virtual manipulation objects in the VR space can touch the objects displayed in the first field of view. In response to a trigger operation on the editing control, the system enters the taskbar editing mode, in which the 3D application icon in the 3D system taskbar is in an editable state; In response to an interactive operation on the first 2D application icon in the application list window, a first 3D application icon is added to the 3D system taskbar, wherein the first 2D application icon and the first 3D application icon correspond to the same application. In response to a trigger operation on the first 2D application icon in the application list window, or on the first 3D application icon in the 3D system taskbar, a VR application interface is displayed within a second field of view in the VR space, wherein the second field of view is farther than the first field of view.

2. The method according to claim 1, characterized in that, The application list window includes an icon display bar and an auxiliary panel. The icon display bar is used to display the 2D application icons, and the auxiliary panel is used to display the application information corresponding to the selected 2D application icon in the icon display bar. The step of adding a first 3D application icon to the 3D system taskbar in response to an interactive operation on the first 2D application icon in the application list window includes: In response to the selection operation of the first 2D application icon in the application list window, the first 3D application icon corresponding to the first 2D application icon is displayed in the auxiliary panel; In response to the operation of adding the first 3D application icon, the first 3D application icon is added to the 3D system taskbar.

3. The method according to claim 2, characterized in that, The step of adding the first 3D application icon to the 3D system taskbar in response to the operation of adding the first 3D application icon includes: When the first 3D application icon is within the reach of the virtual control object, in response to the selection command, the virtual control object is controlled to select the first 3D application icon; When the virtual control object moves, control the first 3D application icon to follow the movement of the virtual control object; When the first 3D application icon is located in the 3D system taskbar, in response to the placement command, the first 3D application icon is added to the 3D system taskbar.

4. The method according to claim 3, characterized in that, The step of responding to a selection command and controlling the virtual control object to select the first 3D application icon includes: Upon recognizing a grasping gesture in the real environment, the selection command is triggered, and the virtual control object is controlled to select the first 3D application icon; The step of adding the first 3D application icon to the 3D system taskbar in response to the placement command includes: Upon recognizing a release gesture in the real environment, the placement command is triggered, and the first 3D application icon is added to the 3D system taskbar.

5. The method according to claim 3, characterized in that, When the first 3D application icon is located in the 3D system taskbar, in response to a placement command, adding the first 3D application icon to the 3D system taskbar includes: When the first 3D application icon is moved to the 3D system taskbar, an icon placement position is displayed in the 3D system taskbar based on the icon position of the first 3D application icon. In response to the placement instruction, the first 3D application icon is added to the icon placement position in the 3D system taskbar.

6. The method according to claim 2, characterized in that, The step of adding the first 3D application icon to the 3D system taskbar in response to the operation of adding the first 3D application icon includes: In response to the triggering operation of adding a control corresponding to the first 3D application icon, the first 3D application icon is displayed at the candidate icon position in the 3D system taskbar; In response to a trigger operation on the edit exit control, the first 3D application icon is added to the candidate icon position in the 3D system taskbar.

7. The method according to claim 6, characterized in that, The method further includes: In response to a trigger operation on the movement control corresponding to the first 3D application icon, the position of the candidate icon of the first 3D application icon in the 3D system taskbar is adjusted.

8. The method according to claim 6, characterized in that, The add control is transformed into a delete control after being triggered; The method further includes: In response to the triggering operation of the delete control corresponding to the first 3D application icon, the display of the first 3D application icon in the 3D system taskbar is stopped.

9. The method according to claim 2, characterized in that, The length of the 3D system taskbar is variable. Before adding the first 3D application icon to the 3D system taskbar, the method further includes: If the number of second 3D application icons in the 3D system taskbar does not reach the maximum limit, the 3D system taskbar will be extended.

10. The method according to claim 9, characterized in that, The method further includes: When the number of the second 3D application icons in the 3D system taskbar reaches the upper limit, the 3D system taskbar is set to a sliding state. In the sliding state, the 3D application icons displayed in the 3D system taskbar can be switched by sliding.

11. The method according to claim 2, characterized in that, The step of displaying the first 3D application icon corresponding to the first 2D application icon in the auxiliary panel in response to the selection operation of the first 2D application icon in the application list window includes: In response to the selection operation of the first 2D application icon in the application list window, if the first 3D application icon corresponding to the first 2D application icon is not included in the 3D system taskbar, the first 3D application icon is displayed in the auxiliary panel. The method further includes: In response to the selection operation of the first 2D application icon in the application list window, if the first 3D application icon corresponding to the first 2D application icon is included in the 3D system taskbar, the auxiliary panel is displayed in a blank state, and the first 3D application icon is not displayed in the auxiliary panel in the blank state.

12. The method according to any one of claims 1 to 11, characterized in that, After entering taskbar editing mode in response to a trigger operation on the editing control, the method further includes: In response to the deletion operation of the second 3D application icon in the 3D system taskbar, the second 3D application icon is removed from the 3D system taskbar.

13. The method according to any one of claims 1 to 11, characterized in that, The method further includes: In response to a trigger operation on the first 2D application icon in the application list window, or on the first 3D application icon in the 3D system taskbar, the application list window and the 3D system taskbar are hidden. In response to a taskbar call-out operation, the 3D system taskbar is displayed within the first field of view in the VR space.

14. A user interface interaction device for a VR system, characterized in that, The device includes: The first display module is used to display an application list window and a 3D system taskbar in the VR system interface. The application list window displays 2D application icons of installed applications. The 3D system taskbar is displayed in 3D form and displays 3D application icons. The application list window and the 3D system taskbar are located within the first field of view of the VR space, and the virtual manipulation objects in the VR space can touch the objects displayed within the first field of view. The editing module is used to enter the taskbar editing mode in response to the trigger operation of the editing control. In the taskbar editing mode, the 3D application icon in the 3D system taskbar is in an editable state. An icon adding module is used to add a first 3D application icon to the 3D system taskbar in response to an interactive operation on the first 2D application icon in the application list window. The first 2D application icon and the first 3D application icon correspond to the same application. The second display module is used to display the VR application interface in a second field of view in the VR space in response to a trigger operation on the first 2D application icon in the application list window, or on the first 3D application icon in the 3D system taskbar. The second field of view is farther than the first field of view.

15. A VR device, characterized in that, The VR device includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to implement the interface interaction method of the VR system as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement the interface interaction method of the VR system as described in any one of claims 1 to 13.

17. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the interface interaction method of the VR system as described in any one of claims 1 to 13.

Images