An interaction method, device and equipment in a virtual reality scene and a storage medium

Abstract

Embodiments of the present disclosure provide an interaction method and device in a virtual reality scene, equipment and a storage medium. The method receives head control data acting in the virtual reality scene; determines a focus area based on the head control data and displays the focus area and a control point, the control point is initially located at a center point of the focus area, and the focus area includes a to-be-interacted object; receives hand gesture data generated in a hand operation area, the hand operation area has a displacement mapping relationship with the focus area; moves the control point to a target display position, the target display position being a mapping position of the hand gesture data in the focus area; and generates an interaction operation in response to the control point acting on the to-be-interacted object. The method can effectively solve the problem of interaction fatigue caused by single interaction on a body part; by first narrowing the interaction range to locate the focus area and then accurately selecting the object in the focus area through the control point, the risk of being unable to accurately select the interaction object is reduced, and the interaction accuracy is improved.

Description

Technical Field

[0001] This disclosure relates to the field of virtual reality technology, and in particular to an interaction method, apparatus, device, and storage medium in a virtual reality scene. Background Technology

[0002] Virtual reality (VR) technology is gradually being applied to people's lives. VR can use electronic devices to simulate and generate three-dimensional virtual reality scenes, and can be presented to users through virtual reality devices (such as virtual reality glasses, virtual reality headsets, etc.), providing an immersive experience in terms of vision, hearing, touch, or other senses.

[0003] Unlike two-dimensional interactions on mobile devices and desktop computers, interactions in virtual reality scenes often achieve a more three-dimensional input through non-contact interaction methods such as gestures, controllers, head movements, and eye movements. However, existing methods often only utilize one of these input methods within a single scene interface to achieve interaction.

[0004] Existing interaction methods suffer from the following problems: 1) Low accuracy: When the presented 2D interface is far from the player's virtual character, it is difficult to select a small button or control, affecting interaction accuracy; 2) Interaction fatigue: When using a single interaction method for an extended period in a scene (such as using a gamepad to control a progress bar to browse a virtual webpage), the corresponding body part needs to remain in a constant state or repeatedly perform an action, easily causing player fatigue. All of these problems negatively impact the player's immersive experience. Summary of the Invention

[0005] This disclosure provides an interaction method, device, equipment, and storage medium in a virtual reality scene to achieve combined interaction in a virtual reality scene and improve the convenience of interaction.

[0006] In a first aspect, embodiments of this disclosure provide an interaction method in a virtual reality scene, the interaction method in the virtual reality scene including:

[0007] Receive head control data acting in a virtual reality scene;

[0008] Based on the head control data, a region of interest is determined, and the region of interest and control points are displayed in the virtual reality scene. The control points are initially located at a set position in the region of interest, and the region of interest includes an object to be interacted with.

[0009] Receive hand posture data generated within the hand operation area, wherein the hand operation area and the area of ​​interest have a displacement mapping relationship;

[0010] Move the control point to the target display position, where the target display position is the mapping position of the hand posture data in the area of ​​interest;

[0011] The interactive operation generated when the control point acts on the object to be interacted with.

[0012] Secondly, embodiments of this disclosure also provide an interactive device for a virtual reality scene, the interactive device for the virtual reality scene comprising:

[0013] The first receiving module is used to receive head control data acting in the virtual reality scene;

[0014] The first display module is used to determine the area of ​​interest based on the head control data, and to display the area of ​​interest and control points in the virtual reality scene. The control points are initially located at a set position in the area of ​​interest, and the area of ​​interest includes an object to be interacted with.

[0015] The second receiving module is used to receive hand posture data generated in the hand operation area, wherein the hand operation area and the area of ​​interest have a displacement mapping relationship;

[0016] The second display module is used to move the control point to the target display position, where the target display position is the mapping position of the hand posture data in the area of ​​interest;

[0017] The first response module is used to respond to the interactive operation generated when the control point acts on the object to be interacted with.

[0018] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0019] One or more processors;

[0020] Storage device for storing one or more programs.

[0021] When the one or more programs are executed by the one or more processors, the one or more processors implement the interaction method in a virtual reality scene as described in any embodiment of the present invention.

[0022] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform an interaction method in a virtual reality scene as described in any embodiment of the present invention.

[0023] The technical solution of this disclosure, through an interaction method in a virtual reality scene, can receive head control data acting on the virtual reality scene; determine an area of ​​interest based on the head control data, and display the area of ​​interest and control points in the virtual reality scene. The control points are initially located at a set position in the area of ​​interest, and the area of ​​interest includes an object to be interacted with; receive hand posture data generated in a hand operation area, where the hand operation area and the area of ​​interest have a displacement mapping relationship; move the control points to a target display position, where the target display position is the mapping position of the hand posture data in the area of ​​interest; and respond to the interaction operation generated when the control points act on the object to be interacted with. The above technical solution, when there is a need for interaction with a virtual reality scene, can first roughly determine an area of ​​interest in the virtual reality scene using the player's head control data (such as head posture data or gaze data), and then refine the selection using control points mapped from the hand posture data in the area of ​​interest. Unlike existing interaction solutions, this technical solution combines head control data and hand posture data in a combined form to achieve interaction in the virtual reality scene. This technology effectively solves the problem of interaction fatigue caused by single interaction on body parts. At the same time, by first narrowing the interaction range to locate the area of ​​interest and then using control points to accurately select interactive objects such as buttons or controls in the area of ​​interest, this technology also reduces the risk of difficulty in accurately selecting interactive objects, improves the accuracy of interaction, and thus enhances the player's immersive experience in virtual reality scenarios. Attached Figure Description

[0024] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0025] Figure 1 A flowchart illustrating an interaction method in a virtual reality scene provided by an embodiment of this disclosure is given;

[0026] Figure 1a A diagram illustrating the effect of the generated region of interest and control points during the execution of the virtual reality scene interaction method provided in this embodiment is given.

[0027] Figure 1b A diagram illustrating the effect of moving the control point at the target display position during the execution of the interaction method in the virtual reality scene provided in this embodiment is given.

[0028] Figure 1c and Figure 1dThe following diagrams illustrate the effect of moving the area of ​​interest during the execution of the interaction method in the virtual reality scene provided in this embodiment.

[0029] Figure 1e and Figure 1f A diagram demonstrating the effect of existing ray-based interaction methods in multi-window interface scenarios is provided.

[0030] Figure 1g and Figure 1h The following diagram illustrates the effect of selecting interactive objects in a multi-window interface scenario based on the method provided in this embodiment.

[0031] Figures 1i to 1k The following diagram illustrates the effect of the interaction method provided in this embodiment in preventing accidental triggering of interactive operations.

[0032] Figure 2 A schematic diagram of the structure of an interactive device in a virtual reality scene provided by an embodiment of this disclosure is given;

[0033] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is given. Detailed Implementation

[0034] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0035] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0036] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0037] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0038] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0039] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0040] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0041] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0042] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0043] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0044] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0045] It's important to note that current virtual reality (VR) scene interactions often rely solely on gestures, controllers, head movements, or eye movements for data input control. For example, mainstream VR headsets typically use controllers as the default interaction method, with raycasting being the most common. In VR scenes, raycasting receives data from the controllers and projects it as a ray in the VR environment. This raycasting creates a virtual controller that the player can manipulate, similar to holding a laser pointer in real-world space, pointing the ray at the desired object. While this provides a relatively intuitive experience, it suffers from several drawbacks. First, it requires the hand to be held upright, leading to arm or hand muscle fatigue. Second, raycasting primarily involves converting angle data to a plane; the further the object is from the player, the greater the change in ray movement for each angle, making precise object selection difficult.

[0046] For example, in virtual reality scenarios, interaction can also be achieved through eye movement or head movement. However, these interaction methods also suffer from problems such as interaction fatigue and low accuracy. Furthermore, these interaction methods are subject to certain scenario limitations, and some scenario interactions cannot be achieved solely through data generated by head movement or eye movement.

[0047] Based on this, this embodiment provides an interaction method that combines multiple input methods to perform virtual reality scene interaction. Figure 1 A flowchart illustrating an interaction method in a virtual reality scene provided by an embodiment of this disclosure is given. This embodiment of the disclosure is applicable to situations where interaction occurs in a virtual reality scene. The method can be executed by an interaction device in the virtual reality scene. The device can be implemented in the form of software and / or hardware, and optionally, it can be implemented by an electronic device, preferably a virtual reality device, such as virtual reality glasses, virtual reality helmets, etc.

[0048] like Figure 1 As shown, the method in this embodiment may specifically include:

[0049] S101, Receive head control data acting in the virtual reality scene.

[0050] In this embodiment, the virtual reality scene can be considered as a scene presented to the player through a virtual reality device using virtual reality technology. It can present different virtual reality scenes based on the player's different choices. For example, it could be a game scene to enter, or an information browsing scene. An information browsing scene could be a single window interface presented from a distance, or multiple window interfaces, such as a menu interface or a webpage interface.

[0051] In this embodiment, it can be assumed that the player enters any virtual reality scene by making a selection, and the virtual reality device, which is the subject of the method provided in this embodiment, can present the scene screen of the virtual reality scene selected by the player.

[0052] In this embodiment, the head control data can specifically be considered as data captured by an eye-tracking device or a head-tracking device. The head control data can be gaze data captured by the eye-tracking device when the player's eyes move, or head posture data captured by the head-tracking device when the player's head moves. This step can receive head posture data or gaze data generated during head movement and / or eye movement.

[0053] S102. Determine the area of ​​interest based on the head control data, and display the area of ​​interest and control points in the virtual reality scene. The control points are initially located at a set position in the area of ​​interest, and the area of ​​interest includes objects to be interacted with.

[0054] In this embodiment, the received head control data can be converted into posture information in virtual space. For example, it can determine the player's orientation and position in the virtual reality scene, as well as the player's gaze direction. The virtual reality scene presents visual or visual window interfaces, such as plants, trees, buildings, or other game characters in game or map scenes, or menu windows or webpage windows in information browsing scenes. The presented scenery or window interfaces include interactive objects for the player to interact with, such as targets, game items, equipment, etc., as well as various buttons and controls within the window interface.

[0055] This step uses orientation or gaze information from head control data to roughly determine a region of focus for the player in the virtual reality scene. In this embodiment, this region can be designated as the focus area and displayed visually. The displayed focus area can be a circular or quadrilateral planar area, and in principle, it should not obstruct the display of other objects in the virtual reality scene. This step can weaken the presentation of the focus area by increasing its transparency. Simultaneously, a center point exists within the focus area. In this embodiment, a control point can be visually displayed at the center point of the focus area; alternatively, the control point can be considered to initially appear at the center point of the focus area.

[0056] It should be noted that the head control data received in this embodiment may include head posture data and gaze data. This step can choose one of them (such as head posture data or gaze data) to determine the region of interest; or the two types of data can be combined to determine the region of interest.

[0057] In one implementation, the orientation of the player in the virtual reality scene can be determined by the head pose data. In this embodiment, a scene area within a set range can be circled along the orientation direction as the focus area. The set range can be set according to the player's viewing range and is ensured not to be greater than the viewing range.

[0058] In another implementation, the line-of-sight direction of the player's eyes in the virtual reality scene can be determined by the implementation data. In this embodiment, a scene area within a set range can be circled along the line-of-sight direction as the focus area. It can be known that this embodiment can also combine the head pose data and the line-of-sight data. First, determine the circling direction of the focus area in the virtual reality scene through the orientation direction in the head pose data, and then determine the location of the focus area through the line-of-sight direction in the line-of-sight data.

[0059] In addition, it should be noted that the purpose of circling the focus area through the head control data in this embodiment is to roughly locate the position of a certain interaction object to be interacted with. The interaction object to be interacted with can be considered as a certain scene presented in the virtual reality scene or a certain button / control in the presented window interface. The head control data received in the above steps can be considered as the data captured by the corresponding capture device when the player focuses on a certain interaction object to be interacted with. These data reflect the position of the interaction object to be interacted with in the virtual reality space from the side.

[0060] Following the above description, for the player, when selecting an interaction object to be triggered in the presented virtual reality scene, the player can project the line of sight onto the interaction object to be triggered through eye movement, or turn the face towards the interaction object to be triggered through head movement. For the virtual reality device, the eye movement acquisition device and the head movement acquisition device配套 with the virtual reality device will correspondingly capture the line-of-sight data and the head pose data. The captured data can be used as the head control data and can be received through the steps of S101 in this embodiment above.

[0061] Therefore, it can be considered that the focus area determined by this embodiment contains the interaction object to be interacted with that the player expects to perform interaction operations on. The focus area and the center point presented in this step can both be regarded as the visual effects in the interaction implementation, enabling the player to have a better sensory experience in the interaction.

[0062] Exemplarily, Figure 1a a display diagram of the effects of the focus area and the control points generated during the execution of the interaction method for the virtual reality scene provided in this embodiment is given. As Figure 1aAs shown, the virtual reality scene presented can be a window interface 11, with the attention area 12 displayed in the window interface 11. The control point 13 is initially located at the center of the attention area 12. The attention area 12 also includes an interactive object 14 that the player expects to interact with, which is preferably displayed as a virtual button.

[0063] S103. Receive hand posture data generated in the hand operation area, wherein the hand operation area and the area of ​​interest have a displacement mapping relationship.

[0064] In this embodiment, this step can receive hand posture data, which can be considered as data projected from the player's hand movements within the constructed hand operation area. Specifically, the hand posture data can be obtained by projecting data captured by the hand posture capture device. The captured data can be data generated by capturing the player's hand movements, and the captured data can be in 6DOF format, which can be projected onto the hand operation area to form two-dimensional hand posture data.

[0065] The hand gesture capture device is preferably a controller that is compatible with the virtual reality device. When the player holds the controller and performs hand movements, the controller can capture the player's hand movements and generate corresponding 6DOF data. Accordingly, the controller can be presented as a virtual controller in the virtual reality scene.

[0066] In this embodiment, the hand operation area can be considered as a virtual planar area constructed with the current position of the virtual controller as the center. It can be displayed visually adjacent to the virtual controller in the virtual reality scene, or it can be logically associated with the player's hand movement area in real space, without being visually presented in the virtual reality scene. When constructing the hand operation area, a circular or quadrilateral planar area can be formed with the current position of the hand as the center. The radius of the circle or the length and width of the quadrilateral can be a preset value.

[0067] As described above, the function of the hand operation area can be equivalent to a mouse control area or a touch panel that serves as an input device for a computer device in the real space. The handle can correspond to the mouse moving in the mouse control area, or to the finger or touch object moving in the touch panel. In the real space, the cursor on the screen interface can move accordingly as the mouse moves or the touch panel is touched.

[0068] In this embodiment, the purpose of constructing the hand operation area can be considered as ensuring that as the player moves the controller within this area, the control point presented in the virtual reality scene can also move within the area of ​​interest. The key to ensuring the control point moves within the area of ​​interest lies in establishing a displacement mapping relationship between the constructed hand operation area and the area of ​​interest. Specifically, this displacement mapping relationship is similar to the movement mapping relationship between a mouse control area or a touch panel and a computer screen interface in real space. This displacement mapping relationship can be used to map movement within the hand operation area to movement of the control point within the area of ​​interest.

[0069] S104. Move the control point to the target display position, where the target display position is the mapping position of the hand posture data in the area of ​​interest.

[0070] In this embodiment, the position information of the handle in the hand operation area can be determined by the received hand posture data. By the displacement mapping relationship between the hand operation area and the area of ​​interest, the mapping position of the position information in the area of ​​interest can be determined. The mapping position can be regarded as the target display position of the control point. This step can move the control point from its original display position to the target display position. The original display position of the control point can be the initial center point of the area of ​​interest, or the historical display position of the control point as the handle moves.

[0071] It should be noted that in this embodiment, S103 and S104 can be considered as steps that can be continuously and iteratively executed. As long as the handle moves within the hand operation area, S103 can receive hand posture data in real time. Accordingly, this step can determine the corresponding mapped position in the area of ​​interest in real time based on the hand posture data received in real time by S103, and can move the control point in real time based on the mapped position. Similarly, when the handle is stationary at a certain position in the hand operation area, the control point will also move and stay at the mapped position corresponding to that position in the area of ​​interest.

[0072] For example, Figure 1b A diagram illustrating the effect of moving the control point to the target display position during the execution of the virtual reality scene interaction method provided in this embodiment is given. For example... Figure 1bAs shown, the virtual reality scene is also a window interface 11. The window interface 11 presents the area of ​​interest 12. The control point 13 moves according to changes in the received hand posture data, and its movement trajectory can be represented by the first trajectory line 15. It can be seen that the control point 13 eventually stops at the triggerable position of the object to be interacted with 14. In this example, the hand operation area used for hand posture data movement is considered a non-visual area, where only logical data interaction exists, and it is not displayed. Figure 1b The description of the virtual controller is not shown in the image.

[0073] S105. Respond to the interactive operation generated when the control point acts on the object to be interacted with.

[0074] In this embodiment, after roughly locating the interactive object locked by the player through the head control area to form a focus area, the interactive object can be finely located by moving the control point within the focus area.

[0075] For example, in this implementation, under the player's control of the controller, the control point can be moved to the presentation position of the object to be interacted with, or to the trigger component associated with the object to be interacted with, through the above steps S103 and S104. After the control point is at the position associated with the object to be interacted with, the player's control behavior on the controller can be converted into a trigger behavior acting on the object to be interacted with, such as clicking, dragging, or sliding.

[0076] In this embodiment, by analyzing the triggering behavior acting on the object to be interacted with, a corresponding interactive operation can be generated. This step can respond to the generated interactive operation, such as executing the functional logic corresponding to the click operation or the functional logic corresponding to the drag operation.

[0077] This disclosure provides an interaction method in a virtual reality scene. When there is a need to interact with the virtual reality scene, a region of interest can be roughly determined in the virtual reality scene using the player's head control data (such as head posture data or gaze data). Then, control points mapped from the region of interest using hand posture data can be used for fine-tuning. Unlike existing interaction solutions, this technical solution combines head control data and hand posture data to achieve interaction in the virtual reality scene. This effectively solves the problem of interaction fatigue caused by single interactions. Simultaneously, by first narrowing the interaction range to locate the region of interest and then precisely selecting interactive objects such as buttons or controls within that region using control points, this technical solution reduces the risk of difficulty in accurately selecting interactive objects, improves interaction accuracy, and thus enhances the player's immersive experience in the virtual reality scene.

[0078] As a first optional embodiment of this disclosure, based on the above embodiments, this first optional embodiment can further optimize the interaction method in the virtual reality scene, specifically including the following execution steps:

[0079] a1) Control the area of ​​interest to move in the virtual reality scene as the received head control data changes.

[0080] In this embodiment, as the player's eye movements or head movements occur, the gaze data or head posture data captured by the corresponding eye-tracking or head-tracking device will also change. This embodiment can record the gaze data and / or head posture data as head control data, and can receive the changed head control data. Furthermore, based on the correlation between the head control data and the area of ​​interest described above, this step can redetermine the corresponding area of ​​interest upon receiving the changed head control data and present the redefined area of ​​interest in the virtual reality scene.

[0081] Considering that the size and presentation of the focus area remain unchanged, from a visualization perspective, this is equivalent to the focus area's position moving with head and / or eye movements. Furthermore, this embodiment can determine the movement attribute information of the focus area, such as the distance it moves relative to its previous position and its speed. In addition, unlike the focus area which moves in real-time with changes in head control data, the control point initially presented at the center of the focus area moves accordingly, depending on the situation.

[0082] For example, to prevent input jitter caused by head movement and / or eye movement, the control point can determine whether to follow the center point of the area of ​​interest based on the movement attribute information of the area of ​​interest after the movement relative to the previous area of ​​interest.

[0083] As described above, this embodiment obtains a movement attribute value associated with the movement distance set and movement speed of the region of interest based on the movement attribute information determined when the region of interest moves. This step can compare the movement attribute value with a preset threshold.

[0084] b1) If the movement attribute value of the area of ​​interest is less than the set threshold after the movement, keep the display position of the control point unchanged.

[0085] Specifically, if the movement attribute value is less than the set threshold, this step can be used to keep the control point in its original display position before it moved within the area of ​​interest.

[0086] c1) If the movement attribute value is greater than or equal to the set threshold, reset the control point to the center point of the area of ​​interest after movement.

[0087] Specifically, if the movement attribute value is greater than or equal to the set threshold, this step can be used to control the movement of the display position of the control point and specifically reset the display to the center point of the area of ​​interest after the movement.

[0088] Figure 1c and Figure 1d The following diagrams illustrate the effect of moving the region of interest during the execution of the interaction method in the virtual reality scene provided in this embodiment. Figure 1c As shown, the area of ​​interest 12 has moved from its original position (lighter colored circular area) to its current position (darker colored circular area), which indicates that... Figure 1c The movement distance of region 12 in the focus area is relatively short, and the corresponding movement attribute value is also relatively small. Based on the description of the above steps, it can be assumed that the movement attribute value at this time is less than the set threshold. Therefore, Figure 1c Control point 13 remains in its original display position and does not move with the movement of the area of ​​interest 12.

[0089] like Figure 1d As shown, the area of ​​interest 12 has moved from its original position (lighter colored circular area) to its current position (darker colored circular area), which indicates that... Figure 1d The movement distance of region 12 in the focus area is relatively long, and the corresponding movement attribute value is also large. Based on the description of the above steps, it can be assumed that the movement attribute value at this time is greater than the set threshold. Therefore, Figure 1d After the control point 13 in the area of ​​interest 12 moves to its current position, its display position is also reset from its original display position to the center point of the area of ​​interest 12.

[0090] The first optional embodiment described above is essentially a functional extension of the interaction method in a virtual reality scene based on the previous embodiment. It further illustrates the correlation between the area of ​​focus and head control data. When the object the player expects to interact with in the virtual reality scene changes, the head control data can be altered through eye-tracking and / or head-tracking operations. This allows for a rough localization of the new object by moving the area of ​​focus. This further demonstrates the effectiveness of the combined interaction method in practical applications and significantly enhances the player experience.

[0091] Based on the first optional embodiment described above, the interaction method in the virtual reality scene can be further optimized. Specifically, when the movement attribute value is greater than or equal to the set threshold, it may further include:

[0092] Cancel the displacement mapping between the hand operation area and the area of ​​interest before movement, and establish a displacement mapping relationship with the area of ​​interest after movement.

[0093] The key to interactive control using combined head, eye, and hand input in the method provided in this embodiment lies in establishing a displacement mapping between the hand operation area corresponding to the hand movement and the attention areas associated with the eyes and head. This allows the control point to move precisely within the attention area through hand movements. It is known that when the position of the attention area changes in the virtual reality scene, the existing mapping between the hand operation area and the original attention area will not support subsequent movement of the control point. To ensure the normal movement of the control point within the attention area, this step in the first optional embodiment can be used to cancel the displacement mapping between the hand operation area and the original attention area before movement, and re-establish the displacement mapping between the hand operation area and the attention area after movement.

[0094] It should be noted that this embodiment can update the displacement mapping relationship between the hand operation area and the area of ​​interest in real time during the movement of the area of ​​interest in this step. However, considering the occupation of computing resources and its non-essentiality, this embodiment can preferably only establish the displacement mapping relationship between the area of ​​interest and the hand operation area after the movement is stable.

[0095] It should also be noted that the hand operation area is constructed based on the position of the virtual controller presented in the virtual reality scene. Therefore, when the position of the virtual controller changes to meet the set conditions, the position of the hand operation area may also change. When the position of the hand operation area changes, it is also necessary to reconstruct its displacement mapping relationship with the area of ​​interest.

[0096] In this embodiment, the construction of the displacement mapping relationship between the hand operation area and the area of ​​interest can be achieved by aligning the coordinate points in the hand operation area and the coordinate points in the area of ​​interest in the same coordinate system and assigning a given motion sensitivity coefficient.

[0097] For example, the effect of the established displacement mapping relationship can be described as follows: the control handle moves in the hand operation area along a first movement trajectory, and the coordinate points of the hand operation area on the first movement trajectory can be obtained; based on the displacement mapping relationship, the coordinate points of the hand operation area on the first movement trajectory aligned with the coordinate points of the area of ​​interest can be determined, and the second movement trajectory formed based on the determined coordinate points of the area of ​​interest can be considered as the trajectory presented after the control handle responds in the area of ​​interest.

[0098] The above-described technical implementation of this first optional embodiment also illustrates the effectiveness of the combined interaction method in practical applications, and further enhances the player experience.

[0099] To better illustrate the effective improvements in interaction fatigue and accuracy provided by the method in this embodiment, the following example demonstrates the effect. This example uses an information browsing scenario with multiple window interfaces. When the interactive object a player wants to interact with changes from one window interface to another, if the player wants to perform an interactive operation on the new interactive object in the other window interface, using the existing controller raycasting requires a significant movement of the controller to ensure that the raycast points to the new interactive object in the other window interface.

[0100] Figure 1e and Figure 1f The diagram illustrates the effect of selecting interactive objects in a multi-window interface scenario using an existing ray-based interaction method. Figure 1e and Figure 1f As shown, a top-down view of the virtual reality scene is given, in which the first window interface 110 and the second window interface 111 are presented as two line segments, and the virtual controller 16 is shown in the top-down view. Figure 1e The initial state before cross-window interface interaction is given. At this time, the virtual handle 16 points to any interactive object in the first window interface 110, and a ray 100 can be presented visually. Figure 1f The process of cross-window interface interaction is presented. During this interaction, the player will control the virtual controller 16 to move within a large angle range of 1000, so that the visual ray 100 points from the first window interface 110 to a certain interactive object on the second window interface.

[0101] It can be seen that the existing interactive implementation relies solely on a virtual controller for input, and switching between multiple window interfaces requires movement over a large angle, resulting in a poor player experience.

[0102] As described above, when switching between multiple window interfaces using the interaction method provided in this embodiment, it is only necessary to first switch the area of ​​focus from one window interface to another by moving the player's head or eyes. Then, the new object to be interacted with can be located by rotating the controller within a small angle range within the area of ​​focus.

[0103] Figure 1g and Figure 1h The following diagram illustrates the effect of selecting interactive objects in a multi-window interface scenario based on the method provided in this embodiment. Figure 1g and Figure 1h As shown, a top-down view of the virtual reality scene is also given, in which the first window interface 110 and the second window interface 111 are presented as two line segments respectively, and the virtual controller 16 is also shown in the top-down view. Figure 1gThe initial presentation state before cross-window interface interaction is given. At this time, the control point corresponding to the virtual controller 16 is presented in the attention area of ​​the first window interface 110. In the figure, the second ray 130 is used to represent the visual effect of the control point and the virtual controller 16. Figure 1h The process of cross-window interface interaction in this embodiment is presented. During this interaction, the player only needs to turn their head so that their face faces the second window interface 111, or the player moves their eyes so that their gaze is directed on the second window interface 111. The facial orientation or gaze direction on the second window interface 111 can be used to present an area of ​​interest for roughly locating the new interactive object. Figure 1f The virtual ray is used to represent the top-down visual effect of the virtual controller 16 and the control point in the area of ​​interest. Then, the virtual controller 16 can be rotated by a small angle range of 1300, so that the control point can point to a new interactive object in the area of ​​interest. At this time, the top-down visual effect of the control point and the virtual controller can be represented by the second ray 130.

[0104] As can be seen, the interaction method provided in this embodiment, which uses a combination of head movement, eye movement and hand movements to achieve the response of interactive operations, can better solve the problems of interaction fatigue and low interaction accuracy, and effectively improve the player experience.

[0105] As a second optional embodiment of the present disclosure, based on the above embodiments, the second optional embodiment can further optimize the interaction method in the virtual reality scene, specifically including the following execution steps:

[0106] a2) Upon receiving a motion sensitivity adjustment operation, update the displacement mapping relationship between the area of ​​interest and the hand operation area.

[0107] In this embodiment, the displacement of the handle within the hand operation area and the length of the corresponding displacement of the control point within the area of ​​interest can be adjusted according to different motion sensitivity levels. In this embodiment, motion sensitivity adjustment can be initiated via a button press or gesture through a generated motion sensitivity adjustment operation.

[0108] This step updates the displacement mapping between the area of ​​interest and the hand operation area after receiving a motion sensitivity adjustment request, thereby adjusting the motion sensitivity. It can be understood that if the received motion sensitivity adjustment request aims to reduce motion sensitivity, it can be done by adjusting the displacement mapping between the area of ​​interest and the hand operation area, thus reducing the displacement length of the control point mapped within the area of ​​interest when the controller moves once. In effect, this is equivalent to the control point only moving a short distance within the area of ​​interest when the player moves the controller once.

[0109] Similarly, if the received motion sensitivity adjustment command wants to increase motion sensitivity, it can be done by adjusting the displacement mapping relationship between the area of ​​focus and the hand operation area, thereby increasing the displacement length value of the control point mapped in the area of ​​focus when the controller moves once. In effect, this is equivalent to the control point responding sensitively to a longer displacement in the area of ​​focus when the player moves the controller once.

[0110] b2) When new hand posture data is received, the mapping position of the hand posture data in the region of interest is determined according to the updated displacement mapping relationship, and used as the new target display position.

[0111] As described above, after adjusting the motion sensitivity through the updated displacement mapping relationship, when this step receives new hand posture data again, the corresponding mapping position of the hand posture data in the area of ​​interest will be adjusted based on the updated displacement mapping relationship. This step can use the adjusted and determined mapping position as the new target display position of the control point.

[0112] It can be seen that if the sensitivity is lowered, the distance between the unadjusted mapping position and the original display position will be smaller compared to the distance between the adjusted mapping position and the original display position; if the sensitivity is higher, the distance between the unadjusted mapping position and the original display position will be larger compared to the distance between the adjusted mapping position and the original display position.

[0113] c2) Move the control point to the new target display position.

[0114] In response to the received hand posture data, this step can move the control point to a new target display position for display.

[0115] This second optional embodiment further enhances the functionality of the interaction method. Based on the above method implementation, it specifically adds the effect of motion sensitivity adjustment operations on the movement of the control point within the region of interest. This technical implementation also demonstrates the effectiveness of combined interaction in practical applications, further improving the control of fine displacement.

[0116] As a third optional embodiment of this disclosure, based on the above embodiments, this third optional embodiment can further optimize the interaction method in the virtual reality scene. Specifically, this third optional embodiment can optimize the implementation of establishing the displacement mapping relationship between the region of interest and the hand operation region into the following steps:

[0117] a3) Based on the hand position data in the hand posture data, define the hand operation area and obtain the preset current movement sensitivity coefficient.

[0118] In this embodiment, the hand position data can be considered as the current position data of the virtual controller. The hand operation area can be expanded using the coordinate point corresponding to the hand position data as the center. This embodiment can also use the coordinate point corresponding to the hand position data as the origin of the operation coordinate system when constructing a coordinate system based on the hand operation area. This step can also obtain the current movement sensitivity coefficient that matches the currently set movement sensitivity.

[0119] b3) Align the coordinate points in the hand operation area with the coordinate points in the area of ​​interest using the selected alignment reference point.

[0120] In this embodiment, the coordinate points of the hand operation area and the coordinate points of the area of ​​interest can be aligned using a selected alignment reference point, with one of the coordinate systems as a reference. For example, the alignment reference point can be the coordinate point of the upper left corner of both the hand operation area and the area of ​​interest, the coordinate point of the center point of both the hand operation area and the area of ​​interest, or the origin of the coordinate system of both the hand operation area and the area of ​​interest.

[0121] c3) Using the current motion sensitivity coefficient, establish the displacement mapping relationship between the coordinate points in the hand operation area and the coordinate points in the area of ​​interest.

[0122] In this embodiment, for example, the origin of the operation coordinates in the coordinate system of the hand operation area and the origin of the region coordinates in the coordinate system of the area of ​​interest can be aligned using one of the coordinate systems as a reference. This step can establish a displacement mapping relationship between the coordinate points in the hand operation area and the coordinate points in the area of ​​interest based on the current motion sensitivity coefficient obtained above. The resulting displacement mapping relationship constitutes the displacement mapping relationship between the area of ​​interest and the hand operation area.

[0123] The above technical solution in this embodiment provides one specific implementation of the mapping relationship between the area of ​​interest and the hand operation area, which is equivalent to providing underlying technical support for the interactive implementation of this embodiment.

[0124] As a fourth optional embodiment of this embodiment, based on the above embodiment, it can be further optimized before the interaction operation generated when the control point acts on the object to be interacted, including: analyzing the triggering behavior of the control point on the triggering component, and generating an interaction operation acting on the object to be interacted; wherein, the triggering component is the object to be interacted, and / or, is a triggering area formed after the area of ​​the object to be interacted is expanded.

[0125] In this embodiment, the specific interactive operations that can be responded to by the interactive object can be generated by analyzing the triggering behavior of the control point relative to the triggering component. The triggering behavior can include clicking, dragging, and rotating. Generally, the interactive object can be directly regarded as a triggering component, and the triggering behavior can be directly applied to the interactive object. However, considering the accidental touch behavior caused when the control point acts on the edge of the interactive object, this embodiment optimizes the triggering component. Alternatively, the triggering area formed by expanding the interactive object can also be regarded as the triggering component.

[0126] It is understood that in this embodiment, the control point can trigger an interactive action whether it is applied to the object to be interacted with or to the corresponding trigger area of ​​the object. As an example of avoiding accidental touches in this embodiment, the object to be interacted with can preferably be each character button on a virtual keyboard presented in the virtual reality scene for text input.

[0127] Based on the fourth optional embodiment described above, the step of determining the trigger region can be optimized as follows:

[0128] a4) When the control point is detected to be located in the edge region of the object to be interacted with, the region is expanded at equal intervals starting from the edge frame of the object to be interacted with.

[0129] In this embodiment, hand gesture data can be used to move the control point to the object to be interacted with for display. If the display position of the control point is detected to be within the edge region of the object to be interacted with, it can be considered that an operation constituting the trigger area of ​​the object to be interacted with has been triggered, where the edge region can be considered as the bounding box of the object to be interacted with.

[0130] This step can expand the area outward from the center of the object to be interacted with when the conditions for building the trigger area are met. Specifically, the area can be expanded outward at equal intervals from the edge box of the object to be interacted with.

[0131] b4) The extended area is defined as the trigger area, wherein the distance value of the equal distance is less than the adjacency distance between the object to be interacted with and the adjacent object.

[0132] This step defines the extended area formed around the object to be interacted with as the extended area of ​​the object to be interacted with. When expanding the area, the equidistant distance between the points of extension must be less than the adjacency distance between the object to be interacted with and other adjacent interactive objects. For example, taking a character button on the presented virtual keyboard as the object to be interacted with, other character buttons adjacent to this character button can be considered as adjacent objects of the object to be interacted with. The adjacency distance can be considered as the shortest distance between this character button and other adjacent character buttons.

[0133] The above-described technical solution in this fourth optional embodiment can also be considered a functional extension of the interaction method provided in this embodiment. While ensuring flexible interaction, it further adds a function to prevent accidental triggering. This function further improves the precision of the interaction.

[0134] To better understand the implementation of the false triggering function in this embodiment, an example is provided for illustration. Figures 1i to 1k The diagram illustrates the effect of the interaction method provided in this embodiment in preventing accidental triggering of interactive operations. For example... Figures 1i to 1k As shown, this embodiment uses character buttons on a virtual keyboard as an example to illustrate the effect. Specifically, as... Figure 1i As shown, control point 13 is located on the edge of the 'Q' character button in the presented virtual keyboard. At this point, considering the accidental triggering mechanism, the control point cannot perform the required triggering behavior on the 'Q' character button. Figure 1j As shown, control points were detected. Figure 1i When the state is presented, the area of ​​the character button 'Q' can be expanded to form a trigger area 17 for the character button 'Q'. It can be seen that the expanded distance of the formed trigger area 17 is less than the minimum adjacent distance between the character button 'Q' and the character button 'W' set, and the character button 'A'. As an extension of the above-mentioned false triggering mechanism, it can be assumed that when the control point 13 leaves the character button 'Q' (the object to be interacted with) or its corresponding trigger area, the formed trigger area 17 will disappear accordingly. Figure 1k As shown, when control point 13 falls on the edge of character button 'W' again, a new trigger area 18 will be formed relative to character button 'W' based on the above operation. It can be seen that the trigger area of ​​character button 'Q' no longer exists at this time.

[0135] As a fifth optional embodiment of this disclosure, based on the above embodiments, this fifth optional embodiment can further optimize the interaction method in the virtual reality scene, specifically, it can include the following execution steps:

[0136] a5) Receives out-of-area posture data generated outside the hand operation area.

[0137] It should be noted that this embodiment is not limited to the control point only being presented when hand posture data is generated within the hand operation area. Based on this fifth optional embodiment, this step can also receive hand posture data generated when the player operates the controller outside the hand operation area, which is referred to as outside area posture data in this embodiment.

[0138] b5) Move the control point to an outside-area display position, which is outside the area of ​​interest and is the scene mapping position of the outside-area pose data relative to the virtual reality scene.

[0139] This step can also display the control points corresponding to the out-of-area pose data in the virtual reality scene. Specifically, you can move the control points to display them at the out-of-area display position corresponding to the out-of-area pose data.

[0140] In this context, the display position outside the designated area can be considered to be outside the area of ​​interest as well, and this display position outside the designated area can be considered as the scene mapping position of the posture data outside the designated area relative to the virtual reality scene. That is, it can be considered that in addition to establishing a displacement mapping relationship between the hand operation area and the area of ​​interest, the virtual controller can also establish a displacement mapping relationship between the activity area outside the hand operation area and the entire virtual reality scene. When the virtual controller moves in a planar area outside the hand operation area, its corresponding hand posture data will also correspond to a mapped position in the virtual reality scene based on the displacement mapping relationship established with the virtual reality scene. In this embodiment, this is denoted as the scene mapping position.

[0141] This fifth optional embodiment can also be regarded as a functional extension of the interaction method provided in this embodiment. It replaces the existing angle mapping between the controller and the virtual reality scene and establishes a new position mapping. As a result, any interactive object in the virtual reality scene can be pointed to in the form of a control point, thus ensuring the flexibility of interaction.

[0142] Based on the fifth optional embodiment described above, further optimizations can be made including: interactive operations generated in response to the control point acting on other objects to be interacted with outside the area of ​​interest.

[0143] It is known that there are interactive objects in the virtual reality scene in addition to the area of ​​interest. Through the above-mentioned control point movement implementation method, this embodiment can also trigger the behavior of interactive objects in other areas outside the area of ​​interest through the control point. The steps added in this optional embodiment can respond to the interactive operations generated when acting on other interactive objects.

[0144] The newly added function in this embodiment can be seen as an improvement on the above-mentioned interaction method. The entire technical solution, through multiple inputs combined with movement, allows the head and eyes to determine the scope of attention, while the hands perform fine control. This combined interaction method effectively solves the problems of interaction fatigue and low interaction precision in existing interactions, thus greatly enhancing the player's immersive experience.

[0145] Figure 2 This is a schematic diagram of the structure of an interactive device in a virtual reality scene provided in an embodiment of this disclosure, as shown below. Figure 2As shown, the device includes: a first receiving module 21, a first display module 22, a second receiving module, a second display module 23, and a first response module 24, wherein...

[0146] The first receiving module 21 is used to receive head control data acting in a virtual reality scene;

[0147] The first display module 22 is used to determine the area of ​​interest based on the head control data, and to display the area of ​​interest and control points in the virtual reality scene. The control points are initially located at a set position in the area of ​​interest, and the area of ​​interest includes an object to be interacted with.

[0148] The second receiving module 23 is used to receive hand posture data generated in the hand operation area, wherein the hand operation area and the area of ​​interest have a displacement mapping relationship;

[0149] The second display module 24 is used to move the control point to the target display position, where the target display position is the mapping position of the hand posture data in the area of ​​interest;

[0150] The first response module 25 is used to respond to the interactive operation generated when the control point acts on the object to be interacted with.

[0151] This disclosure provides an interactive device for virtual reality scenes. When there is a need to interact with the virtual reality scene, a region of interest can be roughly determined in the virtual reality scene using the player's head control data (such as head posture data or gaze data). Then, control points mapped from the region of interest using hand posture data can be used for fine-tuning. Unlike existing interaction solutions, this technical solution combines head control data and hand posture data to achieve interaction in the virtual reality scene. This effectively solves the problem of interaction fatigue caused by single interactions. Simultaneously, by first narrowing the interaction range to locate the region of interest and then precisely selecting interactive objects such as buttons or controls within that region using control points, this technical solution reduces the risk of difficulty in accurately selecting interactive objects, improves interaction accuracy, and thus enhances the player's immersive experience in the virtual reality scene.

[0152] Furthermore, the device may also include:

[0153] A motion control module is used to control the area of ​​interest to move in the virtual reality scene as the received head control data changes;

[0154] The first execution module is used to keep the display position of the control point unchanged if the movement attribute value of the area of ​​interest is less than a set threshold after movement.

[0155] The second execution module is used to reset the control point to the center point of the area of ​​interest after movement when the movement attribute value is greater than or equal to the set threshold.

[0156] Furthermore, the second execution module can also be used to: cancel the displacement mapping between the hand operation area and the area of ​​interest before movement, and re-establish the displacement mapping relationship with the area of ​​interest after movement.

[0157] Furthermore, the device may also include:

[0158] The mapping update module is used to update the displacement mapping relationship between the area of ​​interest and the hand operation area upon receiving a motion sensitivity adjustment operation.

[0159] The information determination module is used to determine the mapping position of the hand posture data in the region of interest according to the updated displacement mapping relationship when new hand posture data is received, and use it as the new target display position.

[0160] The display update module is used to move the control point to the new target display position.

[0161] Furthermore, the device also includes a mapping establishment module, which can be specifically used for:

[0162] Based on the hand position data in the hand posture data, the hand operation area is defined, and the preset current movement sensitivity coefficient is obtained;

[0163] Align the coordinate points in the hand operation area with the coordinate points in the area of ​​interest using the selected alignment reference point;

[0164] The displacement mapping relationship between the coordinate points in the hand operation area and the coordinate points in the area of ​​interest is established using the current movement sensitivity coefficient.

[0165] Furthermore, the device may also include: a trigger analysis module, used for:

[0166] Before responding to the interaction operation generated when the control point acts on the object to be interacted, the triggering behavior of the control point on the triggering component is analyzed, and an interaction operation is generated that acts on the object to be interacted; wherein, the triggering component is the object to be interacted, and / or, is a triggering area formed after the object to be interacted is expanded.

[0167] Furthermore, the device may also include: a trigger area generation module, used for

[0168] Before analyzing the triggering behavior of the control point on the triggering component, when the control point is detected to be located in the edge region of the object to be interacted with, the region is expanded with equal spacing starting from the edge box of the object to be interacted with; the expanded region is determined as the triggering region, wherein the distance value of equal spacing is less than the adjacency distance between the object to be interacted with and the adjacent object.

[0169] Furthermore, the device may also include:

[0170] The third receiving module is used to receive out-of-area posture data generated outside the hand operation area;

[0171] The third display module is used to move the control point to a display position outside the area. The display position outside the area is located outside the area of ​​interest and is the scene mapping position of the posture data outside the area relative to the virtual reality scene.

[0172] Furthermore, the device may also include:

[0173] The second response module is used to respond to the interactive operations generated when the control point acts on other objects to be interacted with outside the area of ​​interest.

[0174] The interactive device in the virtual reality scene provided in the embodiments of this disclosure can execute the interactive method in the virtual reality scene provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0175] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0176] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 3 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 3 The diagram below shows the structure of the terminal device or server 300. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0177] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An edit / output (I / O) interface 305 is also connected to the bus 304.

[0178] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0179] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of embodiments of this disclosure.

[0180] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0181] The electronic device provided in this embodiment and the interaction method in the virtual reality scene provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0182] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the interaction method in the virtual reality scene provided in the above embodiments.

[0183] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0184] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0185] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0186] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: perform material parsing processing on the original material of the business object to obtain the target graphic material of the business object; generate target poster description information based on the target graphic material and the poster generation configuration information of the business object; and generate and display the target poster image of the business object by rendering the target poster description information.

[0187] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0188] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0189] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0190] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0191] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0192] According to one or more embodiments of this disclosure, [Example 1] provides an interaction method in a virtual reality scene, the method comprising:

[0193] Receive head control data acting in a virtual reality scene;

[0194] Based on the head control data, a region of interest is determined, and the region of interest and control points are displayed in the virtual reality scene. The control points are initially located at a set position in the region of interest, and the region of interest includes an object to be interacted with.

[0195] Receive hand posture data generated within the hand operation area, wherein the hand operation area and the area of ​​interest have a displacement mapping relationship;

[0196] Move the control point to the target display position, where the target display position is the mapping position of the hand posture data in the area of ​​interest;

[0197] The interactive operation generated when the control point acts on the object to be interacted with.

[0198] According to one or more embodiments of this disclosure, [Example 2] provides an interaction method in a virtual reality scene, which may further include:

[0199] The area of ​​interest is controlled to move within the virtual reality scene in response to changes in the received head control data;

[0200] If the movement attribute value of the area of ​​interest is less than a set threshold after the movement, the display position of the control point remains unchanged.

[0201] If the movement attribute value is greater than or equal to the set threshold, the control point is reset to the center point of the area of ​​interest after movement.

[0202] According to one or more embodiments of this disclosure, [Example 3] provides an interaction method in a virtual reality scene, which may further include, when the movement attribute value is greater than or equal to the set threshold:

[0203] Cancel the displacement mapping between the hand operation area and the area of ​​interest before movement, and re-establish the displacement mapping relationship with the area of ​​interest after movement.

[0204] According to one or more embodiments of this disclosure, [Example 4] provides an interaction method in a virtual reality scene, which may further include:

[0205] Upon receiving a motion sensitivity adjustment operation, update the displacement mapping relationship between the area of ​​interest and the hand operation area;

[0206] When new hand posture data is received, the mapping position of the hand posture data in the region of interest is determined according to the updated displacement mapping relationship, and used as the new target display position;

[0207] Move the control point to the new target display position.

[0208] According to one or more embodiments of this disclosure, [Example 5] provides an interaction method in a virtual reality scene, the method comprising:

[0209] Optionally, the steps for establishing the displacement mapping relationship between the region of interest and the hand manipulation region include:

[0210] Based on the hand position data in the hand posture data, the hand operation area is defined, and the preset current movement sensitivity coefficient is obtained;

[0211] Align the coordinate points in the hand operation area with the coordinate points in the area of ​​interest using the selected alignment reference point;

[0212] The displacement mapping relationship between the coordinate points in the hand operation area and the coordinate points in the area of ​​interest is established using the current movement sensitivity coefficient.

[0213] According to one or more embodiments of this disclosure, [Example Six] provides an interaction method in a virtual reality scene, which may further include, before responding to an interaction operation generated when the control point acts on the object to be interacted with:

[0214] Analyze the triggering behavior of the control point on the triggering component, and generate interactive operations that act on the object to be interacted with;

[0215] The triggering component is the object to be interacted with, and / or is the triggering area formed after expanding the area of ​​the object to be interacted with.

[0216] According to one or more embodiments of this disclosure, [Example Seven] provides an interaction method in a virtual reality scene, wherein the step of generating a trigger area in the method may include:

[0217] When the control point is detected to be located in the edge region of the object to be interacted with, the region is expanded at equal intervals starting from the edge box of the object to be interacted with.

[0218] The extended area is defined as the trigger area, wherein the distance value of the equal spacing is less than the adjacency distance between the object to be interacted with and the adjacent object.

[0219] According to one or more embodiments of this disclosure, [Example Eight] provides an interaction method in a virtual reality scene, the method further comprising:

[0220] Receive out-of-area posture data generated outside the hand operation area;

[0221] Move the control point to a display position outside the region. The display position outside the region is outside the region of interest and is the scene mapping position of the pose data outside the region relative to the virtual reality scene.

[0222] According to one or more embodiments of this disclosure, [Example Nine] provides an interaction method in a virtual reality scene, the method further comprising:

[0223] The interaction operation generated when the control point acts on other objects outside the area of ​​interest.

[0224] According to one or more embodiments of this disclosure, [Example 10] provides an interactive device in a virtual reality scene, the device comprising:

[0225] The material processing module is used to parse and process the original materials of the business object to obtain the target graphic and text materials of the business object;

[0226] The information generation module is used to generate target poster description information based on the target graphic and textual materials and the poster generation configuration information of the business object;

[0227] The poster generation module is used to generate and display the target poster image of the business object by rendering the target poster description information.

[0228] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0229] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0230] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An interaction method in a virtual reality scene, characterized in that, include: Receive head control data acting in a virtual reality scene, wherein the virtual reality scene is a scene presented to the player through a virtual reality device, wherein the virtual reality device is virtual reality glasses or a virtual reality helmet, and the head control data is at least one of head posture data and gaze data; Based on the head control data, a region of interest is determined, and the region of interest and control points are displayed in the virtual reality scene. The control points are initially located at a set position in the region of interest, and the region of interest includes an object to be interacted with. The system receives hand posture data generated within a hand operation area. The hand operation area and the area of ​​interest have a displacement mapping relationship. The hand posture data is generated by capturing the player's hand movements, which are obtained through a controller held by the player and compatible with the virtual reality device. Move the control point to the target display position, where the target display position is the mapping position of the hand posture data in the area of ​​interest; The interactive operation generated when the control point acts on the object to be interacted with.

2. The method according to claim 1, characterized in that, Also includes: The area of ​​interest is controlled to move within the virtual reality scene in response to changes in the received head control data; If the movement attribute value of the area of ​​interest is less than a set threshold after the movement, the display position of the control point remains unchanged. If the movement attribute value is greater than or equal to the set threshold, the control point is reset to the center point of the area of ​​interest after movement.

3. The method according to claim 2, characterized in that, When the movement attribute value is greater than or equal to the set threshold, the method further includes: Cancel the displacement mapping between the hand operation area and the area of ​​interest before movement, and re-establish the displacement mapping relationship with the area of ​​interest after movement.

4. The method according to claim 1, characterized in that, Also includes: Upon receiving a motion sensitivity adjustment operation, update the displacement mapping relationship between the area of ​​interest and the hand operation area; When new hand posture data is received, the mapping position of the hand posture data in the region of interest is determined according to the updated displacement mapping relationship, and used as the new target display position; Move the control point to the new target display position.

5. The method according to any one of claims 1-4, characterized in that, The steps to establish the mapping relationship between the region of interest and the hand manipulation region include: Based on the hand position data in the hand posture data, the hand operation area is defined, and the preset current movement sensitivity coefficient is obtained; Align the coordinate points in the hand operation area with the coordinate points in the area of ​​interest using the selected alignment reference point; The displacement mapping relationship between the coordinate points in the hand operation area and the coordinate points in the area of ​​interest is established using the current movement sensitivity coefficient.

6. The method according to claim 1, characterized in that, Before the interaction operation generated when the control point acts on the object to be interacted with, the method further includes: Analyze the triggering behavior of the control point on the triggering component, and generate interactive operations that act on the object to be interacted with; The triggering component is the object to be interacted with, and / or is the triggering area formed after expanding the area of ​​the object to be interacted with.

7. The method according to claim 6, characterized in that, The steps for generating the trigger region include: When the control point is detected to be located in the edge region of the object to be interacted with, the region is expanded at equal intervals starting from the edge box of the object to be interacted with. The extended area is defined as the trigger area, wherein the distance value of the equal spacing is less than the adjacency distance between the object to be interacted with and the adjacent object.

8. The method according to claim 1, characterized in that, Also includes: Receive out-of-area posture data generated outside the hand operation area; Move the control point to a display position outside the region. The display position outside the region is outside the region of interest and is the scene mapping position of the pose data outside the region relative to the virtual reality scene.

9. The method according to claim 6, characterized in that, Also includes: The interaction operation generated when the control point acts on other objects outside the area of ​​interest.

10. An interactive device for a virtual reality scene, characterized in that, include: The first receiving module is used to receive head control data acting in a virtual reality scene, wherein the virtual reality scene is a scene presented to the player through a virtual reality device, wherein the virtual reality device is virtual reality glasses or a virtual reality helmet, and the head control data is at least one of head posture data and gaze data. The first display module is used to determine the area of ​​interest based on the head control data, and to display the area of ​​interest and control points in the virtual reality scene. The control points are initially located at a set position in the area of ​​interest, and the area of ​​interest includes an object to be interacted with. The second receiving module is used to receive hand posture data generated in the hand operation area. The hand operation area and the area of ​​interest have a displacement mapping relationship. The hand posture data is generated by capturing the player's hand movements. The player's hand movements are obtained through a controller held by the player and compatible with the virtual reality device. The second display module is used to move the control point to the target display position, where the target display position is the mapping position of the hand posture data in the area of ​​interest; The first response module is used to respond to the interactive operation generated when the control point acts on the object to be interacted with.

11. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-9.

Images