Interaction control method and device, wearable device, and storage medium

Abstract

The application discloses an interactive control method and device, wearable equipment and storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: in response to a first preset gesture, a control component is superimposed and displayed on a target virtual object to be controlled, the control component comprises a plurality of preset control areas and a preset cursor corresponding to each preset control area, and the preset cursor corresponding to each preset control area is associated with a preset control mode; if it is detected that an input target gesture is a preset gesture in the preset control mode associated with the target cursor, the target virtual object is controlled to perform a preset operation corresponding to the target gesture operation, and the target cursor is a preset cursor corresponding to any preset control area in the plurality of preset control areas. In this way, the preset gesture is made when the virtual object is selected by the head-mounted equipment, the head-mounted equipment and the gesture are linked in the interactive process, and the interactive control precision is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an interactive control method, device, wearable device and storage medium. Background Technology

[0002] With the development of extended reality (XR) technology, including virtual reality (VR), mixed reality (MR), and augmented reality (AR), an interactive feedback loop is established between the virtual world, the real world, and the user. In human-computer interaction, head control and gesture operation are the most direct and natural interaction methods. When a user wears a head-mounted display (HMD), they can use their hands to make gestures and further trigger specific functions. These functions can be associated with hardware or software controls. It is easy for the user to control the head-mounted display system using their hands.

[0003] In existing technologies, when users interact between the virtual and the display using head-mounted devices and gestures, they interact with the virtual scene solely through gesture operations or through the head-mounted device. There is no coordinated operation between the head-mounted device and the gesture operations, resulting in a single operation method and low interaction accuracy. Summary of the Invention

[0004] This application proposes an interactive control method, device, wearable device, and storage medium to achieve coordinated control of head-mounted devices and gesture operations, thereby improving the accuracy of interaction.

[0005] In a first aspect, embodiments of this application provide an interactive control method applied to a wearable device. The method includes: responding to a first preset gesture, overlaying and displaying a control component on a target virtual object to be controlled. The control component includes multiple preset control areas and a preset cursor corresponding to each preset control area. The preset cursor corresponding to each preset control area is associated with a preset control mode. If an input target gesture is detected to be a preset gesture under the preset control mode associated with the target cursor, the target virtual object is controlled to perform a preset operation corresponding to the target gesture operation. The target cursor is the preset cursor corresponding to any one of the multiple preset control areas.

[0006] Secondly, embodiments of this application provide an interactive control device applied to a wearable device. The device includes a display module and a control module. The display module is configured to, in response to a first preset gesture, overlay a control component onto a target virtual object to be controlled. The control component includes multiple preset control areas and a preset cursor corresponding to each preset control area. Each preset cursor corresponding to a preset control area is associated with a preset control mode. The control module is configured to, if it detects that the input target gesture is a preset gesture under the preset control mode associated with the target cursor, control the target virtual object to perform a preset operation corresponding to the target gesture operation. The target cursor is the preset cursor corresponding to any one of the multiple preset control areas.

[0007] Thirdly, embodiments of this application provide a wearable device, including: one or more processors; a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the methods described above.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing program code that can be invoked by a processor to execute the methods described above.

[0009] In the solution provided in this application, in response to a first preset gesture, a control component is overlaid and displayed on the target virtual object to be controlled. The control component includes multiple preset control areas and a preset cursor corresponding to each preset control area. Each preset cursor is associated with a preset control method. If the input target gesture is detected as a preset gesture under the preset control method associated with the target cursor, the target virtual object is controlled to perform a preset operation corresponding to the target gesture. The target cursor is any preset cursor corresponding to any of the multiple preset control areas. Thus, when a user selects a virtual object in a virtual scene using a head-mounted device and makes a preset gesture with their hand, the user, based on the prompts from the control component overlaid on the virtual object, performs the corresponding preset operation in the preset control area through gesture operation. The user's head-mounted device and gesture are linked during the interaction. Different preset control methods correspond to different preset control areas. By limiting the execution of corresponding preset operations to different preset control areas, high-precision interactive control can be achieved, improving the accuracy of interactive control and reducing the occurrence of miscontrol problems. Furthermore, the multiple preset control methods make human-computer interaction more diverse and improve the human-computer interaction experience. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating an embodiment of the interactive control method provided in this application is shown.

[0012] Figure 2 A schematic diagram of the preset gestures provided in the embodiments of this application is shown.

[0013] Figure 3 A schematic diagram of the preset control area provided in an embodiment of this application is shown.

[0014] Figure 4 A flowchart illustrating an interactive control method provided in another embodiment of this application is shown.

[0015] Figure 5 A schematic diagram of a preset cursor provided in an embodiment of this application is shown.

[0016] Figure 6 It shows Figure 5 A flowchart illustrating a sub-step of step S340 in one embodiment.

[0017] Figure 7 A schematic diagram of the third preset cursor provided in an embodiment of this application is shown.

[0018] Figure 8 A schematic diagram of a virtual scene according to this embodiment is shown.

[0019] Figure 9 A schematic diagram of another virtual scene in this embodiment is shown.

[0020] Figure 10 This illustration shows a scene diagram of another virtual scene in this embodiment.

[0021] Figure 11 A schematic diagram of another virtual scene in this embodiment is shown.

[0022] Figure 12 This illustration shows another virtual scene in this embodiment.

[0023] Figure 13 A flowchart illustrating an interactive control method provided in another embodiment of this application is shown.

[0024] Figure 14 It shows Figure 13A flowchart illustrating a sub-step of step S510 in one embodiment.

[0025] Figure 15 A structural block diagram of an interactive control device provided in an embodiment of this application is shown.

[0026] Figure 16 A structural block diagram of a wearable device provided in an embodiment of this application is shown.

[0027] Figure 17 A structural block diagram of a computer-readable storage medium provided in an embodiment of this application is shown. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0029] It should be noted that some processes described in the specification, claims, and accompanying drawings of this application include multiple operations that appear in a specific order. These operations may not be performed in the order they appear herein, or they may be performed in parallel. Operation numbers such as S110, S120, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be performed sequentially or in parallel. Also, the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, such that a process, method, system, product, or server that includes a series of steps or sub-modules is not necessarily limited to those steps or sub-modules that are explicitly listed, but may include other steps or sub-modules that are not explicitly listed or that are inherent to such process, method, product, or device.

[0030] In related technologies, when users interact between the virtual and the display using head-mounted devices and gestures, they cannot coordinate and cooperate when interacting with the virtual scene through gestures or head-mounted devices. The operation methods are limited, and traditional head-control and hand-control operations are complex, involving large ranges of motion. The interaction process is complicated and it is difficult to achieve high-precision interactive control.

[0031] To address the aforementioned problems, the inventors propose an interactive control method, device, wearable device, and storage medium. When a virtual object is selected via a head-mounted device, a preset gesture is performed. During the interaction, the head-mounted device and the gesture are linked, simplifying the gesture interaction and improving the accuracy of interactive control. The interactive control method provided in the embodiments of this application is described in detail below.

[0032] Please refer to Figure 1 , Figure 1 A flowchart illustrating an embodiment of the interactive control method provided in this application is shown, applied to a wearable device. The following will combine... Figure 1 The interactive control method provided in the embodiments of this application will be described in detail. The interactive control method may include the following steps:

[0033] Step S110: In response to the first preset gesture, a control component is overlaid on the target virtual object to be controlled. The control component includes multiple preset control areas and a preset cursor corresponding to each preset control area. The preset cursor corresponding to each preset control area is associated with a preset control mode.

[0034] In this embodiment, wearable devices include, but are not limited to, mixed reality (MR) headsets. Wearable devices can also be virtual reality (VR) headsets, augmented reality (AR) headsets, etc.

[0035] Please see Figure 2 , Figure 2 A schematic diagram of preset gestures provided in an embodiment of this application is shown. The interactive control device can pre-store multiple preset gestures associated with preset control methods. In this embodiment, the first preset gesture can be a gesture of opening the back of the hand, that is, a gesture of spreading the five fingers to reveal the back of the hand. When the user interacts with the wearable device through gesture operation, interactive control of the target virtual object is realized. Figure 2The document includes a gesture diagram with the back of the hand open as the first preset gesture. In some embodiments, based on different users' actual usage habits, the first preset gesture can also be set to other gestures such as the palm open gesture. The interactive control device also pre-stores a second preset gesture and a third preset gesture, which users can use to control the target virtual object. The second preset gesture is a gesture with the back of the hand closed, i.e., a fist with the five fingers bent, and the third preset gesture is a gesture with the index finger and thumb pinched together.

[0036] Optionally, when the interactive control device detects that the virtual cursor controlled by the head-mounted device has entered the interactive range of the target virtual object in the virtual scene, it controls the target virtual object to enlarge or change its appearance attributes accordingly, such as changing its color, to prompt the user that the target virtual object is interactive. This means the user can perform a preset gesture to control the target virtual object to execute a corresponding operation, and a bounding box is superimposed on the target virtual object to indicate its interactive range. When the interactive control device detects that the virtual cursor controlled by the head-mounted device has entered the interactive range of the target virtual object in the virtual scene, and the user performs a first preset gesture, the interactive control device controls the display of control components superimposed on the target virtual object. The user can then perform preset control actions associated with the preset cursor within this interactive range. The interactive range of the target virtual object is the area formed by the virtual object's own boundary. When the position of the virtual cursor in the virtual scene overlaps with the area formed by the virtual object's boundary, it has entered the virtual object's interactive range.

[0037] In this embodiment, the shape of the target virtual object can be a regular shape or an irregular shape. When the user performs related control operations on the target virtual object through gestures, the bounding box superimposed on the target virtual object is always a regular rectangle or cube. When the target virtual object is an irregular shape, the smallest rectangle or cube that can enclose the target virtual object is used as the bounding box.

[0038] The bounding box used to limit the interactive range of the target virtual object is divided into multiple preset control areas. When the interactive control device detects that the virtual cursor controlled by the head-mounted device is located in different preset control areas within the bounding box, the bounding box switches to the corresponding preset bounding box, and the virtual cursor switches to the corresponding preset cursor. Furthermore, the preset bounding box and preset cursor are associated with a preset control method. By overlaying the aforementioned preset bounding box and preset cursor on the target virtual object, the user can be prompted to perform a gesture corresponding to the preset cursor within that control area, thereby achieving the associated preset operation.

[0039] Please see Figure 3 , Figure 3A schematic diagram of a preset control area provided in an embodiment of this application is shown. Figure 3 As shown, Figure 3 This embodiment shows that when the bounding box is not a cube, the preset control area may include a preset control area 201 located in the area below the bounding box, a preset control area 202 located at each vertex of the bounding box, and a preset control area 203 located on the boundary of the bounding box parallel to the Y-axis. In other embodiments, the preset control area may be set in different forms, such as the preset control area may also be set on the boundary of the bounding box parallel to the Y-axis.

[0040] In some implementations, wearable devices can pre-store a variety of preset gestures. The interactive control device can acquire the user's gestures through its own image acquisition module or an external image acquisition module that is connected to it, and compare the input gestures with the stored preset gestures. When the user's gesture is detected to match the first preset gesture, the control component is superimposed on the target virtual object.

[0041] All virtual objects displayed in a virtual scene can be used as target virtual objects, and the interactive range of a target virtual object is divided into multiple preset control areas. It can be understood that the boundary size of a target virtual object is the size of its interactive range.

[0042] Step S120: If the input target gesture is detected to be a preset gesture under a preset control mode associated with the target cursor, control the target virtual object to perform a preset operation corresponding to the target gesture operation, wherein the target cursor is the preset cursor corresponding to any preset control area among the plurality of preset control areas.

[0043] In this embodiment, as Figure 3 As shown, the interactive control device overlays a preset bounding box and a preset cursor corresponding to the preset control area onto the target virtual object, prompting the user to make a gesture corresponding to the preset cursor in the control area, and using the preset cursor as the target cursor.

[0044] Optionally, the interactive control device collects the user's gestures and inputs the collected data into the device, using the user's current gesture as the target gesture. The interactive control device compares the target gesture with stored preset gestures associated with the target cursor. When it detects that the input target gesture matches a preset gesture under a preset control mode associated with the target cursor, the user can use the target gesture to control the preset cursor superimposed on the target virtual object to move, thereby controlling the target virtual object to perform a preset operation corresponding to the target gesture operation under the preset control mode.

[0045] In this embodiment, the preset operation can be a rotation operation, a movement operation, a zoom-out operation, a zoom-in operation, or a click operation, a selection operation, an activation operation, or a confirmation operation, etc.

[0046] For example, if the target cursor is a movement operation cursor, and a gesture associated with a movement operation is detected within the interactive range of the target virtual object, the user can control the movement operation cursor superimposed on the target virtual object while maintaining the gesture, so as to control the target virtual object to perform the corresponding movement.

[0047] In this embodiment, when a user selects a virtual object in a virtual scene using a head-mounted device and makes a preset hand gesture, the user executes a corresponding preset operation in a preset control area based on the prompts of the control components superimposed on the virtual object. The user's head-mounted device and the gesture are linked during the interaction. At the same time, different preset control methods correspond to different preset control areas. By limiting the execution of corresponding preset operations to different preset control areas, high-precision interactive control can be achieved, improving the accuracy of interactive control and reducing the occurrence of problems such as miscontrol. Furthermore, the multiple preset control methods make human-computer interaction more diverse and improve the human-computer interaction experience.

[0048] Please refer to Figure 4 , Figure 4 A flowchart illustrating an interactive control method according to another embodiment of this application is shown, applied to a wearable device. The following will combine... Figure 4 The interactive control method provided in the embodiments of this application will be described in detail. The interactive control method may include the following steps:

[0049] Step S310: Display a virtual scene, which includes a first preset cursor and multiple first preset virtual objects.

[0050] In this embodiment, when the interactive control device displays a virtual scene, there may be multiple virtual objects in the virtual scene. The virtual objects displayed in the virtual scene are used as the first preset virtual objects that can be used for interaction. A first preset cursor is also displayed in the virtual scene. The user can control the first preset cursor to move in the virtual scene through the wearable device.

[0051] Virtual scenes can include 3DOF (Degree of Freedom) scenes and 6DOF scenes, where DOF stands for degrees of freedom. When the displayed virtual scene is a 3DOF scene, the virtual objects within it have three degrees of freedom for rotation. When the displayed interactive virtual scene is a 6DOF scene, the virtual objects within it, in addition to the three rotational degrees of freedom, also have three positional degrees of freedom: up / down, forward / backward, and left / right. That is, when a user performs preset operations on a target virtual object in a 3DOF scene using an interactive control device, the user can perform two-dimensional rotation, movement, scaling, and zooming operations on the target virtual object, without involving spatial changes in up / down, forward / backward, or left / right.

[0052] When a user enters a virtual scene using a wearable device, the user's position is displayed by a first preset cursor located 3 meters away from the wearable device. The distance between the first preset cursor and the wearable device is adjustable.

[0053] In this embodiment, when a user enters a virtual scene using a wearable device, a first preset cursor appears in the virtual scene, and the position of the first preset cursor in the virtual scene moves as the user's head freely rotates in different directions while wearing the wearable device.

[0054] In some implementations, when the interactive control device is activated but the user has not entered the virtual scene using the wearable device, the first preset cursor can be set to be displayed in the middle position of the virtual scene. This remains until the interactive control device detects that the user has entered the virtual scene using the wearable device. At this point, the first preset cursor is associated with the wearable device, and its position movement within the virtual scene is controlled by the wearable device. Of course, the initial display position of the first preset cursor can also be set at the bottom edge, top edge, left edge, or right edge of the virtual scene. This application embodiment does not limit the initial display position of the first preset cursor in the virtual scene.

[0055] Step S320: If the positional relationship between the first preset cursor and the second preset virtual object is detected to satisfy the first preset positional relationship, the second preset virtual object is determined as the target virtual object, and the second preset virtual object is any virtual object among the multiple first preset virtual objects.

[0056] In this embodiment, the position of the first preset cursor in the virtual scene moves as the user wears the wearable device and the user's head rotates freely in different directions. When the user controls the movement of the first preset cursor through the wearable device and overlays it onto the second preset virtual object, the virtual object is selected as the target virtual object.

[0057] In this application, any one of the first preset virtual objects can be a second preset virtual object. The first preset positional relationship means that the relative distance between the first preset cursor controlled by the wearable device and the second preset virtual object is less than a preset distance. The first preset cursor is superimposed on the second preset virtual object. The preset distance can be a pre-set value, or it can be adjusted according to the different control precision in actual applications. This embodiment does not limit this.

[0058] In this embodiment, the user can control the movement of the first preset cursor in the virtual scene through the wearable device, so that the user can select the target virtual object from multiple virtual objects in the virtual scene according to the needs by simply rotating the head in different directions. This allows the user to quickly select the target virtual object and control it without performing complex control operations after entering the virtual scene.

[0059] Step S330: Switch the display of the first preset cursor to the second preset cursor.

[0060] In this embodiment, a first preset cursor is superimposed on the interactive range of the target virtual object. The interactive control device switches the cursor displayed in the virtual scene from the first preset cursor to the second preset cursor, and the target virtual object is enlarged or changes color accordingly.

[0061] Please see Figure 5 , Figure 5 A schematic diagram of a preset cursor provided in an embodiment of this application is shown. Figure 5 As shown, the first preset cursor is the initial cursor, which can be a cross shape. The second preset cursor can be a solid dot. In other embodiments, the shape of the preset cursor can also be a square, a rectangle, etc.

[0062] Optionally, in some embodiments, the first preset cursor and the second preset cursor can be set to the same cursor. When the user selects a target virtual object, the first preset cursor no longer switches its cursor pattern to indicate that the user has entered the interactive range of the target virtual object and has made a selection. In this case, the interactive control device directly prompts the user by controlling the target virtual object to zoom in or change its color.

[0063] Step S340: In response to the first preset gesture, a control component is overlaid on the target virtual object to be controlled. The control component includes multiple preset control areas and a preset cursor corresponding to each preset control area. The preset cursor corresponding to each preset control area is associated with a preset control mode.

[0064] Specifically, please refer to Figure 6 , Figure 6 It shows Figure 5A flowchart illustrating a sub-step of step S340 in one embodiment. Step S340 may include the following steps:

[0065] Step S341: In response to the first preset gesture, determine the preset control area where the second preset cursor is located from the plurality of preset control areas, and use it as the first target control area.

[0066] Optionally, when the interactive control device detects that the virtual cursor controlled by the head-mounted device has entered the interactive range of the target virtual object in the virtual scene, and the user makes a first preset gesture, the interactive control device controls the display of control components superimposed on the target virtual object, and divides the target virtual object into multiple preset control areas within the bounding box used to limit the interactive range of the target virtual object. The interactive control device determines the position of the second preset cursor within the bounding box as the first target control area.

[0067] In this embodiment, the multiple preset control areas include at least a movement trigger area, a scaling trigger area, and a rotation trigger area. Users can control the target virtual object to perform movement operations in the movement trigger area; control the target virtual object to perform zoom-in and zoom-out operations in the scaling trigger area; and control the target virtual object to perform rotation operations in the rotation trigger area.

[0068] Step S342: Switch the display of the second preset cursor to the third preset cursor corresponding to the first target control area, and display the preset gesture pattern associated with the third preset cursor.

[0069] Optionally, the third preset cursor corresponding to the first target control area includes at least a movement cursor, a zoom cursor, and a rotation cursor, and the third preset cursor includes a cursor pattern and a gesture pattern. For details, please refer to... Figure 7 , Figure 7 A schematic diagram of a third preset cursor provided in an embodiment of this application is shown. Figure 7 As shown, the movement cursor is related to the second preset gesture, consisting of a clasped hand gesture pattern and a crosshair above it; the zoom cursor is related to the third preset gesture, consisting of a pinched index finger and thumb gesture and two arrows at its upper right and lower right; the rotation cursor is also related to the third preset gesture, consisting of a pinched index finger and thumb gesture and a rotating arrow above it. The preset gesture image shown by the third preset cursor corresponding to the first target control area is used to prompt the user to change the first preset gesture to the preset gesture corresponding to the third preset cursor within the interactive range of the target virtual object, thus achieving target control operation.

[0070] Optionally, the lower area of ​​the user-facing side within the bounding box is set as the movement trigger area. A second preset cursor is overlaid on the lower area of ​​the user-facing side within the bounding box, and the bounding box overlaid on the target virtual object is switched to a movement operation bounding box, i.e., a control bar appears at the lower boundary of the user-facing side of the bounding box. When a first preset gesture is detected, the second preset cursor overlaid on the target virtual object is switched to a movement operation cursor to prompt the user that the target virtual object can be moved within this area using movement operation gestures.

[0071] In some implementations, if the first target control area corresponding to the third preset cursor is the movement trigger area, the second preset cursor is switched to display as the movement cursor pattern and movement gesture pattern corresponding to the movement trigger area.

[0072] Specifically, please refer to Figure 8 , Figure 8 A schematic diagram of a virtual scene according to this embodiment is shown. When the virtual scene is a 3DOF scene, the bounding box superimposed on the target virtual object is a rectangle, and the target virtual object displays "Rudder Angle: Specified Rudder Angle; Rudder Mode: Vertex Hold; Speed ​​Mode: Constant Speed; 2022-02-01; 16:40". A second preset cursor is superimposed on the preset display area 401. In this embodiment, the preset display area 401 can be the lower area of ​​the bounding box. When the second preset cursor is superimposed on the lower area of ​​the bounding box, the bounding box switches to the movement operation bounding box shown in the figure, and a control bar appears on the lower boundary of the bounding box.

[0073] In the aforementioned virtual scenario, after step S342, if a second preset gesture is detected from the user, the target virtual object is enlarged or its color changes. The second preset cursor is switched to display as the movement cursor pattern and movement gesture pattern corresponding to the movement trigger area. The movement operation cursor moves within the interactive range following the user's gesture. The interactive control device extracts the displacement value of the second preset gesture made by the user in the XY axis direction and maps it onto the virtual scene to control the target virtual object to achieve the corresponding movement. If the movement operation cursor corresponding to the second preset gesture made by the user is detected to leave the interactive range, or if the user makes another preset gesture, the interactive control device moves the target virtual object to the position where the second preset gesture was last detected within the interactive range, completing the movement operation.

[0074] Please see Figure 9 , Figure 9A scene diagram of another virtual scene in this embodiment is shown. When the virtual scene is a 6DOF scene, the bounding box superimposed on the target virtual object is a cube, and the second preset cursor is superimposed on the preset display area 402. In this embodiment, the preset display area 402 can be the lower area of ​​the side of the bounding box facing the user. At this time, when the second preset cursor is superimposed on the lower area of ​​the side of the bounding box facing the user, the bounding box switches to the movement operation bounding box shown in the figure, and a control bar appears at the lower boundary of the bounding box.

[0075] In the aforementioned virtual scenario, after step S342, if a second preset gesture is detected from the user, the target virtual object is enlarged or its color changes. The second preset cursor switches to display as the movement cursor pattern and the movement gesture pattern corresponding to the movement trigger area. The movement operation cursor moves within the interactive range following the user's gesture. The interactive control device extracts the displacement value of the second preset gesture made by the user in the XYZ axis direction and maps it onto the virtual scene to control the target virtual object to achieve the corresponding movement, thereby realizing the movement operation of the target virtual object. Further details are omitted here.

[0076] Optionally, each vertex region of the bounding box is set as a zoom trigger region, and only the user-facing zoom trigger region on the bounding box is available for interaction. When the second preset cursor is overlaid on the user-facing zoom trigger region within the bounding box, the bounding box overlaid on the target virtual object switches to a zoom operation bounding box, that is, a zoom style appears on the lower boundary of the user-facing side of the bounding box. Furthermore, when the user makes a first preset gesture, the second preset cursor overlaid on the target virtual object switches to a zoom operation cursor, prompting the user that the target virtual object can be zoomed within this range using zoom operation gestures.

[0077] In some implementations, if the first target control area corresponding to the third preset cursor is the zoom trigger area, the second preset cursor is switched to display as the zoom cursor pattern and zoom gesture pattern corresponding to the zoom trigger area.

[0078] Specifically, please refer to Figure 10 , Figure 10 This illustration shows another virtual scene in this embodiment. When the virtual scene is a 3DOF scene, the bounding box superimposed on the target virtual object is a rectangle, and the second preset cursor is superimposed on the preset display area 403. In this embodiment, the preset display area 403 can be the scaling area of ​​the bounding box. When the second preset cursor is superimposed on the scaling area of ​​the bounding box, the bounding box switches to the scaling operation bounding box shown in the figure, and a scaling style appears.

[0079] In the aforementioned virtual scenario, after step S342, if a third preset gesture is detected from the user, the target virtual object will be enlarged or its color changed. The second preset cursor will switch to display the zoom cursor pattern and zoom gesture pattern corresponding to the zoom trigger area. The zoom operation cursor moves within the interactive range following the user's gesture. The interactive control device extracts the displacement value of the user's third preset gesture in the XY axis direction and maps it onto the virtual scene to control the target virtual object to achieve corresponding scaling. It should be noted that in the above operation, the vertex opposite the vertex that triggers the zoom operation on the selected rectangular bounding box is used as the origin, and isometric scaling is applied to the target virtual object.

[0080] If the cursor moves out of the interactive range when the user makes a third preset gesture, or if the user makes another preset gesture, the interactive control device will scale the target virtual object isometrically to the position where the third preset gesture was last detected within the interactive range, thus completing the scaling operation.

[0081] Please see Figure 11 , Figure 11 This illustration shows another virtual scene in this embodiment. When the virtual scene is a 6DOF scene, the bounding box superimposed on the target virtual object is a cube, and the second preset cursor is superimposed on the preset display area 404. In this embodiment, the preset display area 404 can be the zoom area of ​​the bounding box facing the user. When the second preset cursor is superimposed on the zoom area of ​​the bounding box facing the user, the bounding box switches to the zoom operation bounding box shown in the figure, and a zoom style appears.

[0082] In the aforementioned virtual scenario, after step S342, if a third preset gesture is detected from the user, the target virtual object will be enlarged or its color changed. The second preset cursor will switch to display the zoom cursor pattern and zoom gesture pattern corresponding to the zoom trigger area. The zoom operation cursor moves within the interactive range following the user's gesture. The interactive control device extracts the displacement value of the user's third preset gesture in the XYZ axis direction and maps it onto the virtual scene to control the target virtual object to achieve corresponding shrinking or enlargement, thereby realizing the zoom operation of the target virtual object. This will not be elaborated further. It should be noted that in the above operation, if the vertex triggering the zoom operation is on the user-facing side of the cube's bounding box, the diagonal point of that vertex on the user-facing side is selected as the origin; if the vertex triggering the zoom operation is on a non-user-facing side of the cube's bounding box, the diagonal point of that vertex on the cube is selected as the origin, and isometric scaling of the target virtual object is performed.

[0083] Optionally, the area parallel to the Y-axis on the bounding box is set as the rotation trigger area, and only the user-facing rotation trigger area on the bounding box is available for interaction. When the second preset cursor is overlaid on the user-facing rotation trigger area on the bounding box, the bounding box overlaid on the target virtual object switches to a rotation operation bounding box, that is, a control bar appears on the lower boundary of the user-facing side of the bounding box. Upon detecting a first preset gesture from the user, the second preset cursor overlaid on the target virtual object switches to a rotation operation cursor, prompting the user that the target virtual object can be rotated within this range using rotation operation gestures.

[0084] In some implementations, if the first target control area corresponding to the third preset cursor is the rotation trigger area, the second preset cursor is switched to display as the rotation cursor pattern and rotation gesture pattern corresponding to the rotation trigger area.

[0085] Specifically, please refer to Figure 12 , Figure 12 This illustration shows another virtual scene in this embodiment. When the virtual scene is a 6DOF scene, rotation is possible. The bounding box superimposed on the target virtual object is a cube. A second preset cursor is superimposed on a preset display area 405. In this embodiment, the preset display area 405 can be the boundary line of the bounding box facing the user, parallel to the Y-axis. When the second preset cursor is superimposed on the rotation trigger area of ​​the bounding box facing the user, the bounding box switches to the rotation operation bounding box shown in the illustration. The bounding box superimposed with the second preset cursor and a rotation pattern appears on the rotation trigger area facing the user.

[0086] In the aforementioned virtual scenario, after step S342, if a third preset gesture is detected from the user, the target virtual object enlarges or changes color, and the second preset cursor switches to display the rotation cursor pattern and rotation gesture pattern corresponding to the rotation trigger area. The rotation operation cursor moves within the interactive range following the user's gesture. The interactive control device extracts the displacement value of the user's third preset gesture in the XYZ axis direction and maps it onto the virtual scene to control the target virtual object to achieve the corresponding rotation angle. It should be noted that in the above operation, the vertical center Y-axis of the cube's bounding box is selected as the rotation center to rotate the target virtual object.

[0087] If the cursor for the rotation operation corresponding to the third preset gesture made by the user leaves the interactive range, or if the user makes other preset gestures, the interactive control device will rotate the target virtual object isometrically to the position where the third preset gesture was last detected within the interactive range, thus completing the rotation operation.

[0088] Step S343: Display a fourth preset cursor overlaid on the other preset control areas besides the first target control area in the plurality of preset control areas.

[0089] If the second preset cursor is superimposed on an area other than the aforementioned movement trigger area, scaling trigger area, and rotation trigger area within the interactive range of the target virtual object, the interactive control device will switch the second preset cursor to the fourth preset cursor.

[0090] After step S343, if a third preset gesture is detected from the user, the interactive control device switches the fourth preset cursor to the fifth preset cursor, and the target virtual object is enlarged or its color is changed. In other embodiments, sound feedback can also be set. At this time, the target virtual object in the virtual scene includes, but is not limited to, buttons, drop-down menus, selection, activation, etc. The user can perform, but is not limited to, click, select, activate, and confirm operations on the target virtual object in the above area.

[0091] In this embodiment, as Figure 5 As shown, both the fourth and fifth preset cursors are circular, and the area between the outer and inner circles of the fourth and fifth preset cursors is shaded. The ratio of the outer radius to the inner radius of the fourth preset cursor is greater than the ratio of the outer radius to the inner radius of the fifth preset cursor.

[0092] In this embodiment, by limiting the control of the target virtual object to perform operations such as rotation, movement, scaling, and zooming, the preset cursor corresponding to the gesture needs to be located in different preset control areas within the interactive range of the target virtual object. By restricting the operable range, the user can accurately perform the corresponding preset operation within the preset area, improving the precision of interactive control and avoiding operation control errors. Simultaneously, the control components superimposed on the target virtual object provide prompts for the preset gestures corresponding to the user's preset operations, preventing the user from using incorrect gestures, making the interaction steps clear, and improving the user experience.

[0093] Step S350: If the input target gesture is detected to be a preset gesture under a preset control mode associated with the target cursor, control the target virtual object to perform a preset operation corresponding to the target gesture operation, wherein the target cursor is the preset cursor corresponding to any preset control area among the plurality of preset control areas.

[0094] In this embodiment, the specific implementation of steps S340 to S350 can be found in the content of the foregoing embodiments, and will not be repeated here.

[0095] In this embodiment, after entering the virtual scene, the user selects the target virtual object through the head-mounted device, and the interactive control device controls the first preset cursor to switch to the second preset cursor. At the same time, the user can control the target virtual object to be enlarged or change its color, thereby prompting the user that the target virtual object has been selected. This ensures that the user can clearly select the virtual object to be controlled when operating in the virtual scene, and prevents control deviation.

[0096] Please see Figure 13 , Figure 13 A flowchart illustrating an interactive control method provided in another embodiment of this application is shown.

[0097] Step S510: In response to the first preset gesture, a control component is overlaid on the target virtual object to be controlled. The control component includes multiple preset control areas and a preset cursor corresponding to each preset control area. The preset cursor corresponding to each preset control area is associated with a preset control mode.

[0098] Specifically, please refer to Figure 14 , Figure 14 It shows Figure 13 A flowchart illustrating a sub-step of step S510 in one embodiment. Step S510 may include the following steps:

[0099] Step S511: In response to the first preset gesture, display the target virtual object according to the preset display effect.

[0100] In this embodiment, the virtual cursor controlled by the head-mounted device enters the interactive range of the target virtual object located in the virtual scene. When the interaction control device detects that the user has made a first preset gesture, the target virtual object is correspondingly enlarged or its color changes to indicate that the user has entered the interactive range of the target virtual object. Optionally, the preset display effect is the display effect of controlling the target virtual object to only enlarge or only change its color when the user makes the first preset gesture. It is also possible to control the target virtual object to change its color while enlarging, which is not limited here.

[0101] Step S512: Overlay the control component onto the target virtual object that is displayed with the preset display effect.

[0102] Optionally, a bounding box is overlaid on the target virtual object to indicate the interactive range of the target virtual object to the user, and a second preset cursor is overlaid on the target virtual object. Based on the position of the second preset cursor within the bounding box, when the second preset cursor is overlaid with any preset control area among the movement trigger area, zoom trigger area, and rotation trigger area, the interaction control device switches the original bounding box to the bounding box corresponding to it.

[0103] In another possible implementation, the target virtual scene in which the target virtual object is currently located can also be obtained, and the target virtual scene can be a two-dimensional virtual scene or a three-dimensional virtual scene; and the control components corresponding to the target virtual scene can be superimposed and displayed on the target virtual object.

[0104] Optionally, after displaying the virtual scene, the interactive control device obtains the target virtual scene in which the target virtual object is located, detects the category of the target virtual scene, and thus determines whether the target virtual scene is a two-dimensional virtual scene or a three-dimensional virtual scene.

[0105] When the virtual cursor controlled by the head-mounted device enters the interactive range of the target virtual object in the virtual scene, and the interactive control device detects that the user makes the first preset gesture, the bounding box superimposed on the target virtual object is a rectangle when the target virtual object is in a two-dimensional virtual scene, and a cube when the target virtual object is in a three-dimensional virtual scene.

[0106] In some embodiments, the interactive control device can be configured to display different control components when performing the same movement, zooming, or other operations in a two-dimensional virtual scene and a three-dimensional virtual scene. For example, the control components displayed in a two-dimensional virtual scene can be set to dashed lines, while those displayed in a three-dimensional virtual scene can be set to solid lines, or different colors can be used to distinguish the operations in different virtual scenes.

[0107] Step S520: In response to the target gesture, determine the preset control area where the target gesture is located from the plurality of preset control areas, and use it as the second target control area.

[0108] In this embodiment, the interactive control device detects a user making a target gesture, compares the target gesture with a stored preset gesture, and determines a preset control area associated with the target gesture based on the comparison result. It confirms the position of the target gesture within the interactive range of the target virtual object when the user makes the target gesture, and then uses this preset control area as the second target control area.

[0109] Optionally, in this embodiment, the target gesture made by the user is determined based on the target gesture made, which is a preset control area where the target gesture is located. In other embodiments, the target gesture to be made by the user is determined based on the preset control area where the first preset gesture made by the user is located within the interactive range of the target virtual object.

[0110] Based on this, when users are proficient in using the interactive operations of the current virtual scene, they do not need to wait for the interactive control device to respond after the user makes the first preset gesture and is in the preset control area, prompting the user to make the target gesture. At this time, after the user enters the virtual scene and makes the first preset gesture to interact, they can directly make the target gesture. This avoids the need for the user to wait for the interactive control system to generate preset cursors and other prompts before making gesture operations, simplifying the response process and reducing the user's waiting time.

[0111] Step S530: Obtain the preset cursor corresponding to the second target control area as the target cursor.

[0112] Optionally, after determining the second target control area, the interactive control device obtains the stored preset cursor corresponding to it as the target cursor.

[0113] Step S540: If the input target gesture is detected to be a preset gesture under a preset control mode associated with the target cursor, control the target virtual object to perform a preset operation corresponding to the target gesture operation, wherein the target cursor is the preset cursor corresponding to any preset control area among the plurality of preset control areas.

[0114] In this embodiment, the specific implementation of steps S510 and S540 can be found in the content of the foregoing embodiments, and will not be repeated here.

[0115] In this embodiment, when the user is proficient in using the interactive operations of the current virtual scene, after entering the virtual scene and making the first preset gesture to interact, the user can directly make the target gesture, avoiding the need for the user to wait for the interactive control system to generate a preset cursor or other prompts before making a gesture operation, simplifying the response process and reducing the user's waiting time.

[0116] Please refer to Figure 15 , Figure 15 The diagram illustrates a structural block diagram of an interactive control device 600 according to an embodiment of this application, which is applied to a wearable device. The interactive control device 600 may include a display module 601 and a control module 602.

[0117] The display module 601 is used to respond to a first preset gesture and overlay a control component on the target virtual object to be controlled. The control component includes multiple preset control areas and a preset cursor corresponding to each preset control area. The preset cursor corresponding to each preset control area is associated with a preset control mode.

[0118] The control module 602 is used to control the target virtual object to perform a preset operation corresponding to the target gesture operation if the input target gesture is detected to be a preset gesture under a preset control mode associated with the target cursor. The target cursor is the preset cursor corresponding to any preset control area among the plurality of preset control areas.

[0119] In other embodiments, the display module 601 may be specifically used to display a virtual scene, which includes a first preset cursor and a plurality of first preset virtual objects.

[0120] In this manner, the interactive control device 600 may further include a detection module, which is used to determine the second preset virtual object as the target virtual object if the positional relationship between the first preset cursor and the second preset virtual object satisfies the first preset positional relationship. The second preset virtual object is any one of the multiple first preset virtual objects.

[0121] In this manner, the interactive control device 600 may further include a cursor switching module for switching the first preset cursor to a second preset cursor.

[0122] In other embodiments, a confirmation module is also included, which, in response to the first preset gesture, determines, from among the plurality of preset control areas, the preset control area where the second preset cursor is located, as the first target control area.

[0123] In this mode, the cursor switching module is used to switch the second preset cursor to the third preset cursor corresponding to the first target control area;

[0124] In this mode, the cursor switching module is also used to overlay a fourth preset cursor on the other preset control areas besides the first target control area in the multiple preset control areas.

[0125] In other embodiments, the confirmation module is further configured to, in response to the target gesture, determine from the plurality of preset control areas the preset control area where the target gesture is located, as a second target control area.

[0126] In this mode, the cursor switching module is used to obtain a preset cursor corresponding to the second target control area, and use it as the target cursor.

[0127] In other embodiments, the display module 601 is also used to display the target virtual object according to a preset display effect; and to overlay the control component on the target virtual object displayed with the preset display effect.

[0128] In other embodiments, the display module 601 is further configured to acquire the target virtual scene in which the target virtual object is currently located, the target virtual scene being a two-dimensional virtual scene or a three-dimensional virtual scene; and to overlay and display the control components corresponding to the target virtual scene on the target virtual object.

[0129] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0130] In the several embodiments provided in this application, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0131] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0132] In summary, when a user selects a virtual object in a virtual scene using a head-mounted device and makes a preset hand gesture, the user executes a corresponding preset operation within a preset control area based on prompts from control components superimposed on the virtual object. The head-mounted device and the gesture are linked during the interaction. Furthermore, different preset control methods correspond to different preset control areas. By limiting the execution of preset operations to different preset control areas, high-precision interactive control can be achieved, improving the accuracy of interactive control and reducing the occurrence of miscontrol issues. Additionally, the availability of multiple preset control methods makes human-computer interaction more diverse, enhancing the overall user experience.

[0133] The following will combine Figure 16 This application describes a wearable device 700.

[0134] Reference Figure 16 , Figure 16 The diagram shows a structural block diagram of a wearable device 700 provided in an embodiment of this application. The above-described method provided in this embodiment of the application can be executed by the wearable device 700.

[0135] The wearable device 700 in this application embodiment may include one or more of the following components: processor 701, memory 702, and one or more applications, wherein the one or more applications may be stored in memory 702 and configured to be executed by one or more processors 701, and the one or more applications are configured to perform the methods as described in the foregoing method embodiments.

[0136] Processor 701 may include one or more processing cores. Processor 701 connects to various parts within the wearable device 700 using various interfaces and lines, and performs various functions and processes data of the wearable device 700 by running or executing instructions, programs, code sets, or instruction sets stored in memory 702, and by calling data stored in memory 702. Optionally, processor 701 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 701 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the aforementioned modem can also be integrated into processor 701 and implemented as a separate communication chip.

[0137] The memory 702 may include random access memory (RAM) or read-only memory (ROM). The memory 702 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 702 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the wearable device 700 during use (such as the various correspondences described above).

[0138] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0139] In the several embodiments provided in this application, the coupling or direct coupling or communication connection between the modules shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be electrical, mechanical or other forms.

[0140] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0141] Please refer to Figure 17 , Figure 17 A structural block diagram of a computer-readable storage medium provided in an embodiment of this application is shown. The computer-readable storage medium 800 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0142] The computer-readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 810 may be compressed, for example, in a suitable form.

[0143] In some embodiments, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the steps in the above-described method embodiments.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An interactive control method, characterized in that, Applied to wearable devices, the method includes: In response to a first preset gesture, a control component is overlaid and displayed on the target virtual object to be controlled. The control component includes multiple preset control areas and a preset cursor corresponding to each preset control area. The preset cursor corresponding to each preset control area is associated with a preset control mode. If the input target gesture is detected to be a preset gesture under a preset control mode associated with the target cursor, the target virtual object is controlled to perform a preset operation corresponding to the target gesture operation. The target cursor is the preset cursor corresponding to any preset control area among the multiple preset control areas. Before displaying control components overlaid on the target virtual object to be controlled in response to a first preset gesture, the method further includes: A virtual scene is displayed, which includes a first preset cursor and multiple first preset virtual objects. The user can control the first preset cursor to move in the virtual scene through a wearable device. If the positional relationship between the first preset cursor and the second preset virtual object is detected to satisfy the first preset positional relationship, the second preset virtual object is determined as the target virtual object, and the second preset virtual object is any virtual object among the plurality of the first preset virtual objects; Switch the display of the first preset cursor to the second preset cursor.

2. The method according to claim 1, characterized in that, The response to the first preset gesture, which overlays control components onto the target virtual object to be controlled, includes: In response to the first preset gesture, the preset control area where the second preset cursor is located is determined from the plurality of preset control areas as the first target control area; The second preset cursor is switched to the third preset cursor corresponding to the first target control area, and a preset gesture pattern associated with the third preset cursor is displayed; A fourth preset cursor is superimposed on the other preset control areas besides the first target control area in the plurality of preset control areas.

3. The method according to claim 2, characterized in that, The plurality of preset control areas include at least a movement trigger area, a scaling trigger area, and a rotation trigger area.

4. The method according to claim 3, characterized in that, The third preset cursor includes a cursor pattern and a gesture pattern. Switching the display of the second preset cursor to the third preset cursor corresponding to the first target control area includes: If the first target control area corresponding to the third preset cursor is the movement trigger area, the second preset cursor will be switched to display as the movement cursor pattern and movement gesture pattern corresponding to the movement trigger area. If the first target control area corresponding to the third preset cursor is the zoom trigger area, the second preset cursor will be switched to display as the zoom cursor pattern and zoom gesture pattern corresponding to the zoom trigger area. If the first target control area corresponding to the third preset cursor is the rotation trigger area, the second preset cursor is switched to display as the rotation cursor pattern and rotation gesture pattern corresponding to the rotation trigger area.

5. The method according to claim 1, characterized in that, Before controlling the target virtual object to perform the preset operation corresponding to the target gesture operation if the detected input target gesture is a preset gesture under a preset control mode associated with the target cursor, the method further includes: In response to the target gesture, the preset control area where the target gesture is located is determined from the plurality of preset control areas, and is used as the second target control area; Obtain the preset cursor corresponding to the second target control area and use it as the target cursor.

6. The method according to any one of claims 1-5, characterized in that, The superimposed display control component on the target virtual object to be controlled includes: The target virtual object is displayed according to the preset display effect; The control components are overlaid on the target virtual object that is displayed with the preset display effect.

7. An interactive control device, characterized in that, For use in wearable devices, the device includes: A display module is used to display a virtual scene, which includes a first preset cursor and multiple first preset virtual objects. A user can control the first preset cursor to move within the virtual scene using a wearable device. If the positional relationship between the first preset cursor and a second preset virtual object satisfies a first preset positional relationship, the second preset virtual object is determined as the target virtual object, which is any one of the multiple first preset virtual objects. The first preset cursor is switched to the second preset cursor. In response to a first preset gesture, a control component is overlaid on the target virtual object to be controlled. The control component includes multiple preset control areas and a preset cursor corresponding to each preset control area. Each preset cursor corresponding to a preset control area is associated with a preset control method. The control module is used to control the target virtual object to perform a preset operation corresponding to the target gesture operation if the input target gesture is detected to be a preset gesture under a preset control mode associated with the target cursor, wherein the target cursor is the preset cursor corresponding to any preset control area among the plurality of preset control areas.

8. A wearable device, characterized in that, include: One or more processors; Memory; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 6.

Images