Interaction method, apparatus, electronic device, storage medium, and program product

Abstract

The present disclosure provides an interaction method, device, electronic device, storage medium and program product. Specifically, the interaction method comprises: obtaining a trigger operation of a user; in response to the trigger operation of the user, displaying a first interaction interface in a display screen; wherein the position of the first interaction interface is configured to move with the movement operation of the user; in response to the movement data of the first interaction interface satisfying a preset conversion condition, displaying an identification area in the display screen, and converting the first interaction interface into a second interaction interface displayed at the position of the identification area. By adopting the technical solution of the present disclosure, the user can not only realize quick interaction through the first interaction interface, but also make decisions in a lower power consumption manner through the second interaction interface, thereby improving the convenience of user interaction operation.

Description

Technical Field

[0001] This disclosure relates to the field of extended reality technology, and more particularly to an interactive method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] Extended Reality (XR) refers to the use of computers to combine the real and virtual worlds to create a virtual environment that allows for human-computer interaction. It is also a general term for various technologies such as Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR).

[0003] For extended reality devices, such as VR devices, hand-based interaction without the need for controllers or other grippers is widely used. For hand-based interaction, menus are an indispensable function. However, current technologies struggle to meet user needs in terms of menu access methods. Summary of the Invention

[0004] In view of this, the purpose of this disclosure is to provide an interaction method, apparatus, electronic device, storage medium and program product.

[0005] In view of the above objectives, firstly, this disclosure provides an interaction method, including:

[0006] Get the user's triggered action;

[0007] In response to a user's trigger operation, a first interactive interface is displayed on the screen; wherein the position of the first interactive interface is configured to move with the user's movement operation;

[0008] In response to the movement data of the first interactive interface satisfying the preset conversion conditions, an identification area is displayed on the display screen, and the first interactive interface is converted into a second interactive interface and displayed at the position of the identification area.

[0009] Secondly, embodiments of this disclosure provide an interactive device, including:

[0010] The acquisition module is configured to: acquire user trigger actions;

[0011] The first interaction module is configured to: display a first interaction interface on the display screen in response to a user's trigger operation; wherein the position of the first interaction interface is configured to move with the user's movement operation;

[0012] The second interaction module is configured to: in response to the movement data of the first interaction interface satisfying a preset conversion condition, display an identification area in the display screen and convert the first interaction interface into a second interaction interface displayed at the position of the identification area.

[0013] Thirdly, embodiments of this disclosure provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the interaction method as described in the first aspect.

[0014] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the interaction method as described in the first aspect.

[0015] Fifthly, embodiments of this disclosure provide a computer program product including computer program instructions that, when executed on a computer, cause the computer to perform the interaction method as described in the first aspect.

[0016] As can be seen from the above, the interactive method, device, electronic device, storage medium, and program product provided in this disclosure display a first interactive interface on a display screen by acquiring the user's trigger operation; the position of the first interactive interface can move with the user's movement operation; when the movement data of the first interactive interface meets the preset conversion conditions, an identification area is displayed on the display screen, and the first interactive interface is converted into a second interactive interface displayed at the position of the identification area. At this time, the second interactive interface is fixed in the identification area and is detached from the hand or handle that performed the trigger operation, so that the user can make decisions through the second interactive interface in a lower power consumption manner, thereby improving the convenience of user interaction operation. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this disclosure or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating the effect of an interaction method provided in an embodiment of this disclosure;

[0019] Figure 2 A flowchart illustrating an interaction method provided in an embodiment of this disclosure;

[0020] Figures 3A-3CA schematic diagram illustrating the triggering operation of yet another interaction method provided in this embodiment of the disclosure;

[0021] Figures 4A to 4D A flowchart illustrating another interaction method provided in an embodiment of this disclosure;

[0022] Figure 5 A schematic diagram illustrating the preset conversion range provided for embodiments of this disclosure;

[0023] Figures 6A-6B A schematic diagram illustrating the closing operation of the interaction method provided in this embodiment of the disclosure;

[0024] Figure 7 This is a schematic diagram of an exemplary device provided in the embodiments of this disclosure;

[0025] Figure 8 This is a schematic diagram of the hardware structure of an exemplary electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0027] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this disclosure should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar words used in the embodiments of this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Words such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, but do not exclude other elements or objects.

[0028] As described in the background section, hand-based interaction, without relying on controllers or other devices, is widely used in extended reality devices. Extended reality devices acquire user hand posture data using image acquisition units or smart gloves, and map this posture onto a display screen in head-mounted displays, VR glasses, etc. Users interact with the display screen by controlling their hand posture. Those skilled in the art will understand that the display screen can be a virtual image (e.g., VR) or a combination of real and virtual images (e.g., MR).

[0029] For hands-free interaction, an interactive interface is an indispensable function. For example... Figure 1As shown, the interactive interface is bound to the left hand, while the right hand can select interactive items. This method requires the cooperation of both hands, and the left hand is raised. If there are many menu levels, the user needs to consume a lot of power to select, which affects the user's selection ability and reduces the user experience.

[0030] In view of this, embodiments of this disclosure provide an interaction method, device, electronic device, storage medium, and program product. By acquiring a user's trigger operation, a first interactive interface is displayed on a display screen. The position of the first interactive interface can move with the user's movement operation. When the movement data of the first interactive interface meets a preset conversion condition, an identification area is displayed on the display screen, and the first interactive interface is converted into a second interactive interface displayed at the position of the identification area. At this time, the second interactive interface is fixed in the identification area and detached from the hand or handle that performed the trigger operation, so that the user can make decisions through the second interactive interface in a lower power consumption manner, thereby improving the convenience of user interaction operation.

[0031] Figure 2 A flowchart illustrating the interaction method provided in an embodiment of this disclosure is shown. The interaction method includes:

[0032] Step S201: Obtain the user's trigger operation. Here, different interaction modes correspond to different trigger operations.

[0033] In some embodiments, the interaction method is in a gesture mode; the step of obtaining the user's trigger operation may include:

[0034] The user's first single-hand gesture can be captured; here, image acquisition units or smart gloves can be used to acquire the user's hand posture data.

[0035] As will be understood by those skilled in the art, the image acquisition unit can be set on a head-mounted display device or in a physical space; no specific limitation is made here.

[0036] The method for determining the first single-hand gesture based on posture data (e.g., image data) is existing technology. Those skilled in the art can make reasonable choices based on the type of posture data, and no specific limitations are made here.

[0037] It should be noted that the first single-hand gesture can be a single posture or a combination of postures, without limitation.

[0038] Next, in response to the successful matching of the first single-hand gesture with the trigger gesture, the first single-hand gesture is determined to be the trigger operation.

[0039] Here, the trigger gesture is preset. For example, Figures 3A-3C As shown, the gesture is triggered by Figure 3AThe opening to Figure 3B The gesture is defined as the opening and closing of the hand. When the user's first single-hand gesture is an opening followed by a closing, it can be identified as a trigger gesture, indicating that the user has initiated a trigger operation. Of course, the trigger gesture can also be other gestures, such as the thumb pinching the index finger; this is not a specific limitation here.

[0040] The interaction method of this disclosure is applicable not only to gesture mode, but also to systems that use a controller for interaction.

[0041] Therefore, in some alternative embodiments, when the interaction method is in controller mode, the step of obtaining the user's trigger operation specifically includes:

[0042] The system retrieves the user's trigger action on a first predetermined button on the controller. For example, this first predetermined button could be the Menu button. It should be understood that when the user triggers the first predetermined button, it indicates that the user has performed a trigger action.

[0043] Step S203: In response to the user's trigger operation, display the first interactive interface (e.g., ...) on the display screen. Figure 4A (as shown); wherein the position of the first interactive interface is configured to move with the user's movement operation.

[0044] As mentioned earlier, the displayed image can be shown through a display device. This display device can be a head-mounted display device or smart glasses in an XR (e.g., VR, MR) system. The displayed image can be a virtual image or a combination of virtual and real-world images; there is no limitation on this. It should be understood that the displayed image refers to the image shown through a display device, which can include rendered images, real-world images, and interactive interfaces for user game operations.

[0045] Optionally, the first interactive interface is a shortcut menu. As those skilled in the art will understand, shortcut menus have fewer functions but are more frequently used, facilitating quick user operation and helping to improve interaction efficiency.

[0046] Optionally, the first interactive interface may be displayed above the hand or controller, cover the hand or controller, or in any preset position, without limitation. Displaying the first interactive interface above or covering the hand or controller makes it more intuitive as the hand or controller moves, and helps the user view it.

[0047] Furthermore, the first interactive interface is configured to receive selection operations from a hand or handle that has not yet performed a trigger operation. It should be noted that when the first interactive interface is displayed on the screen, the user can select items on the first interactive interface using the hand or handle that has not yet performed a trigger operation, thereby achieving interaction with the XR device.

[0048] Here, since the first interactive interface moves with the hand or handle that performs the trigger operation, it is more convenient to perform the selection operation with the hand or handle that does not perform the trigger operation, and it can reduce misoperation.

[0049] Optionally, user movement can be achieved by moving the hand or handle that triggers the action. Of course, user movement can also be achieved by moving other parts of the body, such as the head; this is not specifically limited to these methods.

[0050] Step S205: In response to the movement data of the first interactive interface satisfying the preset conversion conditions (e.g.) Figure 4B As shown), an identification area (such as...) is displayed on the display screen. Figure 4C As shown), and convert the first interactive interface into a second interactive interface displayed at the location of the marked area (e.g., Figure 4D (As shown).

[0051] In some embodiments, the position of the marking area is determined based on the position of the left and right hands or the left and right handles during the trigger operation. Those skilled in the art will understand that the position information of the user's left and right hands and the left and right handles can be determined using devices such as image acquisition units and smart gloves. Of course, the position information of the left and right handles can also be determined using their own positioning devices. This disclosure does not specifically limit this aspect.

[0052] It should be noted that the first interactive interface can move with the user's movement operation. In other words, the movement operation can drive the first interactive interface to move. When the movement data meets the preset conversion conditions, the first interactive interface can be converted into a second interactive interface and fixed at the position of the marked area. This allows the user to move more freely without having to maintain a certain posture, reduces the power consumption of the interaction, improves the comfort of the interaction, and helps to improve the user interaction experience.

[0053] In particular, when the user's movement is performed by the hand or handle that triggers the operation, fixing the second interactive interface allows the triggering hand to move freely without having to maintain a single posture, which can greatly improve the power consumption of the triggering hand and increase the ease of operation.

[0054] Optionally, the marking area is located in the middle of the left and right hands or left and right handles. For example, the distance between the marking area and the left hand (left handle) is equal to the distance between the marking area and the right hand (right handle).

[0055] Optionally, the size of the marking area may also exceed the distance between the left and right hands or the left and right handles.

[0056] Optionally, the label area is suspended in the display screen. Optionally, the label area is a feedback box, such as an opaque box.

[0057] As will be understood by those skilled in the art, the position of the identification area in the display screen can be fixed and will not move due to changes in the virtual or real screen.

[0058] In some embodiments, the second interactive interface may be related to the first interactive interface, such as being an expanded version of the first interactive interface. This allows for expansion without requiring individual clicks on the first interactive interface, making it more convenient and improving the user experience. In some embodiments, the second interactive interface may also be a magnified version of the first interactive interface.

[0059] Those skilled in the art will understand that the second interactive interface can also be a preset interface independent of the first interactive interface. For example, the user or system can set the content of the first interactive interface and the content of the second interactive interface.

[0060] Compared to the first interactive interface, the second interactive interface can include more function options, making it easier for users to directly select operations.

[0061] Optionally, some function options in the second interactive interface can be expanded by clicking, which is not limited here.

[0062] Furthermore, the second interactive interface is configured to receive selection operations from a hand or handle that has not yet been triggered. It should be noted that when the second interactive interface is displayed, the user can select items on the interface using the hand or handle that has not been triggered, thereby interacting with the XR device. Here, performing the selection operation using the hand or handle that has not been triggered is more convenient and avoids accidental operation.

[0063] Of course, based on the interactive settings, it can also receive selection operations from the hand or handle to trigger the operation.

[0064] In some embodiments, the preset conversion conditions include a preset conversion range and a preset conversion time. Step S205 specifically includes:

[0065] Please see Figure 4B and Figure 4C In response to the first interactive interface moving to the preset conversion range, an identification area is displayed on the display screen; wherein the preset conversion range is matched with the position of the identification area.

[0066] It should be noted that the matching here refers to a certain correspondence between the preset conversion range and the range of the identifier area. The preset conversion range can be larger or smaller than the size of the identifier area, without any specific limitation.

[0067] Optionally, please refer to Figure 5 In the first direction (e.g., the horizontal direction), the preset conversion range is a certain area between the left and right hands or the left and right handles when the trigger operation is performed (e.g., ...). Figure 5 (As shown by the dashed line in the middle right figure).

[0068] Optionally, the preset conversion range is the area perpendicular to the first direction (e.g., the vertical direction) where the screen displays the field of view in that direction when the operation is triggered, or a specified area of ​​the field of view, such as the middle area (e.g., the area perpendicular to the first direction). Figure 5 (As shown by the dashed lines in the left and middle images).

[0069] In response to the first interactive interface remaining in the preset conversion range for a time not less than the preset conversion time, the first interactive interface is converted into a second interactive interface and displayed at the position of the marked area (e.g., Figure 4D (As shown).

[0070] It should be noted that the preset conversion time can be 200ms, 300ms, etc., and no specific limit is set here.

[0071] In some embodiments, the preset conversion condition further includes a preset gaze position; the interaction method further includes:

[0072] The user's head posture information is acquired; here, the user's head posture information can be obtained through an Inertial Measurement Unit (IMU). As will be understood by those skilled in the art, the inertial measurement unit can be installed in head-mounted display devices, smart glasses, etc., and is not limited here.

[0073] In response to determining, based on the head posture information, that the user's gaze is between the left and right hands or between the left and right handles, i.e., the user's gaze is at a preset gaze position, an identification area is displayed on the display screen; otherwise, the display device is controlled to continue displaying the first interactive interface.

[0074] In some embodiments, when the preset conversion conditions include both a preset conversion range and a preset viewing position, the indicator area will only be displayed on the screen if both conditions are met: the first interactive interface moves to the preset conversion range and the user's gaze is between the left and right hands or the left and right handles. In other words, both conditions are indispensable; the indicator area cannot be displayed if either condition is not met. Of course, in other embodiments, the conditions for displaying the indicator area can be set according to actual needs.

[0075] By determining the user's gaze position, it is possible to further determine whether the user intends to switch to the first interactive interface. If the user's focus is not between their hands, it indicates that the user has no intention of switching to the first interactive interface, thereby improving the accuracy of the interaction.

[0076] In some embodiments, after the step of converting the first interactive interface into a second interactive interface and displaying it at the location of the marked area, the method further includes:

[0077] Get the user's return action; the specific method of the return action can be set here, and is not limited here.

[0078] In response to the user's return operation, the second interactive interface is returned to the first interactive interface and redisplayed on the display screen.

[0079] For example, the interaction method is in gesture mode; the step of obtaining the user's return operation specifically includes:

[0080] Acquire the user's second single-hand gesture; optionally, the second single-hand gesture can be completed within the recognition range of the image acquisition device or near the marked area, which is not limited here.

[0081] In response to the successful match between the second one-handed gesture and the return gesture, the second one-handed gesture is determined to be a return operation.

[0082] Optionally, the return gesture is a closed fist (e.g.) Figure 3B (As shown).

[0083] For example, the interaction method is in a controller mode, and the step of obtaining the user's return operation specifically includes:

[0084] The user triggers a second predetermined button on the controller; wherein the second predetermined button is configured to trigger a return operation when the display screen includes a second interactive interface.

[0085] It should be noted that the second preset button can be a menu button, meaning that the same button can have different functions on different display screens.

[0086] In some embodiments, the interaction method further includes:

[0087] Get the user's close action; the specific method of the close action can be set here, and is not limited here.

[0088] In response to the user's closing action, the display screen stops showing either the first interactive interface or the second interactive interface.

[0089] For example, the interaction method is in gesture mode; the step of obtaining the user's close operation specifically includes:

[0090] Obtain the user's third-hand gesture;

[0091] In response to the successful match between the third one-handed gesture and the close gesture, the third one-handed gesture is determined to be a close operation. Optionally, as follows: Figure 6A and 6B As shown, the closing gesture is to open your hand and place it in front of your field of vision. Figure 6A ), then clench your fist ( Figure 6B Alternatively, when the first interactive interface is displayed, the closing gesture can be an open hand.

[0092] Those skilled in the art will understand that two-handed gestures may also be used, which will not be listed here.

[0093] For example, the interaction method is in a controller mode, and the step of obtaining the user's close operation specifically includes:

[0094] Obtain the motion information of the handle;

[0095] If the motion information matches the motion conditions for a closing operation, then the motion of the handle is determined to be a closing operation. Optionally, the motion condition is shaking the handle.

[0096] It should be noted that the controller can appear in either controller mode or simulated hand mode on the display screen; there is no limitation here.

[0097] It should be noted that the method of this disclosure embodiment can be executed by a single device, such as a head-mounted display device or VR glasses. The method of this embodiment can also be applied in a distributed scenario, where multiple devices cooperate to complete the process. In such a distributed scenario, one of these devices may execute only one or more steps of the method of this disclosure embodiment, and the multiple devices will interact with each other to complete the method described.

[0098] It should be noted that the above description describes some embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] Based on the same inventive concept, corresponding to any of the above-described embodiments, this disclosure also provides an interactive device.

[0100] refer to Figure 7 The interactive device includes:

[0101] Module 701 is configured to: acquire user trigger operations;

[0102] The first interaction module 703 is configured to: display a first interaction interface in a display screen in response to a user's trigger operation; wherein the position of the first interaction interface is configured to move with the user's movement operation;

[0103] The second interaction module 705 is configured to: in response to the movement data of the first interaction interface satisfying a preset conversion condition, display an identification area in the display screen and convert the first interaction interface into a second interaction interface displayed at the position of the identification area.

[0104] In some embodiments, the position of the identification area is determined based on the position of the left and right hands or left and right handles when the operation is triggered.

[0105] In some embodiments, the second interactive interface is an expanded version of the first interactive interface.

[0106] In some embodiments, the interaction method is in a gesture mode; the acquisition module 701 is configured to:

[0107] Get the user's first one-handed gesture;

[0108] In response to the successful matching of the first single-handed gesture and the trigger gesture, the first single-handed gesture is determined to be a trigger operation; or

[0109] The interaction method is in controller mode, and the acquisition module 701 is configured as follows:

[0110] Get the user's trigger operation on the first predetermined button of the controller.

[0111] In some embodiments, the preset conversion conditions include a preset conversion range and a preset conversion time; the second interaction module 705 is further configured to: display an identification area in the display screen in response to the first interaction interface moving to the preset conversion range;

[0112] In response to the first interactive interface staying in the preset conversion range for a time not less than the preset conversion time, the first interactive interface is converted into a second interactive interface and displayed at the position of the marked area;

[0113] The preset conversion range is matched with the position of the identification area.

[0114] In some embodiments, the preset conversion conditions further include a preset gaze position; the acquisition module 701 is further configured to: acquire the user's head posture information;

[0115] The second interaction module 705 is further configured to: in response to determining, based on the head posture information, that the user's gaze is between the left and right hands or the left and right handles, control the display device to display an identification area in the display screen;

[0116] Otherwise, continue displaying the first interactive interface.

[0117] In some embodiments, the first and second interactive interfaces are configured to receive selection operations from a hand or handle that has not performed a trigger operation.

[0118] In some embodiments, a return module is also included, configured to:

[0119] Get the user's response;

[0120] In response to the user's return operation, the second interactive interface is returned to the first interactive interface and redisplayed on the display screen.

[0121] In some embodiments, the interaction method is in a gesture mode; the return module is further configured to: acquire the user's second one-handed gesture;

[0122] In response to the successful match between the second one-handed gesture and the return gesture, the second one-handed gesture is determined to be a return operation; or

[0123] The interaction method is in controller mode, and the return module is further configured as follows:

[0124] The user triggers a second predetermined button on the controller; wherein the second predetermined button is configured to trigger a return operation when the display screen includes a second interactive interface.

[0125] In some embodiments, a shutdown module is also included, configured to: obtain a user's shutdown operation;

[0126] In response to the user's closing action, the display screen stops showing either the first interactive interface or the second interactive interface.

[0127] In some embodiments, the interaction method is in a gesture mode; the closing module is further configured to: acquire the user's third single-hand gesture;

[0128] In response to the successful match between the third one-handed gesture and the close gesture, the third one-handed gesture is determined to be a close operation; or

[0129] The interaction method is in a controller mode, and the closing module is further configured to: acquire the motion information of the controller;

[0130] If the motion information matches the motion conditions for a closing operation, then the motion of the handle is determined to be a closing operation.

[0131] For ease of description, the above apparatus is described in terms of its functions, divided into various modules. Of course, in implementing this disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0132] The apparatus of the above embodiments is used to implement the corresponding interaction method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0133] This disclosure provides an interaction method, including:

[0134] Step S201: Obtain the user's trigger operation;

[0135] Step S203: In response to the user's trigger operation, a first interactive interface is displayed on the display screen; wherein the position of the first interactive interface is configured to move with the user's movement operation;

[0136] Step S205: In response to the movement data of the first interactive interface satisfying the preset conversion conditions, a marker area is displayed on the display screen, and the first interactive interface is converted into a second interactive interface and displayed at the position of the marker area.

[0137] In some embodiments, the position of the identification area is determined based on the position of the left and right hands or left and right handles when the operation is triggered.

[0138] In some embodiments, the second interactive interface is an expanded version of the first interactive interface.

[0139] In some embodiments, the interaction method is in a gesture mode; the step of obtaining the user's trigger operation specifically includes:

[0140] Get the user's first one-handed gesture;

[0141] In response to the successful matching of the first single-handed gesture and the trigger gesture, the first single-handed gesture is determined to be a trigger operation; or

[0142] The interaction method is in controller mode, and the step of obtaining the user's trigger operation specifically includes:

[0143] Get the user's trigger operation on the first predetermined button of the controller.

[0144] In some embodiments, the preset conversion conditions include a preset conversion range and a preset conversion time;

[0145] The step of responding to the movement data of the first interactive interface satisfying a preset conversion condition, displaying a marker area on the display screen, and converting the first interactive interface into a second interactive interface displayed at the position of the marker area specifically includes:

[0146] In response to the first interactive interface moving to the preset conversion range, an identification area is displayed on the display screen;

[0147] In response to the first interactive interface staying in the preset conversion range for a time not less than the preset conversion time, the first interactive interface is converted into a second interactive interface and displayed at the position of the marked area;

[0148] The preset conversion range is matched with the position of the identification area.

[0149] In some embodiments, the preset conversion condition further includes a preset gaze position; the interaction method further includes:

[0150] Obtain the user's head pose information;

[0151] In response to determining, based on the head posture information, that the user's gaze is between the left and right hands or between the left and right handles, an identification area is displayed on the display screen;

[0152] Otherwise, continue displaying the first interactive interface.

[0153] In some embodiments, the first and second interactive interfaces are configured to receive selection operations from a hand or handle that has not performed a trigger operation.

[0154] In some embodiments, after the step of converting the first interactive interface into a second interactive interface and displaying it at the location of the marked area, the method further includes:

[0155] Get the user's response;

[0156] In response to the user's return operation, the second interactive interface is returned to the first interactive interface and redisplayed on the display screen.

[0157] In some embodiments, the interaction method is in a gesture mode; the step of obtaining the user's return operation specifically includes:

[0158] Obtain the user's second single-hand gesture;

[0159] In response to the successful match between the second one-handed gesture and the return gesture, the second one-handed gesture is determined to be a return operation; or

[0160] The interaction method is in controller mode, and the step of obtaining the user's return operation specifically includes:

[0161] The user triggers a second predetermined button on the controller; wherein the second predetermined button is configured to trigger a return operation when the display screen includes a second interactive interface.

[0162] In some embodiments, it also includes:

[0163] Get the user's close action;

[0164] In response to the user's closing action, the display screen stops showing either the first interactive interface or the second interactive interface.

[0165] In some embodiments, the interaction method is in a gesture mode; the step of obtaining the user's close operation specifically includes:

[0166] Obtain the user's third-hand gesture;

[0167] In response to the successful match between the third one-handed gesture and the close gesture, the third one-handed gesture is determined to be a close operation; or

[0168] The interaction method is in controller mode, and the step of obtaining the user's close operation specifically includes:

[0169] Obtain the motion information of the handle;

[0170] If the motion information matches the motion conditions for a closing operation, then the motion of the handle is determined to be a closing operation.

[0171] This disclosure also provides an interactive device, including:

[0172] Module 701 is configured to: acquire user trigger operations;

[0173] The first interaction module 703 is configured to: display a first interaction interface in a display screen in response to a user's trigger operation; wherein the position of the first interaction interface is configured to move with the user's movement operation;

[0174] The second interaction module 705 is configured to: in response to the movement data of the first interaction interface satisfying a preset conversion condition, display an identification area in the display screen and convert the first interaction interface into a second interaction interface displayed at the position of the identification area.

[0175] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the interaction method described in any of the above embodiments.

[0176] Figure 8 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0177] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0178] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0179] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0180] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0181] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0182] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0183] The electronic devices described above are used to implement the corresponding interaction methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0184] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the interaction method as described in any of the above embodiments.

[0185] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0186] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the interactive method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0187] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this disclosure also provides a computer program product, which includes a computer program. In some embodiments, the computer program is executed by one or more processors to cause the processors to perform the interactive methods described in the above embodiments.

[0188] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this disclosure as described above, which are not provided in detail for the sake of brevity.

[0189] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this disclosure, the provided drawings may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this disclosure, and this also takes into account the fact that the details of implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this disclosure will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuitry) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this disclosure may be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0190] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0191] This disclosure is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An interaction method, characterized in that, include: Get the user's triggered action; In response to a user's trigger operation, a first interactive interface is displayed on the screen; wherein the position of the first interactive interface is configured to move with the user's movement operation; In response to the movement data of the first interactive interface satisfying a preset conversion condition, an identification area is displayed on the display screen, and the first interactive interface is converted into a second interactive interface displayed at the position of the identification area; wherein, the second interactive interface is fixed to the identification area.

2. The interaction method according to claim 1, characterized in that, The location of the marked area is determined based on the position of the left and right hands or left and right handles when the operation is triggered.

3. The interaction method according to claim 1, characterized in that, The second interactive interface is an expanded version of the first interactive interface.

4. The interaction method according to claim 1, characterized in that, The interaction method is a gesture mode; the step of obtaining the user's trigger operation specifically includes: Get the user's first one-handed gesture; In response to the successful matching of the first single-handed gesture and the trigger gesture, the first single-handed gesture is determined to be a trigger operation; or The interaction method is in controller mode, and the step of obtaining the user's trigger operation specifically includes: Get the user's trigger operation on the first predetermined button of the controller.

5. The interaction method according to claim 1, characterized in that, The preset conversion conditions include a preset conversion range and a preset conversion time; The step of displaying a marker area on the display screen and converting the first interactive interface into a second interactive interface displayed at the position of the marker area in response to the movement data of the first interactive interface satisfying the preset conversion conditions specifically includes: In response to the first interactive interface moving to the preset conversion range, an identification area is displayed on the display screen; In response to the first interactive interface staying in the preset conversion range for a time not less than the preset conversion time, the first interactive interface is converted into a second interactive interface and displayed at the position of the marked area; The preset conversion range is matched with the position of the identification area.

6. The interaction method according to claim 5, characterized in that, The preset conversion conditions also include a preset gaze position; the interaction method also includes: Obtain the user's head pose information; In response to determining, based on the head posture information, that the user's gaze is between the left and right hands or between the left and right handles, an identification area is displayed on the display screen; Otherwise, continue displaying the first interactive interface.

7. The interaction method according to claim 1, characterized in that, The first and second interactive interfaces are configured to receive selection operations from a hand or handle that has not performed a trigger operation.

8. The interaction method according to claim 1, characterized in that, After the step of converting the first interactive interface into a second interactive interface and displaying it at the location of the marked area, the method further includes: Get the user's response; In response to the user's return operation, the second interactive interface is returned to the first interactive interface and redisplayed on the display screen.

9. The interaction method according to claim 8, characterized in that, The interaction method is a gesture mode; the step of obtaining the user's return operation specifically includes: Obtain the user's second single-hand gesture; In response to the successful match between the second one-handed gesture and the return gesture, the second one-handed gesture is determined to be a return operation; or The interaction method is in controller mode, and the step of obtaining the user's return operation specifically includes: The user triggers a second predetermined button on the controller; wherein the second predetermined button is configured to trigger a return operation when the display screen includes a second interactive interface.

10. The interaction method according to claim 1, characterized in that, Also includes: Get the user's close action; In response to the user's closing action, the display screen stops showing either the first interactive interface or the second interactive interface.

11. The interaction method according to claim 10, characterized in that, The interaction method is a gesture mode; the step of obtaining the user's close operation specifically includes: Obtain the user's third-hand gesture; In response to the successful match between the third one-handed gesture and the close gesture, the third one-handed gesture is determined to be a close operation; or The interaction method is in controller mode, and the step of obtaining the user's close operation specifically includes: Obtain the motion information of the handle; If the motion information matches the motion conditions for a closing operation, then the motion of the handle is determined to be a closing operation.

12. An interactive device, characterized in that, include: The acquisition module is configured to: acquire user trigger actions; The first interaction module is configured to: display a first interaction interface on the display screen in response to a user's trigger operation; wherein the position of the first interaction interface is configured to move with the user's movement operation; The second interaction module is configured to: in response to the movement data of the first interaction interface satisfying a preset conversion condition, display an identification area in the display screen and convert the first interaction interface into a second interaction interface displayed at the position of the identification area; wherein the second interaction interface is fixed to the identification area.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the interaction method as described in any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the interactive method according to any one of claims 1 to 11.

15. A computer program product, characterized in that, It includes computer program instructions that, when executed on a computer, cause the computer to perform the interaction method as described in any one of claims 1 to 11.

Images