Interaction method and apparatus, storage medium, and electronic device

Abstract

Embodiments of the present disclosure provide an interaction method and apparatus, a storage medium, and an electronic device. The interaction method comprises: in a first navigation mode, switching the first navigation mode to a second navigation mode when a switching condition is met, the first navigation mode performing navigation on the basis of first interaction information, and the second navigation mode performing navigation on the basis of second interaction information, wherein a navigation starting point in the second navigation mode is determined on the basis of a navigation end point in the first navigation mode when the first navigation mode is switched to the second navigation mode.

Description

Interaction method, device, storage medium and electronic device

[0001] This application claims priority to Chinese Patent Application No. 202411281504.5, filed on September 12, 2024, the disclosure of which is incorporated herein in its entirety as part of the present application. TECHNICAL FIELD

[0002] The present disclosure provides an interaction method, device, storage medium and electronic device. BACKGROUND

[0003] A table, such as a table in a table document or a table in a text document, has cells corresponding to fields. A field is a property of a row or a column of cells, which describes the type of content that can be carried in the cells with the property. The field is usually displayed in the form of a column name. For example, a date field indicates that the content carried in the cell with the field is a date.

[0004] With the continuous development of immersive technologies such as XR (Extended Reality), XR devices are accepted and used by a large number of users.

[0005] In the immersive navigation scene of extended reality, the navigation flexibility is poor by a single navigation mode, and the operator needs high attention in the single navigation mode, which can cause the operator to be tired. SUMMARY

[0006] The present disclosure provides an interaction method, device, storage medium and electronic device.

[0007] In a first aspect, the embodiments of the present disclosure provide an interaction method, comprising:

[0008] In the first navigation mode, the first navigation mode is switched to a second navigation mode in a case where a switching condition is met, the first navigation mode performs navigation according to first interaction information, and the second navigation mode performs navigation according to second interaction information;

[0009] The navigation starting point in the second navigation mode is determined according to the navigation ending point in the first navigation mode when the first navigation mode is switched to the second navigation mode.

[0010] In a second aspect, the embodiments of the present disclosure further provide an interaction device, comprising:

[0011] The navigation mode switching module is configured to, in the first navigation mode, switch the first navigation mode to a second navigation mode when a switching condition is met, the first navigation mode being navigated according to first interaction information, and the second navigation mode being navigated according to second interaction information.

[0012] The navigation starting point in the second navigation mode is determined according to a navigation ending point in the first navigation mode when the first navigation mode is switched to the second navigation mode.

[0013] In a third aspect, the embodiments of the present disclosure further provide an electronic device, which comprises:

[0014] one or more processors;

[0015] a storage device configured to store one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the interaction method provided by the embodiments of the present disclosure.

[0017] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium containing computer executable instructions, which, when executed by a computer processor, are configured to perform the interaction method provided by the embodiments of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent upon reading the following detailed description in conjunction with the accompanying drawings, in which like reference numerals refer to like elements. Throughout the drawings, the same or similar elements are denoted by the same or similar reference signs. It should be understood that the drawings are schematic, and the elements and features are not necessarily drawn to scale.

[0019] FIG. 1 is a flow diagram of an interaction method according to an embodiment of the present disclosure;

[0020] FIG. 2 is a schematic diagram of navigation mode switching according to an embodiment of the present disclosure;

[0021] FIG. 3 is a flow diagram of an interaction method according to an embodiment of the present disclosure;

[0022] FIG. 4 is a schematic diagram of a navigation process according to an embodiment of the present disclosure;

[0023] FIG. 5 is a schematic diagram of a navigation process according to an embodiment of the present disclosure;

[0024] FIG. 6 is a schematic diagram of an interaction device according to an embodiment of the present disclosure; and

[0025] FIG. 7 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted as being limited to the embodiments set forth herein; rather, these embodiments are provided so that the present disclosure can be more thoroughly and completely understood.

[0027] It should be understood that each of the steps recited in the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0028] The term “comprising” and variations thereof as used in the present disclosure are open-ended, that is, “including but not limited to”. The term “based on” is “based, at least in part, on”. The term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment”; the term “some embodiments” means “at least some embodiments”. Related definitions will be given in the description below.

[0029] It should be noted that the terms “first”, “second”, and the like in the present disclosure are merely used to distinguish different devices, modules or units, and do not imply the order or interdependence of the functions performed by these devices, modules or units.

[0030] It should be noted that the terms “one”, “multiple” in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that “one or more” should be understood unless otherwise explicitly indicated in the context.

[0031] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are merely for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0032] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, scope of use, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained in accordance with relevant laws and regulations.

[0033] For example, in response to receiving an active request of a user, a prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed by the user will require obtaining and using personal information of the user. Thus, the user can autonomously select whether to provide the personal information to the software or hardware, such as an electronic device, an application program, a server or a storage medium, performing the operation of the technical solution of the present disclosure according to the prompt information.

[0034] As an optional but non-limiting implementation, in response to receiving an active request of a user, the prompt information can be sent to the user in the form of a pop-up window, in which the prompt information can be presented in the form of text. In addition, the pop-up window can also carry a selection control for the user to select "agree" or "disagree" to provide the personal information to the electronic device.

[0035] It can be understood that the above notification and obtaining of user authorization process is only illustrative and does not limit the implementation of the present disclosure, and other ways that meet the relevant laws and regulations can also be applied to the implementation of the present disclosure.

[0036] It can be understood that the data involved in the present disclosure (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the relevant laws and regulations and the relevant provisions.

[0037] FIG. 1 is a flow diagram of an interaction method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to a situation of switching an interaction mode by detecting a hand pose of a control person in an extended reality scene. The method can be executed by an interaction device, which can be implemented in the form of software and / or hardware. Optionally, the interaction device can be implemented by an electronic device, which can include but is not limited to a mobile terminal, a PC and an extended reality device, etc. The extended reality device can include but is not limited to an augmented reality device (AR), a virtual reality device (VR) and a mixed reality device (MR), etc. Optionally, the extended reality device is a head-mounted extended reality device, which is easy to wear and can free both hands.

[0038] As shown in FIG. 1, the method includes:

[0039] S110, navigation is performed in a first navigation mode, the first navigation mode being navigated according to first interaction information.

[0040] S120, in a case where a switching condition is met, the first navigation mode is switched to a second navigation mode, the second navigation mode being navigated according to second interaction information.

[0041] The navigation starting point in the second navigation mode is determined according to a navigation ending point in the first navigation mode when the first navigation mode is switched to the second navigation mode.

[0042] The navigation in the embodiment can be understood as a process in which a user intends to move between interactive objects, where the interactive objects can be elements of different scales such as interfaces, controls, and the like. For example, the user intends to switch from a first display page to a second display page; for example, the user intends to move in the same display page, which can include multiple controls, and the user intends to move from a first control to a second control.

[0043] In the above navigation process, the user's intention can be displayed through switching or different display of interactive objects. The different display here can be, for example, highlighting, which is used to display the interactive object corresponding to the user's intention. For example, in the process of the user's intention moving between controls in the display page, for example, the user's intention moves from the first control to the second control, and the display effect in the display page can be that the first control changes from the highlighted state to the non-highlighted state, and the second control changes from the non-highlighted state to the highlighted state.

[0044] The user's intention can also be represented by a visual display marker, for example, the visual display marker can be a cursor located in the display page. The movement of the cursor can represent the movement of the user's intention. For example, the user's intention moves from the first control to the second control, which can be represented in the display page as the cursor moving from the location of the first control to the location of the second control. It can be understood that the above-mentioned display page can be a display page displayed on a display component of an electronic device such as a mobile phone or a PC, or a virtual display page displayed by an extended reality device.

[0045] The navigation mode can be understood as a way to realize the user's intention to move between the interactive objects. Exemplarily, the navigation mode includes, but is not limited to, an eye movement navigation mode, a head movement navigation mode, a gesture navigation mode, a touch navigation mode, an input device navigation mode, and a voice navigation mode, each of which is based on different interaction information for navigation. Among them, the eye movement navigation mode is based on eye movement tracking information for navigation. The eye movement tracking information of the control personnel is collected by the camera equipped on the electronic device, and the user's intention is represented by the measurement information of the eye movement, the eye position and the gaze point in the eye movement tracking information to realize navigation. Exemplarily, the left movement of the eye position in the eye movement tracking information can represent the user's intention to move to the left between the interactive objects. The head movement navigation mode is based on head tracking information for navigation. The head tracking information of the control personnel can be collected by the camera equipped on the electronic device, or when the electronic device is a head-mounted device, the head tracking information can be detected by the pose detection unit equipped on the head-mounted device. The user's intention is represented by the head movement, the head orientation and other information in the head tracking information to realize navigation. Exemplarily, the left rotation of the head in the head tracking information can represent the user's intention to move to the left between the interactive objects. The gesture navigation mode is based on gesture tracking information for navigation. The gesture tracking information is detected by the camera equipped on the electronic device, and the user's intention is represented by the gesture tracking information to realize navigation. Exemplarily, the left movement of the hand in the gesture tracking information can represent the user's intention to move to the left between the interactive objects. The touch navigation mode is navigated by touch information. The electronic device can be equipped with a touch screen. The touch information is generated by detecting the touch operation of the control personnel on the touch screen, and the user's intention is represented by the touch information to realize navigation. Exemplarily, the touch information can be the sliding information or clicking information of the hand or touch pen and the like on the touch screen. When the touch information is left sliding, it can represent the user's intention to move to the left between the interactive objects. The input device navigation mode is based on the input information of the input device configured on the electronic device for navigation. The input device includes, but is not limited to, a mouse, a keyboard, a remote controller and the like. The input information corresponding to the mouse can be the movement information or clicking information of the mouse. The input information of the keyboard and the remote controller can be the direction movement information input by the direction keys. The user's intention is represented by the above-mentioned input information to realize navigation. Exemplarily, when the above-mentioned input information is left movement, it can represent the user's intention to move to the left between the interactive objects. The voice navigation mode can be that the voice information of the control personnel is detected by the voice acquisition unit configured on the electronic device, and the user's intention is represented by the voice information to realize navigation. Exemplarily, the voice information is "move to the left", which can represent the user's intention to move to the left between the interactive objects.

[0046] It can be understood that the controller can be an operating user of the electronic device, and the controller can be a user who is pre-registered and has an operating permission, so as to distinguish a user who does not have the operating permission and avoid the situation that the navigation process is disturbed by other persons.

[0047] In each of the above navigation modes, interaction information corresponding to the navigation mode needs to be collected, and accordingly, the controller needs to interact with the electronic device in a specific manner, so that the electronic device collects the interaction information. For example, in the eye movement navigation mode, the controller interacts with the electronic device through eye movement, so that the electronic device collects eye movement tracking information. In the gesture navigation mode, the controller interacts with the electronic device through the hand, so that the electronic device collects gesture tracking information, and so on. Therefore, in the case that the duration of any navigation mode is long, the controller will consume a large amount of energy and feel tired.

[0048] To solve the above problems, the navigation mode is switched during navigation, and the interaction information required to be detected during navigation is switched through the navigation mode, so that the controller can interact with the electronic device in different ways, improve the flexibility and diversity of interaction, and avoid the problem that the controller consumes a large amount of energy and feels tired due to a single interaction mode.

[0049] The electronic device starting navigation can be triggered by the electronic device starting or a preset application program in the electronic device starting, etc. The first navigation mode can be pre-set and determined according to initial navigation requirements, for example, the gesture navigation mode. The first navigation mode can also be selected by the controller from among the multiple navigation modes, which is not limited here. The first navigation mode can be the currently used navigation mode, for example, the first navigation mode or the navigation mode determined after mode switching. Here, the first navigation mode is any one of the eye movement navigation mode, the head movement navigation mode, the gesture navigation mode, the touch navigation mode, the input device navigation mode, and the voice navigation mode.

[0050] Based on the first navigation mode, the first interaction information is acquired, and navigation is performed through the first interaction information. The switching condition of the navigation mode is pre-set, and the switching of the navigation mode is performed when it is detected that the switching condition is met. Specifically, the first navigation mode is switched to the second navigation mode, wherein the second navigation mode is any one of the eye movement navigation mode, the head movement navigation mode, the gesture navigation mode, the touch navigation mode, the input device navigation mode, and the voice navigation mode, and the second navigation mode is different from the first navigation mode, that is, the first interaction information corresponding to the first navigation mode is different from the second interaction information corresponding to the second navigation mode.

[0051] In some embodiments, the switching condition is a pre-set fixed condition, which can be a voice switching condition such as "mode switching" input in voice form, or a switching condition that the mode switching control or key is clicked, or a pre-set gesture switching condition (e.g. a specific gesture), or a pre-set eye movement switching condition or head movement switching condition (e.g. specific eye movement information or specific head movement information), etc. The navigation mode is switched when the pre-set switching condition is detected. Optionally, the second navigation mode can be randomly selected from other navigation modes except the first navigation mode. Optionally, the switching sequence of the navigation modes is pre-set, and the second navigation mode is determined based on the first navigation mode and the switching sequence, for example, the switching sequence can be eye movement navigation mode-head movement navigation mode-gesture navigation mode-eye movement navigation mode-touch control navigation mode-input device navigation mode-voice navigation mode, in the case that the first navigation mode is eye movement navigation mode, the second navigation mode is head movement navigation mode; in the case that the first navigation mode is gesture navigation mode, the second navigation mode is eye movement navigation mode, and so on.

[0052] In some embodiments, the switching condition is a pre-set plurality of conditions, and different switching conditions correspond to different second navigation modes. For example, the eye movement navigation mode corresponds to switching condition 1, the gesture navigation mode corresponds to switching condition 2, and the head movement navigation mode corresponds to switching condition 3. In the case that the switching condition 1 is detected, the second navigation mode is determined to be eye movement navigation mode, and in the case that the switching condition 2 is detected, the second navigation mode is determined to be gesture navigation mode. The different switching conditions can be pre-set and not specifically limited, and can be pre-set by the user. For example, the different switching conditions can be different gestures, or the different switching conditions can be selection operations on different navigation mode identifiers (or navigation mode controls), or the different switching conditions can be different voice information, etc.

[0053] In some embodiments, the switching condition corresponding to different navigation modes can be the detection of the interaction information corresponding to the navigation mode, i.e. in the case that the second interaction information is detected, it is determined that the switching condition corresponding to the second navigation mode is met, and the first navigation mode is switched to the second navigation mode. For example, in the voice navigation mode, the gesture tracking information is detected, i.e. the switching condition is met, and the voice navigation mode is switched to the gesture navigation mode. For example, in the eye movement navigation mode, the gesture tracking information is detected, i.e. the switching condition is met, and the eye movement navigation mode is switched to the gesture navigation mode.

[0054] Optionally, the switching condition is determined to be met when the second interaction information is detected and a holding time length of the second interaction information exceeds a preset time length. By setting the holding time length of the second interaction information, a certain preparation time length is provided for the control personnel to adapt to the navigation mode switching, and a situation of false navigation mode switching is avoided. For example, when gesture tracking information is detected in the eye movement navigation mode and the holding time length of the gesture tracking information exceeds the preset time length, the switching condition is determined to be met, and the eye movement navigation mode is switched to the gesture navigation mode.

[0055] Optionally, an identifier of the navigation mode can be marked in the display page. For example, when the second navigation mode after the switching is the gesture navigation mode, an identifier of the gesture navigation mode, for example, a hand graphical identifier, can be marked in the display page; when the second navigation mode after the switching is the eye movement navigation mode, an identifier of the eye movement navigation mode, for example, an eye graphical identifier, can be marked in the display page. By marking the identifier of the current navigation mode in the display page, the corresponding interaction mode of the control personnel is prompted. Optionally, after the navigation mode is successfully switched, the identifier of the navigation mode in the display page is updated, so as to prompt the corresponding interaction mode to be changed, and confusion in the navigation process is avoided.

[0056] The technical scheme provided in the embodiments of the present disclosure is that, in the first navigation mode, the first navigation mode is switched to the second navigation mode when the switching condition is met. When the switching condition is not met, it is indicated that the switching intention of the control personnel is not detected, and the first navigation mode is continued. By switching the navigation mode when the switching condition is met, navigation is realized through different interaction modes, the flexibility of the interaction mode in the navigation process is improved, and the fatigue caused by the control personnel navigating through a single interaction mode is reduced.

[0057] In some embodiments, the first navigation mode is an eye movement navigation mode and / or a head movement navigation mode, and correspondingly, the first interaction information is eye movement tracking information and / or head movement tracking information; the second navigation mode is a gesture navigation mode, and correspondingly, the second interaction information is gesture tracking information. For example, in the first navigation mode, when the switching condition is met during navigation based on the eye movement tracking information and / or the head movement tracking information, the first navigation mode is switched to the second navigation mode, and navigation is realized based on the gesture tracking information in the second navigation mode.

[0058] The hand of the control person and the gesture of the hand are detected, and it is determined whether there is a switching intention of the navigation mode. Since the control person needs to provide higher attention in navigation by means of eye movement tracking information and / or head tracking information relative to the hand, the control person can switch the navigation mode to the gesture navigation mode when the duration of the eye movement navigation mode or the head movement navigation mode exceeds the duration threshold. In addition, since the gesture tracking information is more flexible and diverse relative to the eye movement tracking information and / or the head tracking information, and more accurate control operations can be achieved, the control person can switch the navigation mode to the gesture navigation mode through the eye movement navigation mode or the head movement navigation mode according to the control requirement when accurate control operations need to be performed.

[0059] Here, the switching condition can be understood as the hand of the control person being in an interaction preparation state, and the hand being in a control preparation state can be understood as being able to interact through the hand so that the electronic device detects gesture tracking information to achieve navigation. The switching condition can include a hand position condition and / or a gesture condition. The hand position condition can be understood as a condition for limiting the position of the hand in the control preparation state, and the gesture condition can be understood as a condition for limiting the posture of the hand in the control preparation state.

[0060] Optionally, the switching condition can be that the gesture tracking information is detected. Specifically, when the included angle between the line connecting the first key point of the hand and the head and the head orientation is less than a preset angle threshold, and / or the distance between the second key point and the third key point of the hand is less than a preset distance threshold, it is determined that the switching condition is met.

[0061] The included angle between the line connecting the first key point of the hand and the head and the head orientation is less than a preset angle threshold, which can be used to limit the position of the hand and can be understood as a hand position condition. In this embodiment, the position of the first key point of the hand is detected as the position of the hand of the control person, wherein the first key point of the hand can include but is not limited to the palm, the middle finger knuckle, or the position between the thumb and the index finger, etc. Here, the first key point of the hand can be identified by a pre-set key point identification model, and the position of the first key point of the hand is determined. The key point identification model can be a pre-trained machine learning model, such as a deep learning model, etc.

[0062] The position of the head of the control person and the orientation of the head are identified, for example, the position of the head of the control person and the orientation of the head can also be determined by a key point identification model, for example, the center point position of the head can be determined as the position of the head or the position of the nose on the head can be determined as the position of the head, and the determination method of the position of the head is not limited here. The line connecting the position of the first key point of the hand and the position of the head is obtained, that is, the line connecting the first key point of the hand and the head. The head orientation can be the front orientation of the control person.

[0063] An angle between the line connecting the first key point of the hand and the head and the head orientation is determined, and the angle is compared with a preset angle threshold. If the angle is greater than the preset angle threshold, it is determined that the switching condition is not met, and if the angle is less than the preset angle threshold, it is determined that the switching condition is met. The preset angle threshold can be preset, for example, 80°, 75°, etc. In this embodiment, whether the hand position of the controller meets the control preparation state is determined by the angle between the line connecting the head and the hand of the controller and the head orientation.

[0064] The distance between the second key point and the third key point of the hand is less than a preset distance threshold, which can be used to limit the hand gesture and can be understood as a hand gesture condition. The second key point and the third key point of the hand can be the positions of two different finger tips, and the two fingers can be two fingers that can be pinched, such as the thumb and the index finger, or the thumb and the middle finger, etc. For example, the second key point of the hand can be the position of the thumb tip, and the third key point of the hand can be the position of the tip of any one or more fingers other than the thumb. The positions of the second key point and the third key point of the hand are recognized, and the distance between the second key point and the third key point is determined based on the positions of the second key point and the third key point. The distance is compared with the preset distance threshold, and if the distance is less than the preset distance threshold, the switching condition is met; if the distance is greater than the preset distance threshold, the switching condition is not met. The preset distance threshold can be preset, for example, 5 cm or 6 cm, etc. In this embodiment, whether the hand gesture meets the control preparation state is determined by the distance between the key points of the hand of the controller.

[0065] Optionally, the hand position and the hand gesture are simultaneously determined to meet the switching condition, the accuracy of the control preparation state determination is improved, and the situation of the navigation mode being switched by mistake is avoided.

[0066] In the above embodiments, the switching condition can also include that the height of the hand of the controller meets a preset height range.

[0067] In the above embodiments, the switching condition can also be that the angle between the line connecting the first key point of the hand and the head and the head orientation is less than a preset angle threshold, and / or the distance between the second key point and the third key point of the hand is less than a preset distance threshold, and the holding time exceeds a preset time. Specifically, the first navigation mode can be switched to the second navigation mode when the timing reaches the preset time after detecting the hand gesture tracking information. By setting the determination of the holding time of the hand gesture tracking information, a preparation time is provided for the controller, and the smooth transition of the process of switching to the gesture navigation mode is ensured.

[0068] In some embodiments, the first navigation mode is a gesture navigation mode, and the first control part corresponding to the first navigation mode is a hand; the second navigation mode is an eye movement navigation mode or a head navigation mode, and the second control part corresponding to the second navigation mode is an eye or a head; accordingly, when the virtual object is in the gesture navigation mode, the pose of the set part of the control person is detected to obtain a pose detection result; and when the pose detection result meets a second switching condition, the gesture navigation mode is switched to the eye movement navigation mode or the head navigation mode.

[0069] The second switching condition can be understood as that the hand of the control person is out of a control preparation state, for example, any one of the position of the hand of the control person and the posture of the hand is not in the control preparation state, indicating that there is a switching intention of switching the gesture navigation mode to the eye movement navigation mode or the head navigation mode. The hand of the control person is out of the control preparation state may be due to the fatigue of the hand, and by switching the gesture navigation mode to the eye movement navigation mode or the head navigation mode, the effect of relieving the fatigue of the hand is achieved.

[0070] For example, referring to FIG. 2, FIG. 2 is a schematic diagram of navigation mode switching provided by an embodiment of the present disclosure. The dots in the display page in FIG. 2 represent the intention of the user, and the cursor in the display page represents the intention of the user. The first navigation mode is an eye movement navigation mode, navigation is performed by detecting eye movement tracking information, the gesture tracking information of the control person is detected, and the eye movement navigation mode is switched to a gesture navigation mode when the gesture tracking information of the control person meets a switching condition. Navigation is performed by detecting gesture tracking information.

[0071] In the above embodiment, after switching to the second navigation mode, in the second navigation mode, second interaction information is detected, and navigation is performed based on the second interaction information. The navigation starting point in the second navigation mode is determined based on the navigation ending point in the first navigation mode when the first navigation mode is switched to the second navigation mode. For example, the navigation ending point in the first navigation mode can be used as the navigation ending point in the first navigation mode. For example, when the navigation position in the first navigation mode is position A, the first navigation mode is switched to the second navigation mode when the switching condition is met, and in the second navigation mode, position A is used as the navigation starting point to continue navigation based on the detected second interaction information. In the process of navigation mode switching, the navigation position does not change, and after the navigation mode is switched, the navigation is continued from the navigation ending point before the navigation mode is switched, the navigation is connected, and the interference of the navigation mode switching on the navigation process is avoided.

[0072] In the above embodiments, after switching from the first navigation mode to the second navigation mode, the second navigation mode is switched to the first navigation mode when a set condition is met. The set condition can be a voice condition such as "navigation mode withdrawal", or the set condition can be that a navigation mode withdrawal button or control is triggered, or the set condition can be that the second interaction information cannot be detected. Taking the second navigation mode as a gesture navigation mode and the second interaction information as gesture tracking information as an example, the second navigation mode is switched to the first navigation mode when the gesture tracking information cannot be detected. Specifically, when a hand cannot be detected, it is determined that the gesture tracking information cannot be detected. Taking the second navigation mode as an eye movement navigation mode and the second interaction information as eye movement tracking information as an example, the second navigation mode is switched to the first navigation mode when the eye movement tracking information cannot be detected. Specifically, when the control person closes his eyes or the control person's head is away from the detection range, the eye movement tracking information cannot be detected. Taking the second navigation mode as an input device navigation mode as an example, when the input device is disconnected from the electronic device, the interaction information of the input device cannot be detected, and the second navigation mode is switched to the first navigation mode. The first navigation mode is the navigation mode before switching to the second navigation mode.

[0073] In some embodiments, in response to receiving an interaction instruction, an operation corresponding to the interaction instruction is performed, and the second navigation mode is switched to the first navigation mode. Optionally, at least one operation identifier is set in advance, and each operation identifier is used to mark an operation. The operation corresponding to the operation identifier can be used to trigger reverse switching of the navigation mode, i.e., switching the second navigation mode to the first navigation mode. Here, the specific form of the interaction instruction corresponding to the operation set in advance is not limited. After switching the first navigation mode to the second navigation mode, the operation corresponding to the interaction instruction is performed, and if the operation can be used to trigger reverse switching of the navigation mode, the second navigation mode is switched to the first navigation mode. Optionally, at least one interaction instruction is set in advance to trigger reverse switching of the navigation mode. Here, the type of operation corresponding to the interaction instruction set in advance is not limited. When the interaction instruction set in advance is detected, the operation corresponding to the interaction instruction is performed, and reverse switching of the navigation mode is triggered. Here, the operation and reverse switching of the navigation mode can be performed synchronously or sequentially. The interaction instruction can be determined based on the second interaction information. For example, the interaction instruction can be generated when a finger pinching gesture of a hand is detected, or when a click operation of a mouse is detected, or when a confirmation button click operation of a keyboard or a handle is detected, or when a blinking action of an eye is detected, or when a set voice content is detected.

[0074] In the embodiment, the second navigation mode is switched to the first navigation mode in a case where an operation corresponding to the interaction instruction is performed, and the reverse switching process of the navigation mode can be simplified, and the control personnel does not need to perform the reverse switching of the navigation mode through other interaction information.

[0075] Correspondingly, after being switched from the second navigation mode to the first navigation mode, the navigation is continued based on the first navigation mode, and a navigation starting point based on which the navigation is continued based on the first navigation mode is determined based on a navigation ending point in the second navigation mode when the second navigation mode is switched to the first navigation mode, so as to realize the connection of the navigation after the switching of the navigation mode.

[0076] FIG. 3 is a flowchart of an interaction method provided in the embodiment of the present disclosure. Based on the above-mentioned embodiment, the process of performing the navigation in the second navigation mode is provided. Optionally, the set condition of the second control part is detected, and the virtual object is controlled to perform an operation corresponding to the set condition. The method specifically includes the following steps.

[0077] S210, in the first navigation mode, the first navigation mode is switched to the second navigation mode in a case where a switching condition is met, the first navigation mode performs the navigation according to first interaction information, and the second navigation mode performs the navigation according to second interaction information.

[0078] S220, the navigation is performed according to a navigation starting point in the second navigation mode and the second interaction information after being switched to the second navigation mode, and the navigation starting point in the second navigation mode is determined according to a navigation ending point in the first navigation mode when the first navigation mode is switched to the second navigation mode.

[0079] In the embodiment, after being switched to the second navigation mode, the second interaction information is detected, and the navigation is performed based on the navigation starting point in the second navigation mode and the second interaction information. Specifically, the navigation starting point can be located at the position A, and the navigation is continued based on the second interaction information starting from the position A. Optionally, the navigation information is determined based on the second interaction information, and the navigation information includes but is not limited to the navigation moving direction, the navigation moving distance, and the speed information, etc. Correspondingly, based on the navigation starting point, the navigation is performed based on the navigation moving direction and the navigation moving distance.

[0080] Exemplarily, in a case that the second interaction information is eye movement tracking information, a navigation movement direction can be determined according to an eye movement direction in the eye movement tracking information, and the navigation movement direction can be the same as the eye movement direction; a navigation movement distance can be determined according to an eye movement distance in the eye movement tracking information, and the navigation movement distance can be positively correlated with the eye movement distance. In a case that the second interaction information is gesture tracking information, a navigation movement direction can be determined according to a hand movement direction in the gesture tracking information, and the navigation movement direction can be the same as the hand movement direction; a navigation movement distance can be determined according to a hand movement distance in the gesture tracking information, and the navigation movement distance can be positively correlated with the hand movement distance. In a case that the second interaction information is head tracking information, a navigation movement direction can be determined according to a head rotation direction or a head movement direction in the head tracking information, and the navigation movement direction can be the same as the head rotation direction or the head movement direction; a navigation movement distance can be determined according to a head rotation angle or a head movement distance in the head tracking information, and the navigation movement distance can be positively correlated with the head rotation angle or the head movement distance.

[0081] Exemplarily, referring to FIG. 4, FIG. 4 is a schematic diagram of a navigation process provided by an embodiment of the present disclosure. In the left diagram of FIG. 4, a dot in a display page can represent a navigation starting point. According to the second interaction information, a navigation movement direction can be determined to be leftward movement. Based on a navigation movement distance, the display page is moved leftward to obtain the display page shown in the right diagram of FIG. 4.

[0082] It can be understood that the controller interacts with the electronic device to enable the electronic device to detect the second interaction information. In different navigation modes, the controller interacts with the electronic device by using different body parts to collect different interaction information. For example, eye movement tracking information is obtained by detecting the eyes of the controller, gesture tracking information is obtained by detecting the hands of the controller, and head tracking information is obtained by detecting the head of the controller. Because the data ranges corresponding to different interaction information are different, the movement distance and speed information extracted from different interaction information are also different. For example, the eye movement distance in the eye movement tracking information can be 1 mm, and the hand movement distance in the gesture tracking information can be 10 cm. In order to facilitate navigation and avoid inconsistent navigation ranges in different navigation modes, gain data is set. The gain data can be understood as an adjustment coefficient of movement information of the virtual object relative to movement information of the second control part. Accordingly, navigation is performed according to the navigation starting point, the second interaction information after switching to the second navigation mode, and the preset gain information. The gain data can be greater than 0. Optionally, the gain data can be greater than 1.

[0083] In the second navigation mode, the movement information in the second interaction information is extracted, the movement information in the second interaction information is processed by the gain data, the navigation movement information is obtained, and the navigation is performed by using the navigation movement information. The navigation movement information can be understood as movement information of the user's intention in the movement of the interaction object.

[0084] Because the data range corresponding to the second interaction information is different in different navigation modes, the gain data corresponding to different navigation modes can be different. For example, the gain data corresponding to the eye movement navigation mode can be greater than the gain data corresponding to the gesture navigation mode.

[0085] In the above embodiment, in the case where the second navigation mode is the gesture navigation mode, the second interaction information is detected by controlling the hand of the person. The detection can be performed on a key point of the hand to obtain the second interaction information. For example, the key point of the hand can be the tip of the thumb, the tip of the index finger, or the base of the middle finger.

[0086] In some embodiments, the preset posture of the hand can be used to generate the interaction instruction. For example, the preset posture can be a pinch posture, that is, a pinch posture of at least two fingers, for example, a pinch posture of the thumb and the index finger. In order to improve the accuracy of navigation, the hand navigation position of the hand when the preset posture is met can be used as the hand position. Correspondingly, the second interaction information is generated by detecting the hand navigation position of the hand when the preset posture is met. The hand navigation position of the hand when the preset posture is met can be the pinch position when the hand is in the pinch posture. It can be understood that during navigation, the hand is not in the pinch state, which makes it difficult to detect the pinch position corresponding to the pinch posture of the hand.

[0087] To solve the above problem, the offset information between the hand navigation position of the hand when the preset posture is met and the fourth key point of the hand is determined, the second interaction information is determined based on the tracking information of the fourth key point of the hand and the offset information, and the navigation is performed according to the navigation starting point and the second interaction information. The fourth key point of the hand can be a key point of the hand that has relatively stable, for example, the palm or the wrist. By determining the offset information between the hand navigation position when the preset posture is met and the fourth key point, the tracking information corresponding to the hand navigation position can be determined as the gesture tracking information, that is, the second interaction information, based on the tracking information corresponding to the fourth key point and the offset information at any time, which improves the accuracy of the gesture tracking information and further improves the navigation accuracy.

[0088] The offset information between the hand navigation position and the fourth key point when the preset posture is met can include position offset information and angle offset information. The position offset information can be determined by the coordinate difference between the hand navigation position and the position of the fourth key point. The angle offset information can be determined based on the angle difference between the hand navigation position and the position of the fourth key point.

[0089] The offset information between the hand navigation position and the fourth key point is stored. In the gesture navigation mode, the second interaction information is determined based on the tracking information of the fourth key point and the offset information.

[0090] In the above embodiment, the anti-shake processing is performed during navigation to improve navigation accuracy. Optionally, navigation is performed according to the navigation starting point, the second interaction information after switching to the second navigation mode, and the smoothing factor corresponding to the second interaction information. Specifically, the smoothing factor is determined based on the second interaction information, and the navigation process is smoothed based on the smoothing factor.

[0091] The velocity information can be extracted from the second interaction information. The maximum velocity information can be determined based on the velocity information in the second interaction information corresponding to multiple time points in the second navigation mode. The smoothing factor can be determined based on the velocity information of the current time point and the maximum velocity information in the second navigation mode. For example, the ratio of the velocity information of the current time point to the maximum velocity information in the second navigation mode can be determined as the smoothing factor. For example, based on the second interaction information and the gain data, the navigation movement information is determined. The navigation movement information is smoothed based on the smoothing factor to obtain smoothed navigation movement information. The navigation is performed based on the smoothed navigation movement information. For example, the product of the smoothing factor and the navigation movement information can be determined as the smoothed navigation movement information.

[0092] The navigation process is smoothed by the smoothing factor to improve the stability of the navigation process and reduce fluctuations in the navigation process.

[0093] The technical solution provided in the embodiment enables navigation to be performed based on the second interaction information and the navigation starting point when the navigation mode is switched to the second navigation mode, and enables navigation to be continued in the second navigation mode.

[0094] Based on the above embodiment, when the second interaction information meets the set condition during navigation based on the second navigation mode, the operation corresponding to the set condition is performed. In the embodiment, a corresponding relationship between the set condition and the operation is set in advance. The operation can include but is not limited to a selection operation. In different second navigation modes, the same operation can correspond to different interaction information. That is, the set condition corresponding to the second interaction information can be different in different navigation modes.

[0095] In some embodiments, the setting condition corresponding to the second interaction information can be that the hand is in a pinching gesture, where the pinching gesture of the hand can be a pinching gesture of the second key point and the third key point of the hand. For example, it can be a pinching gesture between the thumb and the index finger. The operation corresponding to the hand being in the pinching gesture includes a selection operation. In some embodiments, the setting condition corresponding to the second interaction information can include that the eye is in a gaze state, and the gaze duration of the gaze state exceeds a preset duration threshold. The operation corresponding to the gaze state of the eye includes a selection operation. In some embodiments, the setting condition corresponding to the second interaction information can include a nodding action of the head. The operation corresponding to the nodding action includes a selection operation. In some embodiments, the setting condition corresponding to the second interaction condition can also be voice information such as "determine" or "select", and the operation corresponding to the voice information includes a selection operation. In some embodiments, the setting condition corresponding to the second interaction condition can also be "determine" information transmitted by the input device, or a click operation on the touch device, and the operation corresponding to the setting condition can include a selection operation. It can be understood that the selection operation is only an example, and in other embodiments, different operations can be set according to operation requirements.

[0096] In the above embodiments, the operation corresponding to the setting condition can be performed at the current navigation position when it is detected that the second interaction information meets the setting condition. For example, referring to FIG. 5, which is a schematic diagram of a navigation process provided by an embodiment of the present disclosure. In FIG. 5, the second navigation mode is a gesture navigation mode, the display page includes a controllable control, the user's intention is represented by a cursor, the cursor in FIG. 5 is located at control 2, and it is detected that the gesture tracking information (i.e., the second interaction information) meets the setting condition, i.e., the hand is in a pinching gesture, and a selection operation is performed on control 2. Wherein, the gray control 2 represents that control 2 is in a selected state.

[0097] It can be understood that the display page can also include content in the form of images, text, and hyperlinks, and accordingly, the selection operation can be performed on the images, text, and hyperlinks by using the above setting conditions. Wherein, by selecting the control or hyperlink in the display page, the next page of the display page can be displayed, or a business page corresponding to the control or hyperlink can be entered. By selecting the image or text, a processing control of the image or text can be displayed, such as a copy, cut, or attribute adjustment control. The processing control can be selected by using the setting condition of the second control component to process the image or text.

[0098] The technical solution provided by the embodiments of the present disclosure improves the interaction between the controller and the display page by performing corresponding operations through the second interaction information when the navigation mode is switched to the second navigation mode.

[0099] Fig. 6 is a schematic diagram of an interaction device structure according to an embodiment of the present disclosure. As shown in Fig. 6, the device comprises a mode switching module 410.

[0100] The mode switching module 410 switches the first navigation mode to a second navigation mode in the first navigation mode when a switching condition is met, the first navigation mode being navigated according to first interaction information, and the second navigation mode being navigated according to second interaction information.

[0101] The navigation starting point in the second navigation mode is determined according to a navigation ending point in the first navigation mode when the first navigation mode is switched to the second navigation mode.

[0102] The technical solution provided by the embodiment of the present disclosure switches the first navigation mode to the second navigation mode in the first navigation mode when the switching condition is met. When the switching condition is not met, it indicates that the switching intention of the control person is not detected, and the first navigation mode is continued. By switching the navigation mode when the switching condition is met, the navigation is realized through different interaction modes, the flexibility of the interaction mode in the navigation process is improved, the fatigue caused by the control person navigating through a single interaction mode is reduced, and the navigation is continued during the navigation mode switching process, and the continuity of the navigation process is improved.

[0103] On the basis of the above-mentioned embodiment, optionally, the first interaction information and the second interaction information are different.

[0104] On the basis of the above-mentioned embodiment, optionally, the switching condition is determined to be met when the second interaction information is detected.

[0105] On the basis of the above-mentioned embodiment, optionally, the first interaction information comprises eye tracking information and / or head tracking information, and the second interaction information comprises gesture tracking information.

[0106] On the basis of the above-mentioned embodiment, optionally, when the included angle between the line connecting the first key point of the hand and the head and the direction of the head is less than a preset angle threshold, and / or the distance between the second key point and the third key point of the hand is less than a preset distance threshold, it is determined that the switching condition is met.

[0107] On the basis of the above-mentioned embodiment, optionally, when the included angle between the line connecting the first key point of the hand and the head and the direction of the head is less than a preset angle threshold, and / or the distance between the second key point and the third key point of the hand is less than a preset distance threshold, and the holding time exceeds a preset time, it is determined that the switching condition is met.

[0108] On the basis of the above-mentioned embodiment, optionally, the mode switching module 410 is further configured to switch the second navigation mode to the first navigation mode when the hand is not detected.

[0109] On the basis of the above-mentioned embodiment, optionally, the mode switching module 410 is further configured to, in response to receiving the interaction instruction, perform an operation corresponding to the interaction instruction, and switch the second navigation mode to the first navigation mode.

[0110] On the basis of the above-mentioned embodiment, optionally, the device further comprises a navigation module configured to navigate according to the navigation starting point and the second interaction information after switching to the second navigation mode.

[0111] Optionally, the navigation module is further configured to navigate according to the navigation starting point, the second interaction information after switching to the second navigation mode, and preset gain information.

[0112] Optionally, the navigation module is further configured to navigate according to the navigation starting point, the second interaction information after switching to the second navigation mode, and a smoothing factor corresponding to the second interaction information.

[0113] Optionally, the navigation module is further configured to, when detecting that the second interaction information meets a set condition, perform an operation corresponding to the set condition.

[0114] On the basis of the above-mentioned embodiment, optionally, the interaction device is integrated into a head-mounted extended reality device.

[0115] The interaction device provided in the embodiments of the present disclosure can perform the interaction method provided in any of the embodiments of the present disclosure, and has the corresponding function modules and beneficial effects of the execution method.

[0116] It should be noted that each unit and module included in the above-mentioned device is only divided according to the function logic, and is not limited to the above-mentioned division, as long as the corresponding function can be realized; in addition, the specific name of each functional unit is only for convenient mutual distinction, and is not used to limit the protection scope of the embodiments of the present disclosure.

[0117] FIG. 7 is a structural diagram of an electronic device according to an embodiment of the disclosure. Below, referring to FIG. 7, a structural diagram of an electronic device (e.g., a terminal device or a server in FIG. 7) 500 suitable for implementing an embodiment of the disclosure is illustrated. The terminal device in an embodiment of the disclosure can include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Tablet Personal Computer), a PMP (Portable Multimedia Player), a car terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device illustrated in FIG. 7 is merely an example, and should not impose any limitation on the functions and use range of an embodiment of the disclosure.

[0118] As illustrated in FIG. 7, the electronic device 500 can include a processing device (e.g., a central processor, a graphic processor, etc.) 501 that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0119] Generally, the following devices can be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; a storage device 508 including, for example, a magnetic tape, a hard disk, and the like; and a communication device 509. The communication device 509 can allow the electronic device 500 to communicate with other devices wirelessly or through wires to exchange data. Although FIG. 7 illustrates the electronic device 500 having various devices, it should be understood that all of the illustrated devices are not required to be implemented or possessed. More or fewer devices can be alternatively implemented or possessed.

[0120] According to an embodiment of the disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, an embodiment of the disclosure includes a computer program product including a computer program carried on a non-transitory computer readable medium, the computer program containing program code for executing the methods illustrated in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-described functions defined in the methods of an embodiment of the disclosure are performed.

[0121] Names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes, and are not used to limit the scope of the messages or information.

[0122] The electronic device provided by the embodiments of the present disclosure and the interaction method provided by the above embodiments belong to the same inventive concept, and the technical details not described in detail in the present embodiment can be referred to the above embodiments, and the present embodiment has the same beneficial effects as the above embodiments.

[0123] The embodiments of the present disclosure provide a computer storage medium, which stores a computer program, and the program is executed by a processor to implement the interaction method provided by the above embodiments.

[0124] It should be noted that the computer readable medium of the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, a cable, an RF (radio frequency) or the like, or any suitable combination of the above.

[0125] In some embodiments, the client, server, or other computing machines utilized by the system can communicate over any known or later developed network protocol, such as the Hypertext Transfer Protocol (HTTP), and can be interconnected with any form or medium of digital data communication (for example, a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet, and peer-to-peer networks (for example, ad hoc peer-to-peer networks), as well as any then-current or later developed networks.

[0126] The computer-readable medium described above can be included in the electronic device described above; alternatively, the computer-readable medium can exist as a standalone entity.

[0127] The computer-readable medium described above carries one or more programs which, when executed by the electronic device, cause the electronic device to:

[0128] The computer-readable medium described above carries one or more programs which, when executed by the electronic device, cause the electronic device to: in a first navigation mode, switch the first navigation mode to a second navigation mode in a case where a switching condition is met, the first navigation mode being navigated according to first interaction information, the second navigation mode being navigated according to second interaction information; wherein a navigation starting point in the second navigation mode is determined according to a navigation ending point in the first navigation mode when the first navigation mode is switched to the second navigation mode.

[0129] Computer program code for carrying out operations of the present disclosure can be written in any of one or more programming languages, including but not limited to object oriented programming languages such as Java, Smalltalk, C++, or conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network ("LAN") or a wide area network ("WAN"), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0130] The computer program product of the first aspect can include one or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform the operations of the first aspect. The one or more non-transitory computer-readable media can include, for example, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile memory, non-volatile memory, or a suitable combination of different computer-readable media.

[0131] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the names of the units do not constitute a limitation on the units themselves. For example, the first obtaining unit can also be described as a unit that obtains at least two Internet protocol addresses.

[0132] The functions described in the present disclosure can be performed, at least in part, by one or more hardware logic components. For example, non-limiting examples of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0133] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable storage media can include, without limitation, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include one or more lines of electrical connections, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0134] The above description merely illustrates the embodiments of the present disclosure and a principle of applied technologies. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by the combinations of the technical features described above or their equivalent features without departing from the concept disclosed above. For example, the technical solutions formed by the mutual replacement of the above-described features and the technical features with similar functions disclosed in the present disclosure (but not limited to) can be used.

[0135] Further, although operations are depicted in a particular order, this should not be understood as requiring the operations to be performed in the particular order shown or in a sequential order, and that certain operations can be performed in parallel or concurrently with other operations disclosed herein. In addition, although a variety of exemplary mechanisms implementing the present disclosure have been illustrated, one of ordinary skill in the art will understand that the implementations illustrated are merely exemplary. It should be appreciated that many other implementations can be practiced under the scope of the present disclosure. Similarly, while operations are depicted in the above embodiments as occurring in a particular order, this should not be understood as requiring or implying that such operations occur in the particular order, nor that the operations are all re- quired, or that the described implementation is the only one possible.

[0136] Although the present disclosure has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. An interaction method, comprising: in a first navigation mode, switching the first navigation mode to a second navigation mode if a switching condition is met, the first navigation mode being navigated according to first interaction information, the second navigation mode being navigated according to second interaction information; wherein a navigation starting point in the second navigation mode is determined according to a navigation ending point in the first navigation mode when the first navigation mode is switched to the second navigation mode.

2. The method of claim 1, wherein, The first interaction information and the second interaction information are different.

3. The method of claim 1 or 2, wherein, The switching condition is determined when the second interaction information is detected.

4. The method according to any of claims 1 to 3, wherein, The first interaction information comprises eye tracking information and / or head tracking information, and the second interaction information comprises gesture tracking information.

5. The method of claim 4, wherein, The switching condition is determined when an angle between a line connecting a first key point of a hand and a head and a head orientation is less than a preset angle threshold, and / or a distance between a second key point and a third key point of the hand is less than a preset distance threshold.

6. The method of claim 5, wherein, The switching condition is determined when an angle between a line connecting a first key point of a hand and a head and a head orientation is less than a preset angle threshold, and / or a distance between a second key point and a third key point of the hand is less than a preset distance threshold, and a holding time exceeds a preset time.

7. The method of claim 4, further comprising: The second navigation mode is switched to the first navigation mode when the hand is not detected.

8. The method of claim 1 or 4, further comprising: In response to receiving an interaction instruction, an operation corresponding to the interaction instruction is performed, and the second navigation mode is switched to the first navigation mode. 9.The method of any one of claims 1-8, further comprising: navigating according to the navigation starting point and the second interaction information after switching to the second navigation mode. 10.The method of claim 9, further comprising: navigating according to the navigation starting point, the second interaction information after switching to the second navigation mode, and preset gain information. 11.The method of claim 9 or 10, further comprising: navigating according to the navigation starting point, the second interaction information after switching to the second navigation mode, and a smoothing factor corresponding to the second interaction information. 12.The method of any one of claims 9-11, further comprising: performing an operation corresponding to a set condition when the second interaction information is detected to meet the set condition.

13. The method of any one of claims 1-12, wherein, The interaction method is applied to a head-mounted extended reality device. 14.An interaction apparatus, comprising: a navigation mode switching module configured to, in a first navigation mode, switch the first navigation mode to a second navigation mode if a switching condition is met, the first navigation mode being navigated according to first interaction information, the second navigation mode being navigated according to second interaction information; wherein a navigation starting point in the second navigation mode is determined according to a navigation ending point in the first navigation mode when the first navigation mode is switched to the second navigation mode. 15.An electronic device, comprising: one or more processors; a storage device configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement an interaction method as claimed in any of claims 1-13.

16. A storage medium containing computer-executable instructions for performing an interaction method as claimed in any of claims 1-13 when executed by a computer processor.

Images