Operation method of three-dimensional interface, head control device and operation system

Abstract

The invention relates to the technical field of man-machine interaction, and provides a three-dimensional interface operation method, a head control device and an operation system, the head control device is in Bluetooth connection with a main control ring, the main control ring is in Bluetooth connection with an auxiliary control ring, and the two rings can be worn on two fingers of one hand and can also be placed on a desktop. Therefore, two modes of gesture interaction and desktop touch interaction are realized, and the application scene is wider. In the gesture interaction mode, an aiming line is determined according to self position data and position data of the main control ring, and after the aiming line aims at a target, finger kneading operation judgment is carried out according to the distance between two fingers determined according to opposite position data collected by the two rings; in the desktop touch interaction mode, the aiming line is determined according to the position data of the fingertip acquired by any finger ring, and after the aiming line aims at the target, the finger desktop knocking operation is judged according to the rotation change acquired by the corresponding finger ring. And the two interaction modes are simple to switch, so that the requirements of different use scenes can be met.

Description

Background Art

[0002] As a new type of display, the interactive operation design of head mounted display (HMD) is crucial to the user experience. Currently, the common interactive modes are:

[0003] (1) Touch or button: Some HMDs are designed with touchpads or physical buttons. Users can interact by touching or pressing them. This makes it easy to use. However, due to the limited operating space on the HMD, it is generally used for interactive operations of simple functions.

[0004] (2) Eye tracking: Using eye tracking technology, users interact with the HMD by moving their eyes and looking at them. This is easy to operate, but the interaction accuracy is poor and the usage scenario requirements are high.

[0005] (3) Voice commands: Through voice recognition technology, users can directly give commands to the HMD, which is convenient to operate, but the interaction accuracy is poor and has high requirements for usage scenarios;

[0006] (4) Handheld controllers, similar to handles, can provide more precise and complex interactive operations, but they increase the user's equipment burden and are inconvenient for users to carry;

[0007] (5) Gesture recognition: Users interact with the HMD through different gestures (such as waving, pinching, stretching, etc.). The operation is convenient, but the user needs to keep both hands raised to operate. After long-term use, the user consumes more physical energy and causes fatigue in the arms, which is not conducive to long-term use.

[0008] Therefore, providing a lightweight and accurate interaction method has important research value for HMD. Summary of the invention

[0009] The embodiments of the present application provide a three-dimensional interface operation method, a head control device, and an operating system, which are used to improve the accuracy of interaction with a head-mounted display device and reduce the user interaction burden.

[0010] In a first aspect, an embodiment of the present application provides a method for operating a three-dimensional interface, which is applied to a head control device, wherein the head control device is connected to a main control ring via Bluetooth, and the main control ring is connected to an auxiliary control ring via Bluetooth, and the method includes:

[0011] Determining an interaction mode according to the indication information sent by the master control ring; wherein the indication information is triggered based on a user operation;

[0012] When the interaction mode is the gesture interaction mode, the aiming line in the three-dimensional interface is determined according to its own position data and the posture data of the main control ring, and after the aiming line is aimed at the target of the three-dimensional interface, the distance between the two fingers is determined according to the position data of each other collected by the main control ring and the auxiliary control ring to realize the confirmation operation triggered by finger pinching; wherein, in the gesture interaction mode, the main control ring and the auxiliary control ring are worn on two fingers of one hand;

[0013] When the interaction mode is a desktop touch interaction mode, the aiming line in the three-dimensional interface is determined according to the position data of the fingertips collected by any finger ring, and after the aiming line is aimed at the target of the three-dimensional interface, the rotation change is determined according to the rotation data collected by the corresponding finger ring to realize the confirmation operation triggered by tapping the desktop; wherein, in the desktop touch interaction mode, the main control ring and the auxiliary control ring are placed on the desktop.

[0014] The effective effect of the above technical solution is: the main control ring and the auxiliary control ring have two different usage methods, one is to wear them on two fingers of one hand to realize the gesture interaction mode, and the other is to place them on the desktop to realize the desktop touch interaction mode. In this way, when interacting, the user can trigger the indication information by operating the main control ring according to the actual interaction needs, and the head control device determines the interaction mode according to the indication information, thereby realizing flexible selection of interaction modes in different scenarios, making the operation of the head-mounted display device more flexible and diverse, and the usage scenarios are more extensive. Among them, in the gesture interaction mode, the head control device determines the aiming line in the three-dimensional interface according to its own position data and the posture data of the main control ring. Since the determination of the aiming line uses two parts of data, the head and the hand, the aiming process of the aiming line in the three-dimensional interface is more stable and more accurate, thereby improving the interaction accuracy. In addition, after the aiming line is controlled by the movement of the hand to aim at the target in the three-dimensional interface, the finger pinching judgment is performed according to the distance between the two fingers determined by the position data of the other party collected by the main control ring and the auxiliary control ring, so as to determine whether the user has confirmed the target and realize gesture interaction; while in the desktop touch interaction mode, the head control device determines the aiming line in the three-dimensional interface according to the position data of the fingertips collected by the ring, and after the aiming line is controlled by sliding the finger on the desktop to aim at the target, the desktop tapping judgment is performed according to the rotation change determined by the rotation data collected by the corresponding ring, so as to determine whether the user has confirmed the target and realize desktop touch interaction.

[0015] Optionally, the posture data includes position data and rotation data, and determining the aiming line in the three-dimensional interface according to the position data of the ring and the posture data of the master control ring includes:

[0016] Determine the starting point of the aiming line according to its own position data and the position data of the main control ring;

[0017] Taking the starting point as the origin, a ray directed toward the three-dimensional interface is rendered according to the direction represented by the rotation data of the main control ring to obtain the aiming line.

[0018] The beneficial effects of the above technical solution are as follows: in the gesture interaction mode, the head control device determines the starting point position and direction of the aiming line according to its own position data, the position data and the rotation data of the main control ring, and realizes the joint calibration of the aiming line in the three-dimensional interface by the head and the hand, thereby improving the stability and accuracy of the aiming line, and allowing the user to change the movement of the aiming line in the three-dimensional interface by moving the finger to select the target, which is simple and convenient to operate.

[0019] Optionally, when the master control ring is worn on the left hand, the starting point position is calculated as follows:

[0020] x0=x h -x t *a

[0021] y0=y h -y t *b

[0022] z0=z h

[0023] When the master ring is worn on the right hand, the starting point position is calculated as follows:

[0024] x0=x h +x t *a

[0025] y0=y h -y t *b

[0026] z0=z h

[0027] Among them, (x h ,y h , z h ) is the position coordinate of the head control device, (x t ,y t , z t ) is the position coordinate of the main control ring, a is the offset coefficient of the X-axis, and b is the offset coefficient of the Y-axis.

[0028] The beneficial effects of the above technical solution are as follows: two methods for calculating the trigger point position are designed for the cases where the main control ring and the auxiliary control ring are worn on the right hand and the left hand. Through the above two calculation methods, the starting point position of the aiming line is located in the middle position between the hand and the head, and because the head generally remains motionless during aiming, the stability of the starting point position calculation is improved, thereby making the aiming accuracy and stability higher.

[0029] Optionally, determining the distance between two fingers to implement the confirmation operation triggered by finger pinching according to the position data of each other collected by the main control ring and the auxiliary control ring respectively includes:

[0030] Determine the first position coordinates of the main control ring according to the fingertip position coordinates and at least one finger joint position coordinates of the hand where the main control ring is located, and determine the second position coordinates of the auxiliary control ring according to the fingertip position coordinates and at least one finger joint position coordinates of the hand where the auxiliary control ring is located;

[0031] The distance between two fingers is determined according to the first position coordinates and the second position coordinates, and whether a confirmation operation triggered by pinching the fingers occurs is determined according to a comparison result between the distance between the two fingers and a preset distance threshold.

[0032] The beneficial effects of the above technical solution are as follows: since the main control ring and the auxiliary control ring are worn on two fingers of one hand, the distance between the two fingers can be determined by the position coordinates of the two fingers, and whether the user performs a pinching operation can be determined based on the comparison result between the distance between the two fingers and the preset distance threshold, thereby realizing gesture interaction of the head-mounted display device. On the other hand, the position of the ring is calculated by the fingertip position coordinates and the position coordinates of at least one finger joint, so as to improve the accuracy of the two-finger distance calculation to improve the determination of the pinching operation, thereby improving the interaction accuracy.

[0033] Optionally, when the main control ring is worn on the thumb and the auxiliary control ring is worn on a finger other than the thumb, the first position coordinate is calculated as follows:

[0034] x t =(x tt *c1+x tj *c2) / (c1+c2)

[0035] y t =(y tt *c1+y tj *c2) / (c1+c2)

[0036] z t =(z tt *c1+z tj *c2) / (c1+c2)

[0037] The second position coordinates are calculated as follows:

[0038] x f =(x ft *d1+x fj1 *d2+x fj2 *d3) / (d1+d2+d3)

[0039] y f =(y ft *d1+y fj1 *d2+y fj2 *d3) / (d1+d2+d3)

[0040] z f =(z ft *d1+z fj1 *d2+z fj2 *d3) / (d1+d2+d3)

[0041] Among them, (x tt ,y tt , z tt ) is the coordinate of the fingertip position of the thumb, (x tj ,y tj , z tj ) is the coordinate of the middle joint position of the thumb; (x ft ,y ft , z ft ) is the fingertip position coordinate of the other finger, (x fj1 ,y fj1 , z fj1 ) is the first joint position coordinate of the other fingers, (x fj2 ,y fj2 , z fj2 ) are the second joint position coordinates of the other fingers, and c1, c2, d1, d2 and d3 are weighted average parameters.

[0042] The beneficial effect of the above technical solution is: by wearing the main control ring on the thumb and the auxiliary control ring on other fingers, the distance between the two fingers can be increased, thereby avoiding misjudgment of gestures with a small distance between the two fingers and improving the interaction accuracy.

[0043] Optionally, determining the aiming line in the three-dimensional interface according to the fingertip position data collected by any finger ring includes:

[0044] According to the fingertip position data collected by any finger ring, the touch calibration point on the Overlay calibration plane is determined; the Overlay calibration plane is a transparent screen, located on the upper layer of the three-dimensional interface and parallel to the three-dimensional interface, and the coordinate origin of the Overlay calibration plane is the position of any finger ring;

[0045] Taking the touch calibration point as the origin, a ray perpendicular to the three-dimensional interface is rendered to obtain the aiming line.

[0046] The beneficial effects of the above technical solution are as follows: in the desktop touch interaction mode, the head control device determines the starting position of the aiming line according to the position data of the fingertips corresponding to any finger ring and the aiming method based on the Overlay calibration plane, thereby completing the rendering of the aiming line in the three-dimensional interface with high aiming accuracy. Moreover, in this mode, the user can change the aiming line in the three-dimensional interface by moving the fingertips on the desktop, which is easy to operate.

[0047] Optionally, the aspect ratio of the touch operation area of ​​the finger ring placed on the desktop is the same as the aspect ratio of the Overlay calibration plane, and the step of determining the touch calibration point on the Overlay calibration plane according to the fingertip position data collected by any finger ring includes:

[0048] Determine the distance between the mapping point on the Overlay calibration plane and the coordinate origin according to the distance from the fingertip to the corresponding ring in the touch operation area of ​​any ring and the aspect ratio;

[0049] Determine the direction of the mapping point on the Overlay calibration plane according to the direction of the fingertip relative to the finger ring in the touch operation area of ​​any finger ring;

[0050] According to the distance and direction corresponding to the mapping point, the touch calibration point on the Overlay calibration plane is determined.

[0051] The beneficial effect of the above technical solution is: by creating an Overlay calibration plane, the starting position of the aiming line is switched to the touch calibration point on the plane. Since the aspect ratio of the touch operation area of ​​the set ring is the same as the aspect ratio of the Overlay calibration plane, the movement of the fingertip in the touch operation area of ​​the ring on the desktop can drive the movement of the aiming line in the three-dimensional interface, thereby realizing desktop touch operation.

[0052] Optionally, determining the rotation change amount according to the rotation data collected by the corresponding finger ring to implement the confirmation operation triggered by tapping the desktop includes:

[0053] Determine the rotation change according to the acceleration collected by the corresponding finger ring; wherein the change in acceleration is generated when the corresponding finger ring is subjected to pressure, and the pressure is generated when the fingertip hits the tabletop to cause the tabletop to vibrate;

[0054] According to the comparison result of the rotation change amount and the preset rotation threshold, it is determined whether a confirmation operation triggered by tapping the desktop occurs.

[0055] The beneficial effect of the above technical solution is: detecting whether the fingertip hits the desktop through the rotation change amount, thereby realizing the confirmation operation in the desktop touch interaction mode.

[0056] In a second aspect, an embodiment of the present application provides a head control device, including a posture sensor, a Bluetooth module, a storage chip, and a computing chip;

[0057] The Bluetooth module is used to connect to the main control ring via Bluetooth, and the main control ring is connected to the auxiliary control ring via Bluetooth;

[0058] The memory chip includes a data storage area and a program storage area, the data storage area is used to store the data collected by the chip itself and the data collected by the ring, and the program storage area is used to store computer programs;

[0059] The computing chip performs the following operations according to the computer program:

[0060] Receiving indication information sent by the master ring through the Bluetooth module, and determining the interaction mode according to the indication information; wherein the indication information is triggered based on the user operation;

[0061] When the interaction mode is the gesture interaction mode, the aiming line in the three-dimensional interface is determined according to the position data of the posture sensor itself and the posture data of the main control ring, and after the aiming line is aimed at the target of the three-dimensional interface, the position data of the other party sent by the main control ring and the auxiliary control ring are received through the Bluetooth module, and the distance between the two fingers is determined according to the position data of the other party to realize the confirmation operation triggered by finger pinching; wherein, in the gesture interaction mode, the main control ring and the auxiliary control ring are worn on two fingers of one hand;

[0062] When the interaction mode is a desktop touch interaction mode, the position data of the fingertips collected by any finger ring is received through the Bluetooth module, and the aiming line in the three-dimensional interface is determined according to the position data of the fingertips, and after the aiming line is aimed at the target of the three-dimensional interface, the rotation data collected by the corresponding finger ring is received through the Bluetooth module, and according to the rotation data, the rotation change amount is determined to realize the confirmation operation triggered by tapping the desktop; wherein, in the desktop touch interaction mode, the main control ring and the auxiliary control ring are placed on the desktop.

[0063] Optionally, the posture data includes position data and rotation data, and the specific operation of determining the aiming line in the three-dimensional interface according to the position data of the posture sensor itself and the posture data of the master control ring is as follows:

[0064] Determine the starting point position of the aiming line according to the position data of the position sensor itself and the position data of the master control ring;

[0065] Taking the starting point as the origin, a ray directed toward the three-dimensional interface is rendered according to the direction represented by the rotation data of the main control ring to obtain the aiming line.

[0066] Optionally, when the master control ring is worn on the left hand, the starting point position is calculated as follows:

[0067] x0=x h -x t *a

[0068] y0=y h -y t *b

[0069] z0=z h

[0070] When the master ring is worn on the right hand, the starting point position is calculated as follows:

[0071] x0=x h +x t *a

[0072] y0=y h -y t *b

[0073] z0=z h

[0074] Among them, (x h ,y h , z h ) is the position coordinate of the head control device, (x t ,y t , z t ) is the position coordinate of the main control ring, a is the offset coefficient of the X-axis, and b is the offset coefficient of the Y-axis.

[0075] Optionally, the determining of the distance between two fingers according to the other party's position data to implement the confirmation operation triggered by pinching the fingers is specifically performed as follows:

[0076] Determine the first position coordinates of the main control ring according to the fingertip position coordinates and at least one finger joint position coordinates of the hand where the main control ring is located, and determine the second position coordinates of the auxiliary control ring according to the fingertip position coordinates and at least one finger joint position coordinates of the hand where the auxiliary control ring is located;

[0077] The distance between two fingers is determined according to the first position coordinates and the second position coordinates, and whether a confirmation operation triggered by pinching the fingers occurs is determined according to a comparison result between the distance between the two fingers and a preset distance threshold.

[0078] When the main control ring is worn on the thumb, and the auxiliary control ring is worn on fingers other than the thumb, the first position coordinate is calculated as follows:

[0079] x t =(x tt *c1+x tj *c2) / (c1+c2)

[0080] y t =(y tt *c1+y tj *c2) / (c1+c2)

[0081] z t =(z tt *c1+z tj *c2) / (c1+c2)

[0082] The second position coordinates are calculated as follows:

[0083] x f =(x ft *d1+x fj1 *d2+x fj2 *d3) / (d1+d2+d3)

[0084] y f =(y ft *d1+y fj1 *d2+y fj2 *d3) / (d1+d2+d3)

[0085] z f =(z ft *d1+z fj1 *d2+z fj2 *d3) / (d1+d2+d3)

[0086] Among them, (x tt ,y tt , z tt ) is the coordinate of the fingertip position of the thumb, (x tj ,y tj , z tj ) is the coordinate of the middle joint position of the thumb; (x ft ,y ft , z ft ) is the fingertip position coordinate of the other finger, (x fj1 ,y fj1 , z fj1 ) is the first joint position coordinate of the other fingers, (x fj2 ,y fj2 , z fj2) are the second joint position coordinates of the other fingers, and c1, c2, d1, d2 and d3 are weighted average parameters.

[0087] Optionally, the determining of the aiming line in the three-dimensional interface according to the position data of the fingertip is specifically performed as follows:

[0088] According to the fingertip position data collected by any finger ring, the touch calibration point on the Overlay calibration plane is determined; the Overlay calibration plane is a transparent screen, located on the upper layer of the three-dimensional interface and parallel to the three-dimensional interface, and the coordinate origin of the Overlay calibration plane is the position of any finger ring;

[0089] Taking the touch calibration point as the origin, a ray perpendicular to the three-dimensional interface is rendered to obtain the aiming line.

[0090] Optionally, the aspect ratio of the touch operation area of ​​the finger ring placed on the desktop is the same as the aspect ratio of the Overlay calibration plane, and the touch calibration point on the Overlay calibration plane is determined according to the fingertip position data collected by any finger ring, and the specific operation is:

[0091] Determine the distance between the mapping point on the Overlay calibration plane and the coordinate origin according to the distance from the fingertip to the corresponding ring in the touch operation area of ​​any ring and the aspect ratio;

[0092] Determine the direction of the mapping point on the Overlay calibration plane according to the direction of the fingertip relative to the finger ring in the touch operation area of ​​any finger ring;

[0093] The touch calibration on the Overlay calibration plane is determined according to the distance and direction corresponding to the mapping point.

[0094] Optionally, the step of determining the rotation change amount according to the rotation data to implement a confirmation operation triggered by tapping the desktop is specifically performed as follows:

[0095] Determine the rotation change according to the acceleration collected by the corresponding finger ring; wherein the change in acceleration is generated when the corresponding finger ring is subjected to pressure, and the pressure is generated when the fingertip hits the tabletop to cause the tabletop to vibrate;

[0096] According to the comparison result of the rotation change amount and the preset rotation threshold, it is determined whether a confirmation operation triggered by tapping the desktop occurs.

[0097] In a third aspect, an embodiment of the present application provides an operating system for a three-dimensional interface, including a head control device, a main control ring, and an auxiliary control ring;

[0098] The main control ring and the auxiliary control ring include a first posture sensor, a first Bluetooth module, a first storage chip and a mode button, wherein:

[0099] The first posture sensor is used to collect rotation data and position data, and store the data through the first storage chip;

[0100] The first Bluetooth module is used for Bluetooth connection between finger rings and Bluetooth connection with the head control device;

[0101] The mode button is used to send instruction information for determining an interaction mode to the head control device in response to a user operation;

[0102] The head control device includes a second posture sensor, a second Bluetooth module, a second storage chip and a second computing chip:

[0103] The second posture sensor is used to collect position data and rotation data;

[0104] The second Bluetooth module is used to establish a Bluetooth connection with the master control ring;

[0105] The second storage chip comprises a program storage area and a data storage area, wherein the data storage area is used to store the data collected by the chip itself and the data collected by the ring;

[0106] The computing chip is used to execute any one of the methods in the first aspect according to a computer program stored in a program storage area.

[0107] In a fourth aspect, an embodiment of the present application provides a head-mounted display device, wherein the head-mounted display device includes a head control device that implements any three-dimensional interface operation method provided in the first aspect.

[0108] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a head control device to execute any one of the three-dimensional interface operation methods provided in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0110] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0111] Figure 2A structural diagram of an operating system provided in an embodiment of the present application;

[0112] Figure 3A A gesture interaction mode scene diagram provided in an embodiment of the present application;

[0113] Figure 3B Another gesture interaction mode scene diagram provided in an embodiment of the present application;

[0114] Figure 4 A desktop touch interaction mode scene diagram provided in an embodiment of the present application;

[0115] Figure 5 A flowchart of a method for operating a head mounted display device provided in an embodiment of the present application;

[0116] Figure 6 A flowchart for determining the aiming line in the gesture interaction mode provided in an embodiment of the present application;

[0117] Figure 7 A schematic diagram of the movement of the aiming line in the gesture interaction mode provided in an embodiment of the present application;

[0118] Figure 8 A flowchart of the confirmation operation in the gesture interaction mode provided in an embodiment of the present application;

[0119] Fig. 9 A schematic diagram of gesture interaction operations provided in an embodiment of the present application;

[0120] Fig.10 A flowchart for determining the aiming line in the desktop touch interaction mode provided in an embodiment of the present application;

[0121] Fig.11 A schematic diagram of the coordinate axes for the Overlay calibration plane provided in an embodiment of the present application;

[0122] Fig.12 A flow chart of a method for mapping touch calibration points in a desktop touch interaction mode provided in an embodiment of the present application;

[0123] Fig.13 A flowchart of the confirmation operation in the desktop touch interaction mode provided in an embodiment of the present application;

[0124] Fig.14 A schematic diagram of a desktop touch interaction operation provided in an embodiment of the present application;

[0125] Fig.15 A timing diagram of the interaction between the operating system and the head-mounted display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0126] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0127] Based on the exemplary embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete technical solution separately.

[0128] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0129] The terms "first", "second", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise indicated. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented in an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0130] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0131] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0132] The following is an overview of the design concept of the embodiments of the present application in conjunction with application scenarios.

[0133] At present, the gesture functions commonly used in most head-mounted display devices need to be implemented with the help of the device itself, which will increase the size and weight of the head-mounted display device. In addition, the gesture function of the device itself is simple and usually has only one operation mode. During use, users need to continuously raise their hands to operate, which will cause fatigue in the user's arms for a long time.

[0134] In view of this, the embodiment of the present application provides an operation method of a three-dimensional interface, including a head control device, a main control ring and an auxiliary control ring. The main control ring and the auxiliary control ring can be worn on two fingers of one hand, or placed on a desktop, so as to realize two interaction modes: gesture interaction and desktop touch interaction. In the gesture interaction mode, the head control device determines the aiming line in the three-dimensional interface according to its own position data and the position data and rotation data of the main control ring, and determines the distance between two fingers according to the position data of the main control ring and the auxiliary control ring to realize the confirmation operation triggered by finger pinching. Since the determination of the aiming line in the gesture interaction mode uses the data of the head and the hand, the aiming process of the aiming line in the three-dimensional interface is more stable and more accurate, thereby improving the accuracy of the finger pinching operation judgment and ensuring the accuracy of the interaction; in the desktop touch interaction mode, the head control device determines the aiming line in the three-dimensional interface according to the position data of the fingertips collected by the finger rings on either side, and determines the rotation change according to the rotation data of the corresponding finger ring to realize the confirmation operation triggered by tapping the desktop. By placing the finger rings on the desktop to form a desktop touch area, the user can achieve interaction by moving the fingertips and tapping the desktop. In this way, in actual applications, the user can trigger the indication information by operating the main control ring according to the actual interaction needs, and the head control device can determine the interaction mode according to the indication information, thereby realizing flexible selection of interaction modes in different scenarios, making the operation of the head-mounted display device more flexible and diverse, and the usage scenarios are more extensive.

[0135] See also Figure 1 , is a schematic diagram of an application scenario of an embodiment of the present application, including a head-mounted display device and an operating system for interaction, the operating system including a head control device, a main control ring and an auxiliary control ring. The head control device is fixed to one side of the head-mounted display device, the main control ring and the auxiliary control ring can be worn on the user's hand for gesture interaction, or placed on the desktop for desktop touch interaction.

[0136] Optionally, the head-mounted display device of the embodiment of the present application includes but is not limited to AR glasses, VR glasses and MR glasses lights.

[0137] It should be noted that Figure 1 This is only an example of an application scenario. Optionally, the head control device may also be integrated inside the head-mounted display device.

[0138] See also Figure 2, is a structural diagram of the operating system provided in an embodiment of the present application, wherein the main control ring includes a first posture sensor, a first Bluetooth module, a first storage chip and a mode button. Among them, the first posture sensor is used to collect rotation data and position data, the first Bluetooth module is used to connect to the head control device and the auxiliary control ring by Bluetooth, the first storage chip is used to store the data collected by itself and the data collected by the auxiliary control ring, and the mode button is used to send indication information of determining the interaction mode to the head control device in response to the user operation. The auxiliary control ring also includes a first posture sensor, a first Bluetooth module and a first storage chip. Among them, the first posture sensor is used to collect rotation data and position data, the first Bluetooth module is used to connect to the main control ring by Bluetooth, and the first storage chip is used to store the data collected by itself. The head control device includes a second posture sensor, a second Bluetooth module, a second storage chip and a computing chip. Among them, the second posture sensor is used to collect position data and rotation data, the second Bluetooth module is used to connect to the main control ring by Bluetooth, the second storage chip includes a data storage area and a program storage area, the data storage area is used to store the data collected by itself and the data collected by the ring, and the computing chip is used to calculate the data according to the computer program stored in the program storage area to realize the interactive operation.

[0139] Optionally, the first posture sensor includes a first IMU sensor and a first ToF sensor. In this case, the first IMU sensor in the main control ring is used to collect the rotation data of the finger, and the first ToF sensor is used to collect the position data of the auxiliary control ring. Similarly, the first IMU sensor of the auxiliary control ring is used to collect the rotation data of the finger, and the first ToF sensor is used to collect the position data of the main control ring. The second posture sensor includes a second IMU sensor and a second ToF sensor. In this case, the second IMU sensor is used to collect the rotation data of the head, and the second ToF sensor is used to collect its own position data and the position data of the main control ring.

[0140] Optionally, the main control ring and the auxiliary control ring may further include adjustment knobs for fixing the rings on fingers.

[0141] Optionally, the head control device may further include a fixing buckle for fixing the head control device on the head-mounted display device.

[0142] Optionally, the main control ring and the auxiliary control ring may further include a first system indicator light, and the head control device may further include a second system indicator light, for indicating the Bluetooth connection status among the three.

[0143] Optionally, the main control ring and the auxiliary control ring may further include a first switch button, and the head control device may further include a second switch button for controlling the switch of the device.

[0144] Optionally, the mode button and the first switch button on the main control ring may be the same, and different functions may be represented by setting different button modes.

[0145] In the embodiments of the present application, the usage of the head control device, the main control ring and the auxiliary control ring may be informed to the user in the instruction manual so that the user can use them correctly according to the instructions.

[0146] In one example, the main control ring and the auxiliary control ring can be worn on one hand to implement gesture interaction with a head-mounted display device.

[0147] For example, the head control device is fixed on the outside of the head mounted display device. Figure 3A As shown in FIG. 1 , when the head control device is fixed to the left temple of the head mounted display device through the second fixing buckle, the main control ring and the auxiliary control ring are worn on the left hand, as shown in FIG. Figure 3B As shown, when the head control device is fixed to the right temple of the head-mounted display device through the second fixing buckle, the main control ring and the auxiliary control ring are worn on the right hand.

[0148] It should be noted that when the head control device is fixed on the inner side of the head-mounted display device, it can be integrated on the left side of the device or on the right side of the device, and can also be flexibly set according to the actual volume of the device.

[0149] Generally, the main control ring and the auxiliary control ring can be worn on two different fingers of the same hand.

[0150] In actual application, the main control ring can be worn at the base of the thumb and fixed by adjusting the knob, and the auxiliary control ring can be worn at the base of other fingers (such as the index finger, middle finger, ring finger or little finger) on the same hand and fixed by adjusting the knob, thereby increasing the gap between the two fingers and improving the gesture recognition rate.

[0151] In one example, the main control ring and the auxiliary control ring can be placed on a flat table, such as Figure 4 As shown, the main control ring can be placed on the desktop on the right hand side, and the auxiliary control ring can be placed on the desktop on the left hand side, or the main control ring can be placed on the desktop on the left and right sides, and the auxiliary control ring can be placed on the desktop on the right hand side. In specific implementation, the ring can be placed upright on the desktop, with the first system indicator facing the direction of the finger touching the desktop. On the one hand, the stability of the placement of the ring can be maintained, and on the other hand, the first ToF sensor inside the ring can be closer to the desktop and facing the operating hand, thereby improving the success rate of desktop touch recognition.

[0152] In the embodiment of the present application, before using the above operating system to interact, a Bluetooth connection must be established first.

[0153] Taking the head control device as an example, after the ring is worn or placed, short press the switch button of the head control device, the main control ring and the auxiliary control ring to start the head control device, the main control ring and the auxiliary control ring respectively. During the startup process, the system indicator lights of the head control device, the main control ring and the auxiliary control ring are displayed in blue, and go out after the startup is successful to enter the Bluetooth pairing process. During the pairing process, long press the second switch button of the head control device, the second system indicator light starts to flash green, flashes once every 1 second, and flashes 5 times, indicating that the head control device and the head-mounted display device are successfully connected to the Bluetooth. If the connection fails, the system indicator light of the head control device flashes red, and the user can re-pair by long pressing the switch button again until the Bluetooth connection is successful. Further, long press the first switch button of the main control ring, the first system indicator light starts to flash green, flashes once every 1 second, and flashes 5 times, indicating that the main control ring and the head control device are successfully connected to the Bluetooth. If the connection fails, the first system indicator light of the main control ring flashes red, and the user can re-pair by long pressing the first switch button again until the Bluetooth connection is successful. Finally, long press the first switch button of the auxiliary control ring, and the first system indicator light of the auxiliary control ring begins to flash green, flashing once every 1 second. After flashing 5 times, it means that the auxiliary control ring and the main control ring are successfully connected by Bluetooth. If the connection fails, the first system indicator light of the auxiliary control ring flashes red. The user can re-pair by long pressing the first switch button again until the Bluetooth connection is successful.

[0154] It should be noted that the embodiment of the present application does not impose any restrictive requirements on the indication method of the system indicator light and the usage of the switch button, and they can be adjusted according to actual needs.

[0155] It should be noted that when the head control device is integrated into a head-mounted display device, the Bluetooth connection process between the head control device and the head-mounted display device can be skipped, and the Bluetooth connection process between the main control ring and the head-mounted display device can be directly entered.

[0156] For a three-dimensional interface displayed by a head-mounted display device, an embodiment of the present application provides a method for operating the three-dimensional interface, which is executed by a head control device, such as Figure 5 The following steps are shown:

[0157] S501: Determine the interaction mode according to the indication information sent by the main control ring. When it is the gesture interaction mode, execute S502; when it is the desktop touch interaction mode, execute S504.

[0158] Since the main control ring and the auxiliary control ring have two usage scenarios, one is wearing them on two fingers of one hand, and the other is placing them on a desktop, different situations have different interaction methods, so two interaction modes, gesture interaction and desktop touch, can be realized.

[0159] In one example, after the Bluetooth connection is successful, the user wears the main control ring and the auxiliary control ring on two fingers of one hand, and the head control device enters the gesture interaction mode by default. At this time, the user does not need to operate the switch button of the main control ring, that is, the main control ring does not send instruction information to the head control device. When the user wants to switch to the desktop touch interaction mode, he only needs to take off the main control ring and the auxiliary control ring and place them on the desktop, and send instruction information to the head control device by short pressing the switch button of the main control ring to notify the head control device to enter the desktop touch interaction mode.

[0160] In another example, when a default interaction mode is not set, after the Bluetooth connection is successful, the user can send instruction information by operating the master control ring to select an interaction mode.

[0161] For example, when the user wants to enter the gesture interaction mode, the head control device is fixed on one side of the head-mounted display device, the main control ring and the auxiliary control ring are worn on two fingers of one hand, and the first switch button of the main control ring is short pressed twice in a row so that the main control ring sends an indication information including two continuous level signals to the head control device, thereby informing the head control device to enter the gesture interaction mode; when the user wants to enter the desktop touch interaction mode, the head control device is fixed on one side of the head-mounted display device, the main control ring and the auxiliary control ring are placed on the desktop, and the switch button of the main control ring is short pressed three times in a row so that the main control ring sends an indication information including three continuous level signals to the head-mounted display device, thereby informing the head control device to enter the desktop touch interaction mode.

[0162] It should be noted that the embodiment of the present application does not impose any restrictive requirements on the pressing method (such as long press, short press, number of times) of the switch button of the main control ring and the function indicated by each pressing method.

[0163] S502: Determine the aiming line in the three-dimensional interface according to the position data of the device and the position data of the main control ring.

[0164] In gesture interaction mode, the user uses the hands wearing the main control ring and the auxiliary control ring to operate, and determines the aiming line in the three-dimensional interface of the head-mounted display device by moving the hands up, down, left, right, and rotating. For the specific process, see Figure 6 , mainly includes the following steps:

[0165] S5021: Determine the starting point of the aiming line based on the position data of the device and the position data of the main control ring.

[0166] When entering the gesture interaction mode, the ToF sensor of the head control device will collect its own position data and the position data of the master control ring in real time, and store the collected position data in the data storage area of ​​the storage chip. At the same time, the IMU sensor of the master control ring will also collect its own rotation data in real time, store the collected rotation data in the data storage area of ​​the storage chip, and send it to the head control device via Bluetooth connection. The head control device also stores the rotation data received via Bluetooth connection in the storage chip, and the data in the storage chip is transmitted to the computing chip to calculate the starting point position of the aiming line in the three-dimensional interface.

[0167] When the master ring is worn on the left hand, the starting point is the middle position between the left side of the head (head control device) and the left hand (master ring), and the calculation method is as follows:

[0168] x0=x h -x t *a Formula 1

[0169] y0=y h -y t *b Formula 2

[0170] z0=z h Formula 3

[0171] When the master ring is worn on the right hand, the starting point is the middle position between the right side of the head (head control device) and the right hand (master ring), and the calculation method is:

[0172] x0=x h +x t *a Formula 4

[0173] y0=y h -y t *b Formula 5

[0174] z0=z h Formula 6

[0175] Among them, (x h ,y h , z h ) is the position coordinate of the head control device, (x t ,y t , z t ) is the position coordinate of the main control ring, a is the offset coefficient of the X axis, and b is the offset coefficient of the Y axis.

[0176] Optionally, the master ring is worn on the thumb, the default value of a is 0.15, and the default value of b is 0.16.

[0177] When the head control device and the main control ring are located on one side, the above two calculation methods are used to make the starting point of the aiming line located in the middle of the hand and the head, which is convenient for user operation.

[0178] S5022: Taking the starting point as the origin, a ray directed toward the three-dimensional interface is rendered according to the direction represented by the rotation data of the main control ring to obtain a sighting line.

[0179] When entering gesture interaction mode, the IMU sensor of the main control ring will collect its own rotation data in real time and store it in the storage chip. The rotation data can be sent to the head control device via Bluetooth connection.

[0180] When the head control device is integrated into the head-mounted display device, the head control device directly uses the starting point position as the origin, and renders a ray directed to the three-dimensional interface in the direction represented by the rotation data of the master control ring to obtain the aiming line. When the head control device is independent of the head-mounted display device, the rotation data and the starting point position of the master control ring are sent to the head-mounted display device by the head control device via Bluetooth connection, so that the head-mounted display device uses the starting point position as the origin, and renders a ray directed to the three-dimensional interface in the direction represented by the rotation data to obtain the aiming line.

[0181] Specifically, the rotation data is represented in the form of a quaternion. During the aiming line rendering process, the direction of the aiming line is obtained by multiplying the quaternion with the initialized rotation matrix. Then, with the starting point position as the origin, a ray is rendered in this direction toward the three-dimensional interface to obtain the aiming line for interaction. The movement and rotation direction of the aiming line are consistent with the movement and rotation direction of the hand where the ring is located, but the starting point position always remains fixed.

[0182] like Figure 7 The figure shows a schematic diagram of the movement of the aiming line. The aiming line takes the starting point as the origin and moves with the up, down, left and right movements of the hand.

[0183] In gesture interaction mode, the head control device determines the starting point position and direction of the aiming line based on its own position data, the position data of the main control ring and the rotation data, and realizes the joint calibration of the aiming line in the three-dimensional interface by the head and hand. While improving the stability and accuracy of the aiming line, the user can change the movement of the aiming line in the three-dimensional interface by moving his fingers to select the target, which is simple and convenient to operate.

[0184] S503: After the aiming line is aimed at the target on the three-dimensional interface, the distance between the two fingers is determined according to the position data of each other collected by the main control ring and the auxiliary control ring to realize the confirmation operation triggered by the finger pinching.

[0185] The user controls the aiming line in the three-dimensional interface by moving the finger where the main control ring is located. When the aiming line is aimed at the target in the three-dimensional interface, the user confirms the operation by pinching the two fingers wearing the main control ring and the auxiliary control ring.

[0186] In gesture interaction mode, the ToF sensor of the main control ring will monitor the position data of the auxiliary control ring in real time and store the position data in the storage chip. At the same time, the ToF sensor of the auxiliary control ring will also monitor the position data of the main control ring in real time, store the position data in the storage chip, and send it to the main control ring via Bluetooth connection. The main control ring sends the position data stored in the storage chip (including the position data collected by itself and the position data received from the auxiliary control ring) to the head control device via Bluetooth connection. The computing chip of the head control device executes the confirmation logic after the aiming line is aimed at the target based on the position data collected by the main control ring and the auxiliary control ring, such as Figure 8 As shown, it mainly includes the following steps:

[0187] S5031: Determine the first position coordinates of the main control ring according to the fingertip position coordinates and at least one finger joint position coordinates of the hand where the main control ring is located, and determine the second position coordinates of the auxiliary control ring according to the fingertip position coordinates and at least one finger joint position coordinates of the hand where the auxiliary control ring is located.

[0188] The ToF sensor of the main control ring can monitor the coordinates of each point on the finger where the auxiliary control ring is located. Combined with the hand model, the length of the finger joint can be obtained, thereby determining the coordinates of the fingertip and each joint point on the finger where the auxiliary control ring is located. Similarly, the ToF sensor of the auxiliary control ring can also monitor the coordinates of the fingertip and each joint point on the finger where the main control ring is located.

[0189] For example, the main control ring is worn on the thumb and the auxiliary control ring is worn on the base of other fingers other than the thumb. Assuming that the coordinates of the thumb tip position are (x tt ,y tt , z tt ), the position coordinates of the middle joint are (x tj ,y tj , z tj ), the coordinates of the fingertips of other fingers are (x ft ,y ft , z ft ), the first joint position coordinate is (x fj1 ,y fj1 , z fj1 ), the second joint position coordinates (x fj2 ,y fj2 , z fj2 ), at this time, the calculation method of the first position coordinate of the main control ring (i.e., the thumb) is:

[0190] x t =(x tt *c1+x tj *c2) / (c1+c2) Formula 7

[0191] y t =(y tt *c1+y tj *c2) / (c1+c2) Formula 8

[0192] z t =(z tt *c1+z tj *c2) / (c1+c2) Formula 9

[0193] The calculation method of the second position coordinate of the auxiliary control ring (i.e. other fingers) is:

[0194] x f =(x ft *d1+x fj1 *d2+x fj2 *d3) / (d1+d2+d3) Formula 10

[0195] y f =(y ft *d1+y fj1 *d2+y fj2 *d3) / (d1+d2+d3) Formula 11

[0196] z f =(z ft *d1+z fj1 *d2+z fj2 *d3) / (d1+d2+d3) Formula 12

[0197] Among them, c1, c2, d1, d2 and d3 are weighted average parameters.

[0198] Optionally, c1=2, c2=1, d1=6, d2=3, d3=1.

[0199] In the embodiment of the present application, by wearing the main control ring on the thumb and the auxiliary control ring on other fingers, the distance between the two fingers can be increased, thereby avoiding misjudgment of gestures with a small distance between the two fingers and improving the interaction accuracy. On the other hand, by calculating the position of the ring through the coordinates of the fingertip position and the coordinates of the position of at least one finger joint, the accuracy of the calculation of the distance between the two fingers is improved to improve the judgment of the pinch operation, thereby improving the interaction accuracy.

[0200] It should be noted that the embodiment of the present application does not impose any restrictive requirements on the weighted average parameters, which can be adjusted according to actual applications.

[0201] S5032: Determine the distance between two fingers according to the first position coordinates and the second position coordinates, and determine whether a confirmation operation triggered by finger pinching occurs according to a comparison result between the distance between the two fingers and a preset distance threshold.

[0202] Taking the example of wearing the main control ring on the thumb and the auxiliary control ring on the base of other fingers other than the thumb, the distance between the two fingers is calculated as follows:

[0203] L = sqrt[(x t -x f ) 2 +(y t -y f ) 2 +(z t -z f ) 2 ] Formula 13

[0204] When the head control device is integrated into the head-mounted display device, the head control device directly compares the distance between the two fingers with the preset distance threshold, and determines whether a confirmation operation of finger pinching occurs according to the comparison result. When the head control device is independent of the head-mounted display device, the head control device sends the determined distance between the two fingers to the head-mounted display device via Bluetooth connection, so that the head-mounted display device determines whether a confirmation operation triggered by finger pinching occurs according to the comparison result between the distance between the two fingers and the preset distance threshold.

[0205] Specifically, when the distance between the two fingers is less than a preset distance threshold, and the time less than the preset distance threshold exceeds a preset time threshold, the pinch operation is determined to be successful and the target is selected or opened; otherwise, the pinch operation is determined to have failed and the user has not issued a confirmation operation for the selected target.

[0206] like Fig. 9 As shown, it is a schematic diagram of the gesture interaction operation provided in an embodiment of the present application. During the gesture interaction process, the movement of the hands wearing the main control ring and the auxiliary control ring will drive the movement of the aiming line in the display interface to aim at the target. When the distance between the finger wearing the main control ring and the finger wearing the auxiliary control ring is less than the preset distance threshold, it can be determined that the pinch operation is successful, and the user has performed a confirmation operation to select or open the target.

[0207] In gesture interaction mode, since the main control ring and the auxiliary control ring are worn on two fingers of one hand, the distance between the two fingers can be determined by the position coordinates of the two fingers, and based on the comparison result of the distance between the two fingers and the preset distance threshold, it is determined whether the user performs a pinching operation, thereby realizing gesture interaction of the head-mounted display device.

[0208] S504: Determine a sighting line in the three-dimensional interface according to the fingertip position data collected by any finger ring.

[0209] In the desktop touch interaction mode, the sensor of the ring has a certain detection range. When the main control ring and the auxiliary control ring are placed on the desktop, the detection range will form a certain touch operation area on the desktop. The ToF sensor of the main control ring can detect the position data of the left or right fingertips in the touch operation area, and the ToF sensor of the auxiliary control ring can also detect the position data of the left or right fingertips in the touch operation area. Therefore, the head control device can determine the aiming line in the three-dimensional interface based on the fingertip position data collected by any ring. For the specific process, see Fig.10 , mainly includes the following steps:

[0210] S5041: Determine a touch calibration point on the Overlay calibration plane according to the fingertip position data collected by any finger ring.

[0211] The ToF sensor of the head control device is used to monitor the position change of the head-mounted display device, and the IMU sensor is used to monitor the rotation change of the head-mounted display device. After entering the desktop touch interaction mode, the head-mounted display device creates an Overlay calibration plane based on the position data and rotation data monitored by the head control device. The aspect ratio of the Overlay calibration plane can be set to be the same as the aspect ratio of the touch operation area formed by the ring placed on the desktop. The position of the main control ring or the auxiliary control ring is mapped to the origin of the Overlay calibration plane. At this time, the starting point of the aiming line is located on the Overlay calibration plane, which is recorded as the touch calibration point. The direction of the aiming line is always perpendicular to the three-dimensional interface, such as Fig.11 The figure shows a schematic diagram of the coordinate axes of the Overlay calibration plane.

[0212] When the user's head moves, the Overlay calibration plane will always keep the same position and direction as the 3D interface according to the changes in the position data and rotation data collected by the head control device, and the screen area is consistent with the area of ​​the 3D interface. The Overlay calibration plane is a transparent screen, located on the upper layer of the 3D interface and parallel to the 3D interface.

[0213] After the main control ring or auxiliary control ring is placed on the desktop, it can collect the distance and direction of the user's left or right fingertips and transmit the position data to the head control device in real time via Bluetooth connection. The computing chip of the head control device calculates the mapping relationship between the finger touch position and the position of the aiming line on the display screen through the position data, thereby determining the touch calibration point on the Overlay calibration plane. The mapping process of the touch calibration point is as follows: Fig.12 As shown, it mainly includes the following steps:

[0214] S5041_1: Determine the distance between the mapping point on the Overlay calibration plane and the coordinate origin according to the distance from the fingertip to the corresponding ring in the touch operation area of ​​any ring and the aspect ratio.

[0215] The aspect ratio of the touch operation area of ​​the finger ring placed on the desktop is the same as that of the Overlay calibration plane. In the desktop touch interaction mode, the starting point of the aiming line in the three-dimensional interface is switched to the Overlay calibration plane, that is, there is a mapping point of the starting point on the Overlay calibration plane. The head control device can determine the distance between the mapping point and the coordinate origin based on the distance from the fingertip to the corresponding finger ring in the touch operation area of ​​any finger ring and the aspect ratio.

[0216] For example, assuming that the aspect ratio of the touch operation area and the aspect ratio of the Overlay calibration plane are both 1:5, when the ToF sensor measures the distance from the fingertip to the corresponding ring as d, the mapping point on the Overlay calibration plane is at a position 5d away from the coordinate origin.

[0217] S5041_2: Determine the direction of the mapping point on the Overlay calibration plane according to the direction of the fingertip relative to the finger ring in the touch operation area of ​​any finger ring.

[0218] Since the position of the finger ring is the origin of the Overlay calibration plane, and the aspect ratio of the touch operation area is the same as that of the Overlay calibration plane, the direction of the fingertip in the touch operation area relative to the finger ring is the direction of the mapping point on the Overlay calibration plane.

[0219] S5041_3: Determine the touch calibration point on the Overlay calibration plane according to the distance and direction of the mapping point.

[0220] After determining the distance and direction of the mapping point on the Overlay calibration plane, the touch calibration point corresponding to the aiming line in the three-dimensional interface on the Overlay calibration plane can be obtained.

[0221] S5042: Taking the touch calibration point as the origin, rendering a ray perpendicular to the three-dimensional interface to obtain a sighting line.

[0222] When the head control device is integrated into the head-mounted display device, the head control device directly uses the touch calibration point as the origin to render a ray perpendicular to the three-dimensional interface to obtain the aiming line. When the head control device is independent of the head-mounted display device, the head control device sends the position data of the touch calibration point to the head-mounted display device, and the head-mounted display device renders a aiming line perpendicular to the three-dimensional interface with the touch calibration point as the starting point. The intersection of the aiming line and the three-dimensional interface is the aiming point.

[0223] In the embodiment of the present application, an Overlay calibration plane is created and the starting position of the aiming line is switched to a touch calibration point on the plane. Since the aspect ratio of the touch operation area of ​​the ring is the same as that of the Overlay calibration plane, the movement of the fingertip in the touch operation area can drive the movement of the aiming line in the three-dimensional interface, thereby realizing desktop touch operation.

[0224] S505: After the aiming line is aimed at the target on the three-dimensional interface, the rotation change amount is determined according to the rotation data collected by the corresponding finger ring to realize the confirmation operation triggered by tapping the desktop.

[0225] In the desktop touch interaction mode, the user can slide his finger in the corresponding touch operation area of ​​the main control ring or the auxiliary control ring on the desktop. When the position of the user's finger changes, the position of the touch calibration point will also change, thereby driving the position of the aiming ray in the three-dimensional interface to change to aim at the target.

[0226] The IMU sensor of the main control ring can detect the rotation data of the finger, and the IMU sensor of the auxiliary control ring can also detect the rotation data of the finger. After the aiming line is aimed at the target, the user taps the desktop with his fingertips to send a confirmation command, and the tapping of the desktop by the fingertips will cause the desktop to vibrate. When the pressure generated by the desktop vibration acts on the ring, the acceleration of the IMU data of the ring will change, thereby realizing the confirmation operation in the desktop touch interaction mode. The specific process is as follows Fig.13 As shown, it mainly includes the following steps:

[0227] S5051: Determine the rotation change according to the acceleration collected by the corresponding ring.

[0228] The IMU sensor of the main control ring or the auxiliary control ring sends the acceleration representing the vibration of the desktop to the head control device, and the head control device obtains the rotation change according to the acceleration change.

[0229] S5052: Determine whether a confirmation operation triggered by tapping the desktop occurs based on a comparison result between the rotation change amount and a preset rotation threshold.

[0230] When the head control device is integrated into a head-mounted display device, the head control device directly compares the rotation change amount with a preset rotation threshold, and determines whether a confirmation operation of tapping the desktop with a finger occurs based on the comparison result; when the head control device is independent of the head-mounted display device, the head control device sends the rotation change amount to the head-mounted display device via a Bluetooth connection, and the head-mounted display device compares the rotation change amount with a preset rotation threshold, and determines whether a confirmation operation of tapping the desktop with a finger occurs based on the comparison result.

[0231] Specifically, when determining whether a desktop tap has been made based on the rotation change, if the rotation change is greater than a preset rotation threshold, it is determined that the user has performed a confirmation operation of tapping the desktop, thereby selecting a target or opening a target; otherwise, it is determined that the desktop tap has failed and no confirmation response is made.

[0232] like Fig.14 As shown, it is a schematic diagram of the desktop touch interaction operation provided by an embodiment of the present application. During the desktop touch interaction process, the user can use the right hand or left hand to slide in the touch operation area controlled by a single finger ring respectively. The sliding of the finger will control the movement of the aiming line and the aiming point in the three-dimensional interface. After aiming at the target, the user taps the desktop with his fingers. In this way, after the user's tapping action is detected by the main control ring or the auxiliary control ring, the confirmation operation of aiming at the target is completed.

[0233] In the embodiment of the present application, in the desktop touch interaction mode where the main control ring or the auxiliary control ring is placed on the desktop, the head control device uses the Overlay calibration plane to determine the aiming line based on the position data of the fingertips corresponding to any of the rings, thereby completing the rendering of the aiming line in the three-dimensional interface with high aiming accuracy. On the other hand, the user changes the aiming line in the three-dimensional interface through the movement of the fingertips, which is easy to operate and avoids arm fatigue caused by long-term arm lifting operations, effectively improving the user interaction experience.

[0234] In one example, when the user wants to end the interaction, the operating system can be turned off by pressing the switch button of the head control device. For example, when the switch button of the head control device is pressed three times in a row, the head control device is turned off. Therefore, the Bluetooth connection between the main control ring and the head control device fails, and the main control ring is automatically turned off after the Bluetooth connection fails. Similarly, the auxiliary control ring will also be automatically turned off after the Bluetooth connection between the auxiliary control ring and the main control ring fails.

[0235] In the embodiments of the present application, two interaction modes are realized based on the main control ring and the auxiliary control ring in two usage methods. One is to wear it on two fingers of one hand to realize the gesture interaction mode, and the other is to place it on the desktop to realize the desktop touch interaction mode. The two interaction modes are easy to switch. In this way, when interacting, the user can trigger the indication information by operating the main control ring according to the actual interaction needs, and the head control device determines the interaction mode according to the indication information, thereby realizing flexible selection of interaction modes in different scenarios, making the operation of the head-mounted display device more flexible and diverse, and the usage scenarios are more extensive.

[0236] For example, in a standing scenario, the user can wear the main control ring and the auxiliary control ring on their fingers to enter the gesture interaction mode. In a sitting scenario, the user can place the main control ring and the auxiliary control ring on the desktop to enter the desktop touch interaction mode.

[0237] See also Fig.15, which is a timing diagram of interaction between an operating system and a head mounted display device provided in an embodiment of the present application, mainly includes the following steps:

[0238] S1: After the switch buttons of the head-mounted display device, the head control device, the main control ring and the auxiliary control ring are turned on, the head control device is connected to the head-mounted display device via Bluetooth, the main control ring is connected to the head control device via Bluetooth, and the auxiliary control ring is connected to the main control ring via Bluetooth.

[0239] The Bluetooth connection is used to transmit data collected during the interaction process. The pairing process is described in the above embodiment and will not be described in detail here.

[0240] S2: The gesture interaction mode is started by default in the operating system. The head control device is fixed to one side of the head-mounted display device, and the main control ring and the auxiliary control ring are respectively worn on the thumb and other fingers of the corresponding hand on that side.

[0241] By wearing the main control ring and the auxiliary control ring on the thumb and other fingers of one hand, the gap between the main control ring and the auxiliary control ring can be increased, thereby improving the accuracy of gesture recognition.

[0242] S3: The ToF sensor of the head control device collects its own position data and the position data of the main control ring.

[0243] The position data collected by the head control device may be stored in a storage chip of the head control device for subsequent calculation.

[0244] S4: The computing chip of the head control device determines the starting point position of the aiming line according to its own position data and the position data of the main control ring, and sends it to the head-mounted display device via Bluetooth connection.

[0245] The position data of the head control device itself and the position data of the main control ring stored in the memory chip are transmitted to the computing chip for calculating the starting point position of the aiming line. The specific calculation process is referred to the above embodiment and will not be repeated here.

[0246] S5: The IMU sensor of the main control ring collects the rotation data of the thumb in real time, and sends the rotation data to the head control device via Bluetooth connection, and the head control device sends the rotation data to the head-mounted display device via Bluetooth connection.

[0247] The rotation data is expressed in the form of quaternions and is used by the head-mounted display device to determine the direction of the aiming line.

[0248] S6: The head mounted display device takes the starting point as the origin and renders a ray directed toward the three-dimensional interface according to the direction represented by the rotation data to obtain a sighting line.

[0249] Since the aiming line is determined using the position data and rotation data of the master control ring, the user can control the movement of the aiming line through hand movements to aim at the target on the three-dimensional interface.

[0250] S7: The ToF sensor of the main control ring collects the position data of the auxiliary control ring, and the ToF sensor of the auxiliary control ring collects the position data of the main control ring and sends it to the head control device.

[0251] In gesture mode interaction, the position data of the auxiliary control ring collected in real time by the ToF sensor of the main control ring is stored in the storage chip. At the same time, the position data of the main control ring collected in real time by the ToF sensor of the auxiliary control ring is sent to the main control ring via Bluetooth connection. The main control ring sends the position data stored in the storage chip (including the position data collected by itself and the position data received from the auxiliary control ring) to the head control device via Bluetooth connection.

[0252] S8: The head control device determines the distance between the two fingers according to the position data of the auxiliary control ring and the position data of the main control ring, and sends the distance between the two fingers to the head mounted display device via Bluetooth connection.

[0253] The process of calculating the distance between two fingers is as described in the above embodiment and will not be described again here.

[0254] S9: The head mounted display device compares the distance between the two fingers with a preset distance threshold, and determines whether a confirmation operation triggered by finger pinching occurs based on the comparison result.

[0255] Specifically, when the distance between the two fingers is less than a preset distance threshold, and the time less than the preset distance threshold exceeds a preset time threshold, the pinch operation is determined to be successful and the target is selected or opened; otherwise, the pinch operation is determined to have failed and the user has not issued a confirmation operation for the selected target, thereby realizing gesture interaction.

[0256] S10: Short press the switch button of the head control device, and the head control device, main control ring and auxiliary control ring will switch to desktop touch interaction mode.

[0257] When switching to desktop touch interaction mode, the head control device remains fixed, and the main control ring and auxiliary control ring are placed on a flat desktop.

[0258] S11: The head control device collects posture data and sends it to the head-mounted display device.

[0259] Among them, the posture data includes the position data collected by the ToF sensor and the rotation data collected by the IMU sensor, and is sent to the head-mounted display device via Bluetooth connection.

[0260] S12: The head mounted display device creates an Overlay calibration plane according to the received posture data.

[0261] The head-mounted display device creates an Overlay calibration plane based on the received posture data. The position of the main control ring or the auxiliary control ring is mapped to the origin of the Overlay calibration plane. At this time, the starting point of the aiming line is located on the Overlay calibration plane, which is recorded as the touch calibration point. The direction of the aiming line is always perpendicular to the three-dimensional interface.

[0262] S13: The main control ring or the auxiliary control ring collects the position data of the operating fingertip and sends it to the head control device.

[0263] S14: The head control device determines the touch calibration point on the Overlay calibration plane according to the position data of the fingertip, and sends it to the head mounted display device.

[0264] The process of determining the touch calibration point is as described in the above embodiment and will not be repeated here.

[0265] S15: The head mounted display device uses the touch calibration point as the origin and renders a ray perpendicular to the three-dimensional interface to obtain a sighting line.

[0266] Since the aiming line is determined using the position data of the fingertips, the user can slide his finger in the touch operation area corresponding to the main control ring or the auxiliary control ring on the desktop to change the position of the touch calibration point, thereby causing the position of the aiming ray in the three-dimensional interface to change to aim at the target.

[0267] S16: The IMU sensor of the main control ring or the auxiliary control ring collects the rotation data of the operator and sends it to the head control device.

[0268] S17: The main control device determines the rotation change according to the acceleration of the corresponding ring, and sends it to the head mounted display device.

[0269] Before and after the finger taps the desktop, the angular velocity of the ring on the operating finger will change, so the rotation change can be used to determine whether there is interaction.

[0270] S18: The head mounted display device compares the rotation change with a preset rotation threshold, and determines whether a confirmation operation triggered by tapping the desktop occurs based on the comparison result.

[0271] Specifically, if the rotation change is greater than a preset rotation threshold, it is determined that the user has performed a confirmation operation of tapping the desktop, thereby selecting or opening a target. Otherwise, it is determined that the desktop tapping has failed and no confirmation response is performed, thereby achieving desktop touch interaction.

[0272] S19: Turn off the switch button of the head control device, and the Bluetooth connection between the head control device and the main control ring is disconnected.

[0273] When the user wants to end the interaction, the user can press the switch button of the head control device to turn off the head control device. After the head control device is turned off, the Bluetooth connection between it and the main control ring is disconnected.

[0274] S20: The Bluetooth connection between the auxiliary ring and the main ring is disconnected, and the main ring and the auxiliary ring are turned off.

[0275] In the embodiment of the present application, the operating system composed of the head control device, the main control ring and the auxiliary control ring is independent of the head-mounted display device, has a small size, can be quickly installed during interaction, and can be quickly disassembled after the interaction is over, thereby reducing the interaction burden caused by the weight and volume of the head-mounted display device. Among them, the main control ring and the auxiliary control ring can be worn on the finger or placed on the desktop, thereby providing the head-mounted display device with two modes of gesture interaction and desktop touch interaction, making the operation of the head-mounted display device more flexible and diverse. Among them, in the gesture interaction mode, the head control device determines the aiming line in the three-dimensional interface according to its own position data and the posture data of the main control ring. Since the determination of the aiming line uses two parts of data, the head and the hand, the aiming process of the aiming line in the three-dimensional interface is more stable and more accurate, thereby improving the interaction accuracy. In addition, after the aiming line is controlled by the movement of the hand to aim at the target in the three-dimensional interface, the finger pinching judgment is performed according to the distance between the two fingers determined by the position data of the other party collected by the main control ring and the auxiliary control ring, so as to determine whether the user has confirmed the target and realize gesture interaction; while in the desktop touch interaction mode, the head control device determines the aiming line in the three-dimensional interface according to the position data of the fingertips collected by the ring, and after the aiming line is controlled by sliding the finger on the desktop to aim at the target, the desktop tapping judgment is performed according to the rotation change determined by the rotation data collected by the corresponding ring, so as to determine whether the user has confirmed the target and realize desktop touch interaction.

[0276] Based on the same technical concept, an embodiment of the present application provides a head control device, which can implement the steps of the above-mentioned three-dimensional interface operation method and achieve the same technical effect.

[0277] The structure of the head control device is as follows Figure 2 As shown, it includes a second posture sensor, a second Bluetooth module, a second storage chip and a computing chip;

[0278] The second posture sensor is used to collect position data and rotation data;

[0279] The second Bluetooth module is used to connect to the main control ring via Bluetooth, and the main control ring is connected to the auxiliary control ring via Bluetooth;

[0280] The second storage chip includes a data storage area and a program storage area, the data storage area is used to store the data collected by itself and the data collected by the ring, and the program access area is used to store computer programs;

[0281] The computing chip is used to execute the steps of any three-dimensional interface operation method according to a computer program.

[0282] Based on the same technical concept, an embodiment of the present application provides an operating system that can implement the steps of the above-mentioned three-dimensional interface operation method and achieve the same technical effect.

[0283] The structure of the operating system is as follows Figure 2 As shown, it includes a head control device, a main control ring and an auxiliary control ring, wherein:

[0284] The main control ring and the auxiliary control ring include a first posture sensor, a first Bluetooth module, a first storage chip and a mode button, wherein:

[0285] The first posture sensor is used to collect rotation data and position data, and store the data through the first storage chip;

[0286] The first Bluetooth module is used for Bluetooth connection between the rings and the head control device;

[0287] The mode button is used to send instruction information for determining the interaction mode to the head control device in response to a user operation;

[0288] The head control device includes a second posture sensor, a second Bluetooth module, a second storage chip and a computing chip:

[0289] The second posture sensor is used to collect position data and rotation data;

[0290] The second Bluetooth module is used to establish a Bluetooth connection with the main control ring;

[0291] The second storage chip comprises a program storage area and a data storage area, wherein the data storage area is used to store the data collected by the chip itself and the data collected by the ring;

[0292] The computing chip is used to execute the steps of any three-dimensional interface operation method according to the computer program stored in the program storage area.

[0293] In the embodiment of the present application, the memory chip may be a volatile memory, such as a random-access memory (RAM); the memory chip may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory chip is any other medium that can be used to carry or store a desired computer program in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory chip may be a combination of the above memories.

[0294] The computing chip may include one or more central processing units (CPU), GPU or digital processing unit, etc. The computing chip is used to implement the steps of any of the above three-dimensional interface operation methods when calling the computer program stored in the storage chip.

[0295] In the embodiment of the present application, the head control device, the main control ring and the internal devices of the auxiliary control ring can be connected through a bus, and the bus can be divided into an address bus, a data bus, a control bus, etc.

[0296] For the convenience of description, the operating system can be divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0297] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0298] Based on the same technical concept, an embodiment of the present application provides a head-mounted display device, which includes a head control device that can implement the above-mentioned three-dimensional interface operation method and can achieve the same technical effect.

[0299] The embodiment of the present application also provides a computer-readable storage medium for storing some instructions, which, when executed, can complete the steps of any three-dimensional interface operation method in the aforementioned embodiments.

[0300] An embodiment of the present application also provides a computer program product for storing a computer program, wherein the computer program is used to execute the steps of any one of the three-dimensional interface operation methods in the aforementioned embodiments.

[0301] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0302] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0303] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0304] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0305] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for operating a three-dimensional interface, characterized in that: Applied to a head control device, the head control device is connected to a main control ring via Bluetooth, and the main control ring is connected to an auxiliary control ring via Bluetooth, and the method includes: Determining an interaction mode according to the indication information sent by the master control ring; wherein the indication information is triggered based on a user operation; When the interaction mode is the gesture interaction mode, the aiming line in the three-dimensional interface is determined according to its own position data and the posture data of the main control ring, and after the aiming line is aimed at the target of the three-dimensional interface, the distance between the two fingers is determined according to the position data of each other collected by the main control ring and the auxiliary control ring to realize the confirmation operation triggered by finger pinching; wherein, in the gesture interaction mode, the main control ring and the auxiliary control ring are worn on two fingers of one hand; When the interaction mode is a desktop touch interaction mode, the aiming line in the three-dimensional interface is determined according to the position data of the fingertips collected by any finger ring, and after the aiming line is aimed at the target of the three-dimensional interface, the rotation change is determined according to the rotation data collected by the corresponding finger ring to realize the confirmation operation triggered by tapping the desktop; wherein, in the desktop touch interaction mode, the main control ring and the auxiliary control ring are placed on the desktop.

2. The method according to claim 1, characterized in that The posture data includes position data and rotation data, and determining the aiming line in the three-dimensional interface according to the position data of the main control ring and the posture data of the main control ring includes: Determine the starting point of the aiming line according to its own position data and the position data of the main control ring; Taking the starting point as the origin, a ray directed toward the three-dimensional interface is rendered according to the direction represented by the rotation data of the main control ring to obtain the aiming line.

3. The method according to claim 2, characterized in that When the master ring is worn on the left hand, the starting point position is calculated as follows: x0=x h -x t *a y0=y h -y t *b z0=z h When the master ring is worn on the right hand, the starting point position is calculated as follows: x0=x h +x t *a y0=y h -y t *b z0=z h Among them, (x h ,y h , z h ) is the position coordinate of the head control device, (x t ,y t , z t ) is the position coordinate of the main control ring, a is the offset coefficient of the X-axis, and b is the offset coefficient of the Y-axis.

4. The method according to claim 1, characterized in that The determining the distance between two fingers to implement the confirmation operation triggered by finger pinching according to the position data of each other collected by the main control ring and the auxiliary control ring respectively includes: Determine the first position coordinates of the main control ring according to the fingertip position coordinates and at least one finger joint position coordinates of the hand where the main control ring is located, and determine the second position coordinates of the auxiliary control ring according to the fingertip position coordinates and at least one finger joint position coordinates of the hand where the auxiliary control ring is located; The distance between two fingers is determined according to the first position coordinates and the second position coordinates, and whether a confirmation operation triggered by pinching the fingers occurs is determined according to a comparison result between the distance between the two fingers and a preset distance threshold.

5. The method according to claim 4, characterized in that When the main control ring is worn on the thumb, and the auxiliary control ring is worn on fingers other than the thumb, the first position coordinate is calculated as follows: x t =(x tt *c1+x tj *c2) / (c1+c2) y t =(y tt *c1+y tj *c2) / (c1+c2) With t =(from tt *c1+z tj *c2) / (c1+c2) The second position coordinates are calculated as follows: x f =(x ft *d1+x fj1 *d2+x fj2 *d3) / (d1+d2+d3) <h2 style=";text-align:left;direction:ltr">y<h2 style=";text-align:left;direction:ltr"> f <h2 style=";text-align:left;direction:ltr"> (y)<h2 style=";text-align:left;direction:ltr"> ft <h2 style=";text-align:left;direction:ltr"> *d1+y<h2 style=";text-align:left;direction:ltr"> fj1 <h2 style=";text-align:left;direction:ltr"> *d2+y<h2 style=";text-align:left;direction:ltr"> fj2 <h2 style=";text-align:left;direction:ltr"> *d3) / (d1+d2+d3) <h2 style=";text-align:left;direction:ltr">z<h2 style=";text-align:left;direction:ltr"> f <h2 style=";text-align:left;direction:ltr"> (z)<h2 style=";text-align:left;direction:ltr"> ft <h2 style=";text-align:left;direction:ltr"> *d1+z<h2 style=";text-align:left;direction:ltr"> fj1 <h2 style=";text-align:left;direction:ltr"> *d2+z<h2 style=";text-align:left;direction:ltr"> fj2 <h2 style=";text-align:left;direction:ltr"> *d3) / (d1+d2+d3) Among them, (x tt ,y tt , z tt ) is the coordinate of the fingertip position of the thumb, (x tj ,y tj , z tj ) is the coordinate of the middle joint position of the thumb; (x ft ,y ft , z ft ) is the fingertip position coordinate of the other finger, (x fj1 ,y fj1 , z fj1 ) is the first joint position coordinate of the other fingers, (x fj2 ,y fj2 , z fj2 ) are the second joint position coordinates of the other fingers, and c1, c2, d1, d2 and d3 are weighted average parameters.

6. The method according to claim 1, characterized in that Determining the aiming line in the three-dimensional interface according to the fingertip position data collected by any finger ring includes: According to the fingertip position data collected by any finger ring, the touch calibration point on the Overlay calibration plane is determined; the Overlay calibration plane is a transparent screen, located on the upper layer of the three-dimensional interface and parallel to the three-dimensional interface, and the coordinate origin of the Overlay calibration plane is the position of any finger ring; Taking the touch calibration point as the origin, a ray perpendicular to the three-dimensional interface is rendered to obtain the aiming line.

7. The method according to claim 6, characterized in that The aspect ratio of the touch operation area of ​​the finger ring placed on the desktop is the same as the aspect ratio of the Overlay calibration plane, and the determining of the touch calibration point on the Overlay calibration plane according to the fingertip position data collected by any finger ring includes: Determine the distance between the mapping point on the Overlay calibration plane and the coordinate origin according to the distance from the fingertip to the corresponding ring in the touch operation area of ​​any ring and the aspect ratio; Determine the direction of the mapping point on the Overlay calibration plane according to the direction of the fingertip relative to the finger ring in the touch operation area of ​​any finger ring; According to the distance and direction corresponding to the mapping point, the touch calibration point on the Overlay calibration plane is determined.

8. The method according to claim 1, characterized in that The step of determining the rotation variation according to the rotation data collected by the corresponding finger ring to implement the confirmation operation triggered by tapping the desktop includes: Determine the rotation change according to the acceleration collected by the corresponding finger ring; wherein the change in acceleration is generated when the corresponding finger ring is subjected to pressure, and the pressure is generated when the fingertip hits the tabletop to cause the tabletop to vibrate; According to the comparison result of the rotation change amount and the preset rotation threshold, it is determined whether a confirmation operation triggered by tapping the desktop occurs.

9. A head control device, characterized in that: Including posture sensor, Bluetooth module, storage chip and computing chip; The Bluetooth module is used to connect to the main control ring via Bluetooth, and the main control ring is connected to the auxiliary control ring via Bluetooth; The memory chip includes a data storage area and a program storage area, the data storage area is used to store the data collected by itself and the data collected by the ring, and the program storage area is used to store computer programs; The computing chip performs the following operations according to the computer program: Receiving indication information sent by the master ring through the Bluetooth module, and determining the interaction mode according to the indication information; wherein the indication information is triggered based on the user operation; When the interaction mode is the gesture interaction mode, the aiming line in the three-dimensional interface is determined according to the position data of the posture sensor itself and the posture data of the main control ring, and after the aiming line is aimed at the target of the three-dimensional interface, the position data of the other party sent by the main control ring and the auxiliary control ring are received through the Bluetooth module, and the distance between the two fingers is determined according to the position data of the other party to realize the confirmation operation triggered by finger pinching; wherein, in the gesture interaction mode, the main control ring and the auxiliary control ring are worn on two fingers of one hand; When the interaction mode is a desktop touch interaction mode, the position data of the fingertips collected by any finger ring is received through the Bluetooth module, and the aiming line in the three-dimensional interface is determined according to the position data of the fingertips, and after the aiming line is aimed at the target of the three-dimensional interface, the rotation data collected by the corresponding finger ring is received through the Bluetooth module, and the rotation change amount is determined according to the rotation data to realize the confirmation operation triggered by tapping the desktop; wherein, in the desktop touch interaction mode, the main control ring and the auxiliary control ring are placed on the desktop.

10. An operating system for a three-dimensional interface, characterized in that: It includes a head control device, a main control ring and an auxiliary control ring; The main control ring and the auxiliary control ring include a first posture sensor, a first Bluetooth module, a first storage chip and a mode button, wherein: The first posture sensor is used to collect rotation data and position data, and store the data through the first storage chip; The first Bluetooth module is used for Bluetooth connection between finger rings and Bluetooth connection with the head control device; The mode button is used to send instruction information for determining an interaction mode to the head control device in response to a user operation; The head control device includes a second posture sensor, a second Bluetooth module, a second storage chip and a computing chip: The second posture sensor is used to collect position data and rotation data; The second Bluetooth module is used to establish a Bluetooth connection with the master control ring; The second storage chip comprises a program storage area and a data storage area, wherein the data storage area is used to store the data collected by the chip itself and the data collected by the ring; The computing chip is used to execute the method according to any one of claims 1 to 8 according to the computer program stored in the program storage area.

Images