Positioning method and positioner of combined handle, combined handle and virtual system

Abstract

The application provides a positioning method and a positioner of a combined handle, a combined handle and a virtual system, and solves the technical problem that a control handle commonly used in VR head-mounted equipment and a game handheld machine is urgently needed in the prior art to reduce resource waste. The positioning method of the combined handle provided by the application can be used when the combined handle can form a game device together with the game handheld machine and can also form a virtual system together with the head-mounted equipment, thereby improving the versatility of the combined handle. In addition, when the control handle is installed on the handle shell again, the control handle and the handle shell after being installed again need to be recalibrated, which can make the positioning of the combined handle more accurate when a user uses the combination to experience virtual functions, thereby improving the fluency of human-computer interaction and improving the virtual experience of the user.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of virtual reality, in particular to a positioning method and a positioner of a combined handle, a combined handle and a virtual system. BACKGROUND

[0002] A VR (Virtual Reality) device includes a head-mounted display and a control handle. When a user experiences the VR head-mounted device, the user wears the head-mounted display on the head and holds the control handle in the hand to operate. In the prior art, the control handle is usually designed in correspondence with the head-mounted display. Therefore, in general, the control handle suitable for the head-mounted display can only be suitable for the VR head-mounted device, and cannot be suitable for a game console having similar game functions. Since the control handle of the game console and the control handle of the VR head-mounted device have many similarities, if the user needs to purchase the control handle of the VR head-mounted device and the control handle of the game console respectively, resource waste will be caused.

[0003] In use, the spatial position information of the control handle of the VR head-mounted device is closely related to the VR experience, but the game handle does not need spatial position information. Therefore, the control handle of the game console and the control handle of the VR head-mounted device cannot be simply exchanged or shared. At present, there is an urgent need for a control handle that can be used for the VR head-mounted device and the game console to reduce resource waste. SUMMARY

[0004] Therefore, the present application provides a positioning method and a positioner of a combined handle, a combined handle and a virtual system, which solves the technical problem that there is an urgent need for a control handle that can be used for the VR head-mounted device and the game console to reduce resource waste in the prior art.

[0005] According to one aspect of the present application, the present application provides a positioning method of a combined handle, the combined handle including a control handle and a handle shell, wherein the positioning method of the combined handle includes:

[0006] When the control handle is in communication connection with the head-mounted device, initial positioning information of the control handle and initial positioning information of the handle shell are acquired, positioning calibration values of the control handle and the handle shell are acquired, and the initial positioning information of the control handle and the initial positioning information of the handle shell are calibrated according to the positioning calibration values of the control handle and the handle shell to generate spatial positioning information of the combined handle.

[0007] In an embodiment of the present application, the calibration of the initial position information of the control handle and the initial position information of the handle shell according to the position calibration value of the control handle and the position calibration value of the handle shell, and the generation of the spatial position information of the combined handle, comprises: the calibration of the initial position information of the control handle according to the position calibration value of the control handle, and the generation of the position information of the control handle; the calibration of the initial position information of the handle shell according to the position calibration value of the handle shell, and the generation of the position information of the handle shell; and the integration of the position information of the control handle and the position information of the handle shell, and the generation of the spatial position information of the combined handle.

[0008] In an embodiment of the present application, the handle shell comprises a ring-shaped part and a holding part, wherein a recess is arranged at the center of the holding part to accommodate and fix the control handle; wherein the combined handle further comprises a first infrared sensor arranged on the control handle and a second infrared sensor arranged on the ring-shaped part; wherein, after the control handle is communicatively connected with the head-mounted device, the acquisition of the initial position information of the control handle and the acquisition of the initial position information of the handle shell, comprises: after the control handle is communicatively connected with the head-mounted device, the acquisition of the position information of the head-mounted device.

[0009] The acquisition of the first light spot image of the first infrared sensor and the IMU data of the control handle, and the calculation of the initial position information of the control handle according to the first light spot image, the IMU data of the control handle and the position information of the head-mounted device; and the acquisition of the second light spot image of the second infrared sensor and the IMU data of the handle shell, and the calculation of the initial position information of the handle shell according to the second light spot image, the IMU data of the handle shell and the position information of the head-mounted device.

[0010] In an embodiment of the present application, the acquisition of the position information of the head-mounted device, comprises: the acquisition of a spatial image where the head-mounted device is located; the acquisition of the IMU data of the head-mounted device; and the calculation of the spatial image where the head-mounted device is located and the IMU data of the head-mounted device, to generate the position information of the head-mounted device.

[0011] In an embodiment of the present application, before the acquisition of the initial position information of the control handle and the initial position information of the handle shell after the control handle is communicatively connected with the head-mounted device, the position method of the combined handle further comprises: before the combined handle is shipped, the factory calibration of the test position information of the control handle and the test position information of the handle shell in the combined handle, to generate the position calibration value of the control handle and the position calibration value of the handle shell.

[0012] In an embodiment of the present application, the test positioning information of the control handle and the test positioning information of the handle shell in the combined handle are factory calibrated to generate the positioning calibration value of the control handle and the positioning calibration value of the handle shell, including: obtaining the first test light spot image of the first infrared sensor and the test IMU data of the control handle, the second test light spot image of the second infrared sensor and the test IMU data of the handle shell; obtaining the test positioning information of the control handle according to the first test light spot image and the test IMU data of the control handle; obtaining the test positioning information of the handle shell according to the second test light spot image and the test IMU data of the handle shell; comparing the test positioning information of the control handle with the standard positioning information of the control handle to generate the positioning calibration value of the control handle; and comparing the test positioning information of the handle shell with the standard positioning information of the handle shell to generate the positioning calibration value of the handle shell.

[0013] In an embodiment of the present application, the combined handle further includes a first memory disposed on the control handle and a second memory disposed on the handle shell; wherein after the positioning calibration value of the control handle and the positioning calibration value of the handle shell are generated, the positioning method of the combined handle further includes: storing the positioning calibration value of the control handle and the positioning calibration value of the handle shell in the first memory and the second memory respectively; wherein the obtaining the positioning calibration value of the control handle and the positioning calibration value of the handle shell includes: calling the positioning calibration value of the control handle and the positioning calibration value of the handle shell from the first memory and the second memory respectively.

[0014] In an embodiment of the present application, the positioning calibration value of the control handle includes the position offset value of the control handle and the angle offset value of the control handle; and the positioning calibration value of the handle shell includes the position offset value of the handle shell and the angle offset value of the handle shell.

[0015] As a second aspect of the present application, the present application provides a positioning controller of a combined handle, including: a connection state determination unit for determining the connection state of a control handle and a head-mounted device; an information determination unit for obtaining the initial positioning information of the control handle and obtaining the initial positioning information of the handle shell, obtaining the positioning calibration value of the control handle and the positioning calibration value of the handle shell; and a spatial positioning unit for calibrating the initial positioning information of the control handle and the initial positioning information of the handle shell according to the positioning calibration value of the control handle and the positioning calibration value of the handle shell to generate the spatial positioning information of the combined handle.

[0016] In an embodiment of the present application, the spatial positioning unit comprises: a first calibration module, configured to calibrate initial positioning information of the control handle according to a positioning calibration value of the control handle, and generate positioning information of the control handle; a second calibration module, configured to calibrate initial positioning information of the handle shell according to a positioning calibration value of the handle shell, and generate positioning information of the handle shell; and a positioning module, configured to integrate the positioning information of the control handle and the positioning information of the handle shell, and generate spatial positioning information of the combined handle.

[0017] In an embodiment of the present application, the information determination unit comprises: a head-mounted display information determination module, configured to obtain positioning information of the head-mounted display device; a first positioning information determination module, configured to obtain a first light spot image of the first infrared sensor and IMU data of the control handle, and calculate initial positioning information of the control handle according to the first light spot image, the IMU data of the control handle, and the positioning information of the head-mounted display device; and a second positioning information determination module, configured to obtain a second light spot image of the second infrared sensor and IMU data of the handle shell, and calculate initial positioning information of the handle shell according to the second light spot image, the IMU data of the handle shell, and the positioning information of the head-mounted display device.

[0018] In an embodiment of the present application, the head-mounted display information determination module comprises: an image acquisition module, configured to obtain a spatial image in which the head-mounted display device is located; a degree-of-freedom data acquisition module, configured to obtain IMU data of the head-mounted display device; and a calculation module, configured to calculate the spatial image in which the head-mounted display device is located and the IMU data of the head-mounted display device, and generate positioning information of the head-mounted display device.

[0019] In an embodiment of the present application, the spatial positioning unit further comprises: a factory calibration module, configured to perform factory calibration on test positioning information of the control handle and test positioning information of the handle shell in the combined handle before the combined handle is shipped, and generate a positioning calibration value of the control handle and a positioning calibration value of the handle shell.

[0020] As a third aspect of the present application, the present application further provides a combined handle, comprising: a control handle; a first infrared sensor arranged on the control handle; a handle shell, the handle shell comprising a ring-shaped portion and a holding portion, wherein a recessed portion is arranged at the center of the holding portion to accommodate and fix the control handle; and a second infrared sensor arranged on the ring-shaped portion.

[0021] In an embodiment of the present application, the combined handle further comprises: a first inertial sensor arranged on the control handle, the first inertial sensor being configured to measure IMU data of the control handle; and a second inertial sensor arranged on the handle shell, the second inertial sensor being configured to measure IMU data of the handle shell.

[0022] In an embodiment of the present application, the combined handle further comprises: a first memory arranged on the control handle, the first memory being configured to store a positioning calibration value of the control handle; and a second memory arranged on the handle shell, the second memory being configured to store a positioning calibration value of the handle shell.

[0023] In an embodiment of the present application, the combined handle further comprises: a master control chip arranged on the control handle, the master control chip being connected with the first memory and the second memory respectively; the master control chip being configured to retrieve the positioning calibration value of the control handle from the first memory and retrieve the positioning calibration value of the handle shell from the second memory.

[0024] As a fourth aspect of the present application, the present application further provides a virtual system, comprising: the combined handle described above; the positioning controller of the combined handle described above; a head-mounted device; and a camera arranged on the head-mounted device, the camera being configured to capture a first light spot image of the first infrared sensor, capture a second light spot image of the second infrared sensor, and capture an image of a space where the head-mounted device is located; wherein the positioning controller of the combined handle is arranged on the head-mounted device, and the camera and the control handle are in communication connection with the positioning controller of the combined handle.

[0025] In an embodiment of the present application, the virtual system further comprises: a third inertial sensor arranged on the head-mounted device, the third inertial sensor being configured to measure IMU data of the head-mounted device.

[0026] In an embodiment of the present application, the virtual system comprises a virtual reality system or an augmented reality system.

[0027] The positioning method of the combined handle provided by the present application can improve the versatility of the combined handle, because the combined handle can not only form a game device together with a game console, but also form a virtual system together with a head-mounted device. In addition, when the control handle is installed on the handle shell again, the control handle and the handle shell after being installed again need to be recalibrated, which can make the positioning of the combined handle more accurate when a user uses the combination to experience virtual functions, thereby improving the fluency of human-computer interaction and further improving the virtual experience of the user. BRIEF DESCRIPTION OF DRAWINGS

[0028] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout the figures. The figures together with the detailed description below are incorporated in and constitute a part of the specification, and serve to explain the principles of the present application, and to enable others skilled in the art to make and use the application. In the drawings, like reference numerals refer to like parts throughout the various views.

[0029] Figure 1 Fig. 1 shows a structural schematic diagram of a combined handle according to an embodiment of the present application;

[0030] Figure 2 Fig. 2 shows a flowchart of a position calibration method of a combined handle according to an embodiment of the present application;

[0031] Figure 3 Fig. 3 shows a flowchart of a position calibration method of a combined handle according to another embodiment of the present application;

[0032] Figure 4 Fig. 4 shows a flowchart of a position calibration method of a combined handle according to another embodiment of the present application;

[0033] Figure 5 Fig. 5 shows a flowchart of a position calibration method of a combined handle according to another embodiment of the present application;

[0034] Figure 6 Fig. 6 shows a flowchart of a position calibration method of a combined handle according to another embodiment of the present application;

[0035] Figure 7 Fig. 7 shows a flowchart of a position calibration method of a combined handle according to another embodiment of the present application;

[0036] Figure 8 Fig. 8 shows a flowchart of a position calibration method of a combined handle according to another embodiment of the present application;

[0037] Figure 9 Fig. 9 shows a working principle diagram of a positioning controller of a combined handle according to an embodiment of the present application;

[0038] Figure 10 Fig. 10 shows a working principle diagram of a positioning controller of a combined handle according to another embodiment of the present application;

[0039] Figure 11 Fig. 11 shows a working principle diagram of a positioning controller of a combined handle according to another embodiment of the present application;

[0040] Figure 12 Fig. 12 shows a working principle diagram of a positioning controller of a combined handle according to another embodiment of the present application;

[0041] Figure 13 Fig. 4 shows a working principle diagram of the positioning controller of the combined handle provided by another embodiment of the present application;

[0042] Figure 14 Fig. 4 shows a working principle diagram of the positioning controller of the combined handle provided by another embodiment of the present application;

[0043] Figure 15 Fig. 4 shows a working principle diagram of the positioning controller of the combined handle provided by another embodiment of the present application;

[0044] Figure 16 Fig. 1 shows a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0045] In the description of the present application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise explicitly and specifically limited. All directional indications (such as upper, lower, left, right, front, back, top, bottom, etc.) in the embodiments of the present application are only used to explain the relative position relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications also change accordingly. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units that are not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0046] In addition, the reference to "embodiments" in this document means that the specific features, structures or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean that it refers to the same embodiment, nor is it independent or alternative to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0047] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0048] The head-mounted device needs to be used with the control handle to have a better experience, for example, for a user of a virtual reality game, the VR head-mounted device controller receives the operation instruction and spatial position information uploaded by the control handle to realize the positioning tracking and game control of the control handle.

[0049] The handheld game device generally realizes the control of the host through the keys on the control handle, and the control handle is generally installed on both sides of the host.

[0050] Although the head-mounted device and the control handle of the handheld game device have many similarities in function and key setting, the handle in the current head-mounted device can only be used with the virtual head-mounted handle, and the control handle in the handheld device can only be used with the host in the handheld device, and the two cannot be used together, that is, the user wants to play games on the head-mounted device and the handheld device, and needs to use two different handles. And most of the current control handles are a whole, that is, the parts of the handle cannot be disassembled, and it is difficult to realize universality.

[0051] Exemplary Combination Handle

[0052] The application provides a positioning method of a combined handle which can be applied to a head-mounted device and a handheld device at the same time. Specifically, referring to Figure 1 , Figure 1 The combined handle 32 includes a control handle 33, a first infrared sensor 331 arranged on the control handle 33, a first inertial sensor 332 arranged on the control handle 33, the first inertial sensor 332 being used for measuring the IMU data of the control handle, a handle shell 36, the handle shell 36 including a ring-shaped part 361 and a holding part 362, wherein the central part of the holding part 362 is provided with a recess part to accommodate and fix the control handle 33, a second inertial sensor 364 arranged on the handle shell 36, the second inertial sensor 364 being used for measuring the IMU data of the handle shell 36, and a second infrared sensor 363 arranged on the ring-shaped part 361.

[0053] Specifically, the first infrared sensor 331 and the second infrared sensor 363 are provided with micro infrared lamps for emitting infrared rays. When the micro infrared lamps are in an open state, the micro infrared lamps are photographed by the camera, and the light spots of the micro infrared lamps can be captured. Therefore, the positions of the light spots can be determined as the positions of the infrared emitters. Generally, each first infrared sensor 331 and second infrared sensor 363 is provided with a plurality of micro infrared lamps, and the plurality of lamps of the first infrared sensor 331 and the second infrared sensor 363 form a lamp ring. That is, the first light spot image of the first infrared sensor 331 and the second light spot image of the second infrared sensor 363 can be photographed by the camera. Therefore, the position information of the control handle and the handle shell can be determined by the first light spot image and the second light spot image, respectively. For example, the translational freedom information of the control handle and the handle shell in space can be determined by the first light spot image and the second light spot image.

[0054] The first inertial sensor 332 can detect the IMU data of the control handle 33, i.e., the three rotational freedom data of the control handle 33. The second inertial sensor 364 can detect the IMU data of the handle shell 36, i.e., the three rotational freedom data of the handle shell 36.

[0055] The combined handle in the embodiment of the application can control the first infrared sensor 331 on the control handle 33 to emit light, so that the first light spot image can be photographed, and the position information of the control handle 33 can be determined from the first light spot image. The first inertial sensor 331 on the control handle 33 can detect the three rotational freedom data of the control handle 33. The spatial positioning information of the control handle, i.e., the six freedom data, can be determined according to the first light spot image and the three rotational freedom data of the control handle 33. Similarly, the second infrared sensor 363 on the handle shell 36 can be controlled to emit light, so that the second light spot image can be photographed, and the position information of the handle shell 36 can be determined from the second light spot image. The second inertial sensor 364 on the handle shell 36 can detect the three rotational freedom data of the handle shell 36. The spatial positioning information of the handle shell 36, i.e., the six freedom data, can be determined according to the second light spot image and the three rotational freedom data of the handle shell 36.

[0056] Optionally, the combined handle further comprises a first memory disposed on the control handle 33, the first memory being configured to store the positioning calibration value of the control handle 33; and a second memory disposed on the handle housing 36, the second memory being configured to store the positioning calibration value of the handle housing 36. Since the combined handle is calibrated before leaving the factory, the positioning calibration value of the control handle and the positioning calibration value of the handle housing are obtained for re-calibration of the control handle and the handle housing after the control handle in the combined handle is re-installed on the handle housing, and the calibrated positioning calibration value of the control handle 33 is stored in the first memory integrated on the control handle 33. The calibrated positioning calibration value of the handle housing 36 is stored in the second memory integrated on the handle housing 36.

[0057] Optionally, the combined handle further comprises a master control chip disposed on the control handle 33, the master control chip being connected with the first memory and the second memory respectively; the master control chip is configured to retrieve the positioning calibration value of the control handle from the first memory and retrieve the positioning calibration value of the handle housing from the second memory.

[0058] When the combined handle is calibrated before leaving the factory, and the control handle 33 and the handle housing 36 are re-calibrated after the control handle 33 is re-installed on the handle housing 36, the master control chip directly retrieves the positioning calibration value of the control handle from the first memory and directly retrieves the positioning calibration value of the handle housing from the second memory.

[0059] Positioning method for exemplary combination handle

[0060] Since the control handle in the combined handle described above is detachably installed in the recess of the holding part, when the control handle is used in combination with the game device, the control handle is detached from the recess of the holding part and then installed on the game device for use in combination with the game device for the user to experience the game. When the control handle in the combined handle is used in combination with the head-mounted device to form a virtual system, the control handle is installed in the recess of the holding part.

[0061] At this time, the control handle is detached and then re-installed in the recess of the holding part, so when the combined handle and the head-mounted device form a virtual system, the user experiences the virtual scene through the virtual system, and positioning of the combined handle is required when the user uses the virtual system to experience the virtual scene. Therefore, the application provides a positioning method of the combined handle, Figure 2 As shown in the flowchart of the positioning method of the combined handle provided by an embodiment of the application, the positioning method of the combined handle comprises the following steps: Figure 2 As shown in the flowchart of the positioning method of the combined handle provided by an embodiment of the application, the positioning method of the combined handle comprises the following steps:

[0062] Step S1: After the control handle is communicatively connected with the head-mounted device, the initial positioning information of the control handle is obtained and the initial positioning information of the handle housing is obtained.

[0063] Specifically, the head-mounted device comprises a virtual helmet and a display mounted on the virtual helmet. The communication connection between the control handle and the head-mounted device means that the control handle is in communication connection with the device on the head-mounted device that needs to transmit control instructions, images and other information, for example, the display on the head-mounted device is in communication connection with the control handle; for another example, the head-mounted device has a sensor, when the user wears the virtual helmet, the sensor transmits the sensing model to the control handle, and the control handle judges the virtual function executed by the control handle according to the sensing signal, that is, the control handle and the head-mounted device jointly constitute a virtual system, at this time, the control handle is in communication connection with the sensor.

[0064] It can be understood that no matter which device on the head-mounted device the control handle is in communication connection with, it means that the control handle needs to execute a virtual function, that is, the control handle and the head-mounted device jointly constitute a virtual system, therefore, the specific mode of determining the communication connection between the control handle and the head-mounted device is not limited, as long as the control handle and the head-mounted device jointly constitute a virtual system for the user to experience a virtual scene.

[0065] When it is determined that the control handle is in communication connection with the head-mounted device, it means that the control handle and the head-mounted device jointly constitute a virtual system, the user can wear the head-mounted device and manipulate the control handle to experience a virtual scene, so that the user experiences a virtual scene. When the user experiences a virtual scene, the user's actions in the real environment are mapped in the virtual system, at this time, the spatial positioning information of the combination handle needs to be obtained, since the combination handle comprises the control handle and the handle shell, the control handle and the handle shell need to be positioned and tracked, therefore, the initial positioning information of the control handle and the initial positioning information of the handle shell need to be obtained.

[0066] Step S2: obtaining the positioning calibration value of the control handle and the positioning calibration value of the handle shell;

[0067] Since the control handle in the combination handle and the game console can constitute a game device, and the combination handle and the head-mounted device can constitute a virtual system for the user to experience a virtual scene, when the control handle in the combination handle is used together with the game console to constitute a game device, and then the combination handle is used together with the head-mounted device to constitute a virtual system, the control handle is detached from the handle shell and then reinstalled, therefore, when the control handle is reinstalled in the handle shell, there will be a slight deviation in the relative position between the reinstalled control handle and the handle shell. Therefore, the positioning calibration value of the reinstalled control handle and the positioning calibration value of the handle shell need to be obtained.

[0068] Optionally, the positioning calibration value of the control handle comprises a position offset value of the control handle and an angle offset value of the control handle; and the positioning calibration value of the handle shell comprises a position offset value of the handle shell and an angle offset value of the handle shell.

[0069] Step S3: calibrate the initial positioning information of the control handle and the initial positioning information of the handle shell according to the positioning calibration value of the control handle and the positioning calibration value of the handle shell, and generate the spatial positioning information of the combined handle.

[0070] According to the positioning calibration value of the control handle and the positioning calibration value of the handle shell obtained in step S2, the initial positioning information of the control handle and the initial positioning information of the handle shell obtained in step S1 are calibrated to obtain the spatial positioning information of the combined handle.

[0071] The positioning method of the combined handle provided by the application can improve the versatility of the combined handle, and when the control handle is installed on the handle shell again, the control handle and the handle shell after installation need to be calibrated again, which can make the positioning of the combined handle more accurate when the user uses the combination to experience virtual functions, thereby improving the fluency of human-computer interaction and improving the virtual experience of the user.

[0072] In an embodiment of the application, Figure 3 The flowchart of the positioning method of the combined handle provided by another embodiment of the application is shown in the figure. Figure 3 In the step S3 (calibrating the initial positioning information of the control handle and the initial positioning information of the handle shell according to the positioning calibration value of the control handle and the positioning calibration value of the handle shell, and generating the spatial positioning information of the combined handle) in the positioning method of the combined handle, the following steps are included:

[0073] Step S31: calibrate the initial positioning information of the control handle according to the positioning calibration value of the control handle, and generate the positioning information of the control handle;

[0074] That is, the initial positioning information of the control handle is calibrated according to the positioning calibration value of the control handle to improve the accuracy of the positioning information of the control handle.

[0075] Specifically, the positioning calibration value of the control handle can be pre-calibrated and stored, for example, before the combined handle is shipped, that is, the control handle and the handle shell are calibrated before shipment, and the positioning calibration value of the control handle and the positioning calibration value of the handle shell after calibration before shipment are stored.

[0076] Step S32: calibrate the initial positioning information of the handle shell according to the positioning calibration value of the handle shell, and generate the positioning information of the handle shell;

[0077] That is, the initial positioning information of the handle shell is calibrated according to the positioning calibration value of the handle shell, so as to improve the accuracy of the positioning information of the handle shell.

[0078] Step S33: integrating the positioning information of the control handle and the positioning information of the handle shell to generate the spatial positioning information of the combined handle.

[0079] The positioning information of the calibrated control handle and the positioning information of the handle shell are integrated to obtain the spatial positioning information of the combined handle, thereby improving the positioning accuracy of the combined handle.

[0080] In an embodiment of the present application, Figure 4 As shown in the flowchart of the positioning method of the combined handle provided by another embodiment of the present application, Figure 4 As shown, step S1 (after the control handle is communicatively connected with the head-mounted device, obtaining the initial positioning information of the control handle and obtaining the initial positioning information of the handle shell) specifically includes the following steps:

[0081] Step S11: after the control handle is communicatively connected with the head-mounted device, obtaining the positioning information of the head-mounted device;

[0082] Step S12: obtaining the first light spot image of the first infrared sensor and the IMU data of the control handle, and calculating the initial positioning information of the control handle according to the first light spot image, the IMU data of the control handle and the positioning information of the head-mounted device;

[0083] Specifically, the first infrared sensor is provided with a micro infrared lamp for emitting infrared rays. When the micro infrared lamp is in an open state, the micro infrared lamp is captured by the camera, and the light spot of the micro infrared lamp can be captured. Therefore, the position of the light spot can be determined as the position of the infrared emitter. Generally, each first infrared sensor is provided with a plurality of micro infrared lamps, and the plurality of lamps of the first infrared sensor form a lamp ring. That is, the first light spot image of the first infrared sensor can be captured by the camera, and therefore, the first light spot image of the first infrared sensor can be obtained from the camera.

[0084] When the disassembled control handle is reassembled, the micro infrared lamp of the first infrared emitter is controlled to be turned on, and the first light spot image including the light spot of the micro infrared lamp of the first infrared emitter is obtained by capturing through the camera.

[0085] Specifically, the inertial sensor (Inertial Measurement Unit, IMU for short) is a device for measuring the three-axis attitude angle (or angular rate) and acceleration of an object. Generally, an IMU contains three single-axis accelerometers and three single-axis gyroscopes. The accelerometer detects the acceleration signal of the object in the independent three-axis coordinate system of the carrier, and the gyroscope detects the angular velocity signal of the carrier relative to the navigation coordinate system. Therefore, the inertial sensor can measure the angular velocity and acceleration of the object in three-dimensional space, and calculate the attitude of the object, such as the rotational degrees of freedom of the object, which refers to the three position-related degrees of freedom of up, down, front, back and left and right. The IMU data is the result data detected by the inertial sensor, that is, the angular velocity and acceleration data of an object in three-dimensional space detected by the inertial sensor. Therefore, the first inertial sensor arranged on the control handle can detect the IMU data of the control handle, and the IMU data of the control handle can be used to calculate the attitude of the control handle, such as the rotational degrees of freedom of the control handle, which refers to the three position-related degrees of freedom of up, down, front, back and left and right.

[0086] In summary, the six degrees of freedom (6DOF) data of the control handle, that is, the 6DOF data of the control handle, can be determined according to the first light spot image and the IMU data of the control handle detected by the first inertial sensor.

[0087] According to the 6DOF data of the control handle and the positioning information of the head-mounted device, the initial positioning information of the control handle can be determined.

[0088] Step S13: obtaining the second light spot image of the second infrared sensor and the IMU data of the handle shell, and calculating the initial positioning information of the handle shell according to the second light spot image, the IMU data of the handle shell and the positioning information of the head-mounted device.

[0089] Similarly, the second infrared sensor is provided with a micro infrared lamp for emitting infrared rays. When the micro infrared lamp is in an open state, the light spot of the micro infrared lamp can be captured by the camera, so that the position of the light spot can be determined as the position of the infrared emitter. Generally, each second infrared sensor is provided with a plurality of micro infrared lamps, and the plurality of lamps of the second infrared sensor form a lamp ring. That is, the second light spot image of the second infrared sensor can be captured by the camera, so that the second light spot image of the second infrared sensor can be obtained from the camera.

[0090] When the disassembled control handle is reassembled, the micro infrared lamp controlling the second infrared emitter is opened, and the second light spot image including the light spot of the micro infrared lamp of the second infrared emitter is obtained by the camera. The translational IMU data of the handle shell can be determined from the second light spot image.

[0091] The second inertia sensor arranged on the handle shell can detect the IMU data of the handle shell, and thus the IMU data of the handle shell can be acquired from the second inertia sensor. The IMU data of the handle shell can be used to calculate the posture of the handle shell, for example, to obtain the rotational degrees of freedom of the handle shell, which refer to three position-related degrees of freedom, i.e., up-down, forward-backward and left-right. The six degrees of freedom data of the handle shell, i.e., 6DOF data of the handle shell, can be determined according to the second light spot image and the IMU data of the handle shell detected by the second inertia sensor.

[0092] According to the 6DOF data of the handle shell and the positioning information of the head-mounted device, the initial positioning information of the handle shell can be determined.

[0093] When a user uses the virtual reality device, the head-mounted device is worn on the head to view a game interface through a display on the head-mounted device, and the keys on the control handle are operated by both hands to generate motion control instructions. The motion control instructions include forward movement, backward movement, leftward movement, rightward movement, rotation, jumping and the like. When the user's head and hands move, the corresponding position information changes, and the six degrees of freedom data of the combined handle and the six degrees of freedom data of the head-mounted device are determined and recorded. The motion trajectory of the combined handle is determined according to the change of the six degrees of freedom data of the combined handle, and the motion trajectory of the head-mounted device is determined according to the change of the six degrees of freedom data of the head-mounted device. The corresponding human-computer interaction is generated based on the motion trajectory, for example, the display scene of the game interface is converted according to the motion trajectory. Therefore, after the motion trajectory is determined, the display page is switched, and the corresponding game motion is generated according to the motion control instructions sent by the combined handle.

[0094] It should be noted that the camera for shooting the first light spot image of the first infrared sensor on the control handle and the camera for shooting the second light spot image of the second infrared sensor on the handle shell can be the same camera or different cameras. For example, the camera for shooting the first infrared sensor and the second infrared sensor is the same camera, and can be arranged on the head-mounted device. When the head-mounted device includes a helmet and a mobile terminal, the camera can be arranged on the helmet or the mobile terminal. For another example, the camera for shooting the first infrared sensor and the camera for shooting the second infrared sensor are not the same camera. For example, the camera for shooting the first infrared sensor can be a fixed camera arranged in space, and the camera for shooting the second infrared sensor can be a camera arranged on the head-mounted device.

[0095] In an embodiment of the present application, Figure 5 Fig. 4 shows a flowchart of a positioning method of a combined handle according to another embodiment of the present application, and Figure 5As shown, step S11 (acquiring the positioning information of the head-mounted device after the control handle is communicatively connected with the head-mounted device) specifically comprises the following steps:

[0096] Step S110: Acquire the space image in which the head-mounted device is located.

[0097] Specifically, the space image of the head-mounted device can be captured by a camera. For example, a camera can be arranged on the head-mounted device, and the camera can capture the space image in which the head-mounted device is located. When the head-mounted device comprises a helmet and a mobile terminal, the camera can be arranged on the helmet, or can be a rear camera of the mobile terminal.

[0098] According to the space image in which the head-mounted device is located, the position information of the head-mounted device in the space, i.e., the translational freedom degree data, can be determined.

[0099] Step S111: Acquire the IMU data of the head-mounted device.

[0100] Specifically, the IMU data of the head-mounted device can be detected by a third inertial sensor arranged on the head-mounted device, i.e., the third inertial sensor can detect the IMU data of the head-mounted device.

[0101] Step S112: Calculate the space image in which the head-mounted device is located and the IMU data of the head-mounted device to generate the positioning information of the head-mounted device.

[0102] According to the space image in which the head-mounted device is located and the IMU data of the head-mounted device, the six-degree-of-freedom data of the head-mounted device, i.e., the 6DOF data, can be determined. The positioning information of the head-mounted device is the six-degree-of-freedom data of the head-mounted device.

[0103] In an embodiment of the present application, Figure 6 As shown, the flowchart of the positioning method of the combined handle provided by another embodiment of the present application is as shown in the figure. Figure 6 As shown, before step S1 (acquiring the initial positioning information of the control handle and acquiring the initial positioning information of the handle shell after the control handle is communicatively connected with the head-mounted device), the positioning method of the combined handle further comprises the following steps:

[0104] Step S10: Before the combined handle is shipped, the test positioning information of the control handle and the test positioning information of the handle shell in the combined handle are calibrated to generate the positioning calibration value of the control handle and the positioning calibration value of the handle shell.

[0105] That is, the positioning calibration value of the control handle and the positioning calibration value of the handle shell in step S2 are both calibration values after calibration before the combined handle is shipped.

[0106] Specifically, as shown in the figure, Figure 7As shown, the step S10 (factory calibration of the test positioning information of the control handle in the combination handle and the test positioning information of the handle shell, generating the positioning calibration value of the control handle and the positioning calibration value of the handle shell) specifically includes the following steps:

[0107] Step S101: acquiring the first test light spot image of the first infrared sensor, the test IMU data of the control handle, the second test light spot image of the second infrared sensor and the test IMU data of the handle shell;

[0108] Before the combination handle is factory-calibrated, the first test light spot image of the first infrared sensor arranged on the control handle is photographed by the camera, and the second test light spot image of the second infrared sensor arranged on the handle shell is photographed; the test IMU data of the control handle is detected by the first inertial sensor arranged on the control handle, and the test IMU data of the handle shell is detected by the second inertial sensor arranged on the handle shell.

[0109] Step S102: acquiring the test positioning information of the control handle according to the first test light spot image and the test IMU data of the control handle;

[0110] The test position information of the control handle can be determined through the first test light spot image, and the test angle information of the control handle can be determined through the IMU data of the control handle.

[0111] Step S103: acquiring the test positioning information of the handle shell according to the second test light spot image and the test IMU data of the handle shell;

[0112] The test position information of the handle shell can be determined through the second test light spot image, and the test angle information of the handle shell can be determined through the IMU data of the handle shell.

[0113] Step S104: comparing the test positioning information of the control handle with the standard positioning information of the control handle, and generating the positioning calibration value of the control handle;

[0114] Specifically, the positioning calibration value of the control handle includes a first position offset value and a first angle offset value; and the standard positioning information of the control handle includes a first standard position value and a first standard angle value.

[0115] Wherein, the test position information of the control handle is determined according to the first test light spot image, and the first position offset value is determined according to the test position information and the first standard position value in the standard positioning information.

[0116] The test angle information of the control handle is determined according to the test IMU data of the control handle, and the first angle offset value is determined according to the test angle information and the first standard angle value in the standard positioning information.

[0117] Step S105: Comparing the test positioning information of the handle shell with the standard positioning information of the handle shell to generate a positioning calibration value of the handle shell.

[0118] Specifically, the positioning calibration value of the handle shell includes a second position offset value and a second angle offset value; and the standard positioning information of the handle shell includes a second standard position value and a second standard angle value.

[0119] The second position offset value is determined according to the test position information of the handle shell determined according to the second test light spot image and the second standard position value in the standard positioning information.

[0120] The second angle offset value is determined according to the test angle information of the handle shell determined according to the test IMU data of the handle shell and the second standard angle value in the standard positioning information.

[0121] When the control handle is installed on the hand-held part, and before the factory shipment, the positioning of the control handle and the handle shell is needed to obtain the calibration value. In the subsequent use, if the control handle of the combined handle is disassembled and reassembled, the position of the control handle and the handle shell is calibrated according to the calibration value before the motion trajectory tracking of the combined handle is performed.

[0122] In an embodiment of the present application, the combined handle further includes a first memory arranged on the control handle and a second memory arranged on the handle shell; as shown in Figure 8 After step S105 and before step S2, the positioning method of the combined handle further includes:

[0123] Step S106: Storing the positioning calibration value of the control handle and the positioning calibration value of the handle shell in the first memory and the second memory respectively;

[0124] That is, when the positioning calibration value of the control handle and the positioning calibration value of the handle shell are determined before the factory shipment of the combined handle, the positioning calibration value of the control handle is stored in the first memory arranged on the control handle, and the positioning calibration value of the handle shell is stored in the second memory arranged on the handle shell. The present application stores the positioning calibration value of the control handle and the positioning calibration value of the handle shell in different memories, which facilitates the accurate extraction of the positioning calibration value and further reduces the probability of data loss.

[0125] When the positioning calibration values are stored respectively, the positioning calibration value of the control handle and the positioning calibration value of the handle shell are obtained by calling from the first memory and the second memory respectively. That is, step S2 (obtaining the positioning calibration value of the control handle and the positioning calibration value of the handle shell) specifically includes:

[0126] Step S21: retrieve the positioning calibration value of the control handle and the positioning calibration value of the handle shell from the first memory and the second memory respectively.

[0127] Exemplary positioning controller

[0128] As a third aspect of the present application, the present application also provides a positioning controller of a combination handle, which can be integrated in a head-mounted device or in a control handle in the combination handle. Figure 9 Fig. 1 shows a working principle diagram of a positioning controller of a combination handle according to an embodiment of the present application, and Fig. 2 shows a working principle diagram of a positioning controller of a combination handle according to another embodiment of the present application. Figure 9 Fig. 1 shows a working principle diagram of a positioning controller of a combination handle according to an embodiment of the present application, and Fig. 2 shows a working principle diagram of a positioning controller of a combination handle according to another embodiment of the present application. Figure 1 Fig. 1 shows a working principle diagram of a positioning controller of a combination handle according to an embodiment of the present application, and Fig. 2 shows a working principle diagram of a positioning controller of a combination handle according to another embodiment of the present application.

[0129] The connection state determining unit 11 is configured to determine the connection state of the control handle and the head-mounted device.

[0130] Specifically, the head-mounted device comprises a virtual helmet and a display installed on the virtual helmet. The communication connection of the control handle and the head-mounted device refers to the communication connection of the control handle and the device on the head-mounted device that needs to transmit control instructions, images and other information, for example, the communication connection of the display on the head-mounted device and the control handle; for example, the head-mounted device has a sensor, at this time, the sensor is in communication connection with the connection state determining unit 11, when the user wears the virtual helmet, the sensor transmits the sensing signal to the connection state determining unit 11, and the connection state determining unit 11 determines the virtual function of the control handle according to the sensing signal, that is, the control handle and the head-mounted device together constitute a virtual system, at this time, the control handle is in communication connection with the sensor.

[0131] The information determining unit 12 is configured to obtain the initial positioning information of the control handle and the initial positioning information of the handle shell, and obtain the positioning calibration value of the control handle and the positioning calibration value of the handle shell.

[0132] That is, the information determining unit 12 performs steps S1 and S2 in the positioning method of the combination handle described above.

[0133] Specifically, as shown in Fig. 1, the information determining unit 12 comprises: Figure 9As shown, the information determining unit 12 can be communicatively connected with a first camera 333 for capturing a first light spot image of a first infrared sensor 331 on the control handle 33, a second camera 365 for capturing a second light spot image of a second infrared sensor 363 on the annular portion 361 of the handle shell 36, a first inertial sensor 332 disposed on the control handle 33, and a second inertial sensor 364 disposed on the annular portion 361 of the handle shell 36. The information determining unit 12 receives the first light spot image of the first infrared sensor 331 sent by the first camera 333, receives the IMU data of the control handle 33 sent by the first inertial sensor 332, and determines the initial positioning information of the control handle 33 according to the first light spot image and the IMU data of the control handle 33. The information determining unit 12 receives the second light spot image of the second infrared sensor 363 sent by the second camera 365, receives the IMU data of the handle shell 36 sent by the second inertial sensor 364, and determines the initial positioning information of the handle shell 36 according to the second light spot image and the IMU data of the handle shell 36.

[0134] The spatial positioning unit 13 is configured to calibrate the initial positioning information of the control handle and the initial positioning information of the handle shell according to the positioning calibration value of the control handle and the positioning calibration value of the handle shell, and generate the spatial positioning information of the combined handle.

[0135] The spatial positioning unit 13 is configured to perform step S3 in the positioning method of the combined handle as described above.

[0136] Specifically, the spatial positioning unit 13 is communicatively connected with the information determining unit 12, and the spatial positioning unit 13 receives the initial positioning data of the control handle and the initial positioning data of the handle shell sent by the information determining unit 12, and calibrates the initial positioning information of the control handle and the initial positioning information of the handle shell according to the positioning calibration value of the control handle and the positioning calibration value of the handle shell, and generates the spatial positioning information of the combined handle.

[0137] Specifically, as shown in Figure 10 The positioning calibration value of the control handle and the positioning calibration value of the handle shell are respectively stored in a first memory 333 and a second memory 366, wherein the first memory 333 is integrated in the control handle 33, the second memory 336 is integrated in the handle shell 36, and the spatial positioning unit 13 is communicatively connected with the first memory 333 and the second memory 336, respectively, and the spatial positioning unit 13 retrieves the positioning calibration value of the control handle from the first memory 333 and retrieves the positioning calibration value of the handle shell from the second memory 336.

[0138] The positioning controller of the combined handle provided by the application can be combined with a game console to form a game device, and can also be combined with a head-mounted device to form a virtual system, thereby improving the versatility of the combined handle. In addition, when the control handle is installed on the handle shell again, the control handle and the handle shell after being installed again need to be recalibrated, so that the positioning of the combined handle is more accurate when a user uses the combination to experience virtual functions, thereby improving the fluency of human-computer interaction and improving the virtual experience of the user.

[0139] In an embodiment of the application, Figure 11 The working principle diagram of the positioning controller of the combined handle provided by the application is shown in FIG. 1. Figure 11 As shown in the figure, the spatial positioning unit 13 comprises:

[0140] The first calibration module 131 is configured to calibrate the initial positioning information of the control handle according to the positioning calibration value of the control handle, and generate the positioning information of the control handle.

[0141] Specifically, the first calibration module 131 is in communication connection with the information determination unit 12, and the first calibration module 131 receives the positioning calibration value of the control handle transmitted by the information determination unit 12. The first calibration module 131 can be in communication connection with the first storage 333 storing the positioning calibration value of the control handle, and call the positioning calibration value of the control handle from the first storage. After the first calibration module 131 receives the initial positioning information of the control handle and the positioning calibration value of the control handle, the initial positioning information of the control handle is calibrated according to the positioning calibration value of the control handle, and the positioning information of the control handle is generated.

[0142] The second calibration module 132 is configured to calibrate the initial positioning information of the handle shell according to the positioning calibration value of the handle shell, and generate the positioning information of the handle shell.

[0143] Specifically, the second calibration module 132 is in communication connection with the information determination unit 12, and the second calibration module 132 receives the positioning calibration value of the handle shell transmitted by the information determination unit 12. The first calibration module 131 can be in communication connection with the second storage 366 storing the positioning calibration value of the handle shell, and call the positioning calibration value of the handle shell from the second storage. After the second calibration module 132 receives the initial positioning information of the handle shell and the positioning calibration value of the handle shell, the initial positioning information of the handle shell is calibrated according to the positioning calibration value of the handle shell, and the positioning information of the handle shell is generated.

[0144] The positioning module 133 is configured to integrate the positioning information of the control handle and the positioning information of the handle shell, and generate the spatial positioning information of the combined handle.

[0145] The positioning module 133 is communicatively connected to the first calibration module 131 and the second calibration module 132, respectively. It receives the positioning information of the control handle transmitted by the first calibration module and the positioning information of the handle shell transmitted by the second calibration module. It integrates the positioning information of the control handle and the positioning information of the handle shell to generate the spatial positioning information of the combined handle.

[0146] In one embodiment of the present invention, Figure 12 as well as Figure 13 The diagram shown illustrates the working principle of a positioning controller for a combined handle according to an embodiment of the present invention. Figure 12 as well as Figure 13 As shown, the spatial positioning unit 13 also includes:

[0147] The factory calibration module 134 is used to perform factory calibration on the test positioning information of the control handle and the test positioning information of the handle shell in the combination handle before the combination handle leaves the factory, and generate the positioning calibration value of the control handle and the positioning calibration value of the handle shell.

[0148] Specifically, such as Figure 12 As shown, after the factory calibration module 134 completes the calibration of the combined handle, it can store the positioning calibration values ​​of the control handle and the handle shell. That is, both the positioning calibration values ​​of the control handle and the handle shell are stored in the factory calibration module 134. The factory calibration module 134 is communicatively connected to the first calibration module 131 and the second calibration module 132. Both the first calibration module 131 and the second calibration module 132 can retrieve the positioning calibration values ​​of the control handle and the handle shell from the factory calibration module 134.

[0149] In addition, such as Figure 13 As shown, after the factory calibration module 134 completes the calibration of the combined handle, it can store the positioning calibration value of the control handle in the first memory 333 integrated within the control handle, and the positioning calibration value of the handle shell in the second memory 366 integrated within the handle shell. At this time, the factory calibration module 134 is communicatively connected to both the first memory 333 and the second memory 366, storing the positioning calibration value of the control handle in the first memory 333 and the positioning calibration value of the handle shell in the second memory 366. The first calibration module 131 is communicatively connected to the first memory 333 and can retrieve the positioning calibration value of the control handle from the first memory 333. The second calibration module 132 is communicatively connected to the second memory 366 and can retrieve the positioning calibration value of the handle shell from the second memory 366.

[0150] In one embodiment of the present invention,Figure 14 Fig. 1 shows a working principle diagram of a positioning controller of a combined handle provided by an embodiment of the application, and Fig. 2 shows a working principle diagram of a positioning controller of a combined handle provided by another embodiment of the application. Figure 14 As shown, the information determining unit 12 specifically comprises:

[0151] A head-mounted device information determining module 121 is configured to acquire positioning information of a head-mounted device.

[0152] A first positioning information determining module 122 is configured to acquire a first light spot image of a first infrared sensor and IMU data of a control handle, and calculate initial positioning information of the control handle according to the first light spot image, the IMU data of the control handle and the positioning information of the head-mounted device.

[0153] Specifically, the first positioning information determining module 122 is in communication connection with the head-mounted device information determining module 121, the first calibration module 131, the first inertial sensor 332 and the first camera 333 respectively. The first positioning information determining module 122 receives the IMU data of the control handle transmitted by the first inertial sensor 332 and the first light spot image of the first infrared sensor 332 shot by the first camera 333, and calculates the IMU data of the control handle, the first light spot image and the positioning information of the head-mounted device to determine the initial positioning information of the control handle. When the first positioning information determining module 122 determines the initial positioning information of the control handle, the first positioning information determining module 122 sends the initial positioning information of the control handle to the first calibration module 131.

[0154] A second positioning information determining module 123 is configured to acquire a second light spot image of a second infrared sensor 364 and IMU data of a handle shell, and calculate initial positioning information of the handle shell according to the second light spot image, the IMU data of the handle shell and the positioning information of the head-mounted device.

[0155] Specifically, the second positioning information determining module 123 is in communication connection with the head-mounted device information determining module 121, the second calibration module 132, the second inertial sensor 364 and the second camera 365 respectively. The second positioning information determining module 123 receives the IMU data of the handle shell transmitted by the second inertial sensor 364 and the second light spot image of the second infrared sensor 364 shot by the second camera 365, and calculates the IMU data of the handle shell, the first light spot image and the positioning information of the head-mounted device to determine the initial positioning information of the handle shell. When the second positioning information determining module 123 determines the initial positioning information of the handle shell, the second positioning information determining module 123 sends the initial positioning information of the handle shell to the second calibration module 132.

[0156] In an embodiment of the application, Figure 15 Fig. 1 shows a working principle diagram of a positioning controller of a combined handle provided by an embodiment of the application, and Fig. 2 shows a working principle diagram of a positioning controller of a combined handle provided by another embodiment of the application. Figure 15As shown, the head-mounted device information determination module 121 specifically includes:

[0157] An image acquisition module 1211, configured to acquire a space image of a space where the head-mounted device is located.

[0158] Specifically, when the head-mounted device is provided with a third camera 40, the third camera 40 can capture a space image of a space where the head-mounted device is located, at this time, the image acquisition module 1211 is in communication connection with the third camera 40, and receives the space image of the space where the head-mounted device is located transmitted by the third camera 40.

[0159] A degree-of-freedom data acquisition module 1212, configured to acquire IMU data of the head-mounted device.

[0160] Specifically, the IMU data of the head-mounted device can be detected by a third inertial sensor 41 installed on the head-mounted device, at this time, the degree-of-freedom data acquisition module 1212 is in communication connection with the third inertial sensor 41 installed on the head-mounted device, and receives the IMU data of the head-mounted device detected by the third inertial sensor 41.

[0161] A calculation module 1213, configured to calculate the space image where the head-mounted device is located and the IMU data of the head-mounted device, and generate positioning information of the head-mounted device.

[0162] The calculation module 1213 is in communication connection with the image acquisition module 1211, the degree-of-freedom data acquisition module 1212, the first positioning information determination module 122 and the second positioning information determination module 123 respectively. The calculation module 1213 receives the space image where the head-mounted device is located transmitted by the image acquisition module 1211 and the IMU data of the head-mounted device transmitted by the degree-of-freedom data acquisition module 1212, and calculates the space image where the head-mounted device is located and the IMU data of the head-mounted device, determines the positioning information of the head-mounted device, and transmits the positioning information of the head-mounted device to the first positioning information determination module 122 and the second positioning information determination module 123.

[0163] Exemplary virtual system

[0164] As a fourth aspect of the present application, the present application further provides a virtual system, comprising: the combined handle described above; the positioning controller of the combined handle described above; a head-mounted device; and a camera device arranged on the head-mounted device, the camera device being configured to capture a first light spot image of the first infrared sensor, capture a second light spot image of the second infrared sensor, and capture an image of a space where the head-mounted device is located; wherein the positioning controller of the combined handle is arranged on the head-mounted device, and the camera device and the control handle are in communication connection with the positioning controller of the combined handle.

[0165] Specifically, the camera device can include the first camera, the second camera and the third camera described above, the first camera is used for shooting the first light spot image of the first infrared sensor, the second camera is used for shooting the second light spot image of the second infrared sensor, and the third camera is used for shooting the image of the space where the head-mounted device is located.

[0166] The working principle of the virtual system is described above, and will not be described again.

[0167] Optionally, the virtual system further comprises a third inertial sensor arranged on the head-mounted device, and the third inertial sensor is used for measuring the IMU data of the head-mounted device.

[0168] Optionally, the virtual system comprises a virtual reality system or an augmented reality system.

[0169] Exemplary electronic device

[0170] Hereinafter, the electronic device according to the embodiment of the present application will be described with reference to Figure 16 The structure schematic diagram of the electronic device provided by the embodiment of the present application is shown. Figure 16

[0171] As Figure 16 shown, the electronic device 600 includes one or more processors 601 and a memory 602.

[0172] The processor 601 can be a central processing unit (CPU) or other form of processing unit having data processing and / or information executing capabilities, and can control other components in the electronic device 600 to perform desired functions.

[0173] The memory 601 can include one or more computer program products, which can include various forms of computer readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM), cache memory, and / or the like. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program information can be stored on the computer readable storage medium, and the processor 601 can run the program information to implement the positioning method of the combination handle of the various embodiments of the present application described above or other desired functions.

[0174] In one example, the electronic device 600 can further include an input device 603 and an output device 604, which are interconnected by a bus system and / or other forms of connection mechanism (not shown).

[0175] The input device 603 can include, for example, a keyboard, a mouse, and the like.​

[0176] The output device 604 can output various information to the outside. The output device 604 can include, for example, a display, a communication network and a remote output device connected thereto, and the like.

[0177] Of course, in order to simplify, Figure 16 Only some of the components of the electronic device 600 related to the present application are shown in FIG. 6, and components such as a bus, an input / output interface, and the like are omitted. In addition to this, the electronic device 600 can include any other appropriate components according to the specific application.

[0178] In addition to the above-mentioned method and device, an embodiment of the present application can be a computer program product including computer program information that, when executed by a processor, causes the processor to perform the steps in the positioning method of the combination handle according to various embodiments of the present application described in the specification.

[0179] The computer program product can be written in any combination of one or more programming languages to perform the operations of the embodiments of the present application, including object-oriented programming languages such as Java, C++, and the like, and conventional procedural programming languages such as "C" language or similar programming languages. The program code can be executed entirely on a user computing device, partially on a user device, as a separate software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0180] In addition, an embodiment of the present application can also be a computer readable storage medium having stored thereon computer program information, which, when executed by a processor, causes the processor to perform the steps in the positioning method of the combination handle according to various embodiments of the present application described in the specification.

[0181] The computer readable storage medium can employ any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any appropriate combination of the above.

[0182] The basic principles of the present application are described above in connection with specific embodiments, but it is to be noted that the advantages, benefits, effects and the like mentioned in the present application are only examples and are not to be considered as limiting, and it is not to be considered that these advantages, benefits, effects and the like are necessarily to be possessed by each embodiment of the present application. In addition, the above-described specific details are only for the purpose of illustration and understanding, and are not to be considered as limiting the present application to the above-described specific details, and the present application is not limited to the above-described specific details.

[0183] The block diagrams of the devices, apparatuses, equipment, systems involved in the present application are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations shown in the block diagrams must be made. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, mean "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.

[0184] The above description is only the preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for positioning a combined handle, characterized in that, The combined handle includes a control handle and a handle housing, wherein the positioning method of the combined handle includes: After the control handle is connected to the head-mounted display device, the initial positioning information of the control handle and the initial positioning information of the handle shell are obtained. Obtain the positioning calibration value of the control handle and the positioning calibration value of the handle housing; and The initial positioning information of the control handle and the initial positioning information of the handle shell are calibrated based on the positioning calibration value of the control handle and the positioning calibration value of the handle shell to generate the spatial positioning information of the combined handle.

2. The positioning method for the combined handle according to claim 1, characterized in that, The initial positioning information of the control handle and the initial positioning information of the handle shell are calibrated based on the positioning calibration values ​​of the control handle and the handle shell to generate the spatial positioning information of the combined handle, including: The initial positioning information of the control handle is calibrated according to the positioning calibration value of the control handle to generate the positioning information of the control handle; The initial positioning information of the handle shell is calibrated based on the positioning calibration value of the handle shell to generate the positioning information of the handle shell; and The positioning information of the control handle and the positioning information of the handle shell are integrated to generate the spatial positioning information of the combined handle.

3. The positioning method for the combined handle according to claim 1, characterized in that, The handle housing includes an annular portion and a grip portion, wherein a recess is provided in the center of the grip portion to accommodate and fix the control handle; wherein the combined handle also includes a first infrared sensor disposed on the control handle and a second infrared sensor disposed on the annular portion; Specifically, after the control handle is communicatively connected to the head-mounted display device, the initial positioning information of the control handle and the initial positioning information of the handle shell are obtained, including: Once the control handle is connected to the head-mounted display device, the positioning information of the head-mounted display device is obtained; Acquire the first light spot image from the first infrared sensor and the IMU data from the control handle, and calculate the initial positioning information of the control handle based on the first light spot image, the IMU data from the control handle, and the positioning information of the head-mounted display device; and The second spot image of the second infrared sensor and the IMU data of the handle shell are acquired, and the initial positioning information of the handle shell is calculated based on the second spot image, the IMU data of the handle shell, and the positioning information of the head-mounted display device.

4. The positioning method for the combined handle according to claim 3, characterized in that, Obtaining the positioning information of the head-mounted display device includes: Acquire an image of the space in which the head-mounted display is located; Acquire the IMU data of the head-mounted display device; and The positioning information of the head-mounted display device is generated by calculating the spatial image of the head-mounted display device and the IMU data of the head-mounted display device.

5. The positioning method for the combined handle according to claim 3, characterized in that, After the control handle is communicatively connected to the head-mounted display device, before obtaining the initial positioning information of the control handle and the initial positioning information of the handle shell, the positioning method of the combined handle further includes: Before the combined handle leaves the factory, the test positioning information of the control handle and the test positioning information of the handle shell are calibrated at the factory to generate the positioning calibration value of the control handle and the positioning calibration value of the handle shell.

6. The positioning method for the combined handle according to claim 5, characterized in that, The test positioning information of the control handle and the test positioning information of the handle shell in the combined handle are calibrated at the factory to generate positioning calibration values ​​for the control handle and the handle shell, including: Acquire the first test spot image of the first infrared sensor and the test IMU data of the control handle, the second test spot image of the second infrared sensor and the test IMU data of the handle shell; The test positioning information of the control handle is obtained based on the first test spot image and the test IMU data of the control handle; The test positioning information of the handle shell is obtained based on the second test spot image and the test IMU data of the handle shell. The test positioning information of the control handle is compared with the standard positioning information of the control handle to generate the positioning calibration value of the control handle; and The test positioning information of the handle shell is compared with the standard positioning information of the handle shell to generate the positioning calibration value of the handle shell.

7. The positioning method for the combined handle according to claim 5, characterized in that, The combined handle also includes a first memory disposed on the control handle and a second memory disposed on the handle housing; The positioning method for the combined handle, after generating the positioning calibration values ​​for the control handle and the handle housing, further includes: The positioning calibration value of the control handle and the positioning calibration value of the handle housing are stored in the first memory and the second memory, respectively. The step of obtaining the positioning calibration value of the control handle and the positioning calibration value of the handle housing includes: The positioning calibration values ​​of the control handle and the handle housing are retrieved from the first memory and the second memory, respectively.

8. The positioning method for the combined handle according to claim 1, characterized in that, The positioning calibration value of the control handle includes the position offset value of the control handle and the angle offset value of the control handle; The positioning calibration value of the handle housing includes the position offset value of the handle housing and the angle offset value of the handle housing.

9. A positioning controller for a combined handle, characterized in that, The combined handle includes a control handle and a handle housing, wherein the positioning controller includes: The connection status determination unit is used to determine the connection status between the control handle and the head-mounted display device; An information determination unit is configured to acquire initial positioning information of the control handle and initial positioning information of the handle housing, acquire positioning calibration values ​​of the control handle and the handle housing; and A spatial positioning unit is used to calibrate the initial positioning information of the control handle and the initial positioning information of the handle shell according to the positioning calibration value of the control handle and the positioning calibration value of the handle shell, and generate the spatial positioning information of the combined handle.

10. The positioning controller for the combined handle according to claim 9, characterized in that, The spatial positioning unit includes: The first calibration module is used to calibrate the initial positioning information of the control handle according to the positioning calibration value of the control handle, and generate the positioning information of the control handle; The second calibration module is used to calibrate the initial positioning information of the handle shell according to the positioning calibration value of the handle shell, and generate the positioning information of the handle shell; and The positioning module is used to integrate the positioning information of the control handle and the positioning information of the handle shell to generate the spatial positioning information of the combined handle.

11. The positioning controller for the combined handle according to claim 9, characterized in that, The handle housing includes an annular portion and a gripping portion, wherein a recess is provided at the center of the gripping portion to accommodate and fix the control handle. The combined handle further includes a first infrared sensor disposed on the control handle and a second infrared sensor disposed on the annular portion. The information determination unit includes: The head-mounted display information determination module is used to obtain the positioning information of the head-mounted display device; The first positioning information determination module is used to acquire the first light spot image of the first infrared sensor and the IMU data of the control handle, and calculate the initial positioning information of the control handle based on the first light spot image, the IMU data of the control handle and the positioning information of the head-mounted display device. The second positioning information determination module is used to acquire the second spot image of the second infrared sensor and the IMU data of the handle shell, and calculate the initial positioning information of the handle shell based on the second spot image, the IMU data of the handle shell and the positioning information of the head-mounted display device.

12. The positioning controller for the combined handle according to claim 11, characterized in that, The head-mounted display information determination module includes: The image acquisition module is used to acquire the spatial image of the head-mounted display device. A degree-of-freedom data acquisition module is used to acquire IMU data from the head-mounted display device; and The calculation module is used to calculate the spatial image of the head-mounted display device and the IMU data of the head-mounted display device to generate the positioning information of the head-mounted display device.

13. The positioning controller for the combined handle according to claim 10, characterized in that, The spatial positioning unit further includes: The factory calibration module is used to perform factory calibration on the test positioning information of the control handle and the test positioning information of the handle shell in the combination handle before the combination handle leaves the factory, and generate the positioning calibration value of the control handle and the positioning calibration value of the handle shell.

14. A virtual system, characterized in that, include: Control handle; A first infrared sensor is mounted on the control handle; A handle housing, the handle housing including an annular portion and a grip portion, wherein a recess is provided in the center of the grip portion to receive and fix the control handle; A second infrared sensor is disposed on the annular portion; Positioning controller for the combined handle according to any one of claims 9-13; Head-mounted display devices; and The camera device installed on the head-mounted display device is used to capture a first light spot image of the first infrared sensor, a second light spot image of the second infrared sensor, and an image of the space where the head-mounted display device is located; The positioning controller of the combined handle is located on the head-mounted display device, and the camera device and the control handle are both communicatively connected to the positioning controller of the combined handle.

15. The virtual system according to claim 14, characterized in that, Also includes: A third inertial sensor is installed on the head-mounted display device, which is used to measure the IMU data of the head-mounted display device.

16. The virtual system according to claim 14, characterized in that, The virtual system includes: a virtual reality system or an augmented reality system.

17. The virtual system according to claim 14, characterized in that, Also includes: A first inertial sensor is mounted on the control handle, and the first inertial sensor is used to measure the IMU data of the control handle; as well as A second inertial sensor is disposed on the handle housing, the second inertial sensor being used to measure IMU data of the handle housing.

18. The virtual system according to claim 14, characterized in that, Also includes: A first memory is provided on the control handle, the first memory being used to store the positioning calibration value of the control handle; as well as A second memory is disposed on the handle housing, the second memory being used to store the positioning calibration values ​​of the handle housing.

19. The virtual system according to claim 18, characterized in that, Also includes: The main control chip is disposed on the control handle and is connected to the first memory and the second memory respectively; the main control chip is used to retrieve the positioning calibration value of the control handle from the first memory and the positioning calibration value of the handle shell from the second memory.

Images