Combined handle positioning method and locator, combined handle and virtual system

The combination handle system with VR head-mounted displays and gaming mobile terminals uses infrared sensors and IMUs for accurate calibration, addressing the issue of resource wastage and incompatibility by enabling seamless use across devices, thus enhancing user experience and interaction fluency.

JP7593676B2Active Publication Date: 2024-12-03PIMAX TECH (SHANGHAI) CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023019193
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2022-07-29
Filing Date
2023-02-10
Publication Date
2024-12-03
Estimated Expiration
2043-02-10

AI Technical Summary

Technical Problem

Existing VR head-mounted display devices and gaming mobile terminals require separate control handles, leading to resource wastage and incompatibility, as control handles for VR devices are not easily interchangeable with gaming mobile terminals.

Method used

A combination handle system with a control handle and handle case, utilizing infrared sensors and inertial measurement units (IMUs) for spatial positioning calibration, allowing the handle to be used with both VR head-mounted displays and gaming mobile terminals, and a method for calibrating the handle's positioning information to ensure accurate alignment.

Benefits of technology

Enhances the versatility of the control handle, allowing it to be used across different devices, improving man-machine interaction fluency and user experience by ensuring precise positioning and tracking.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007593676000001
    Figure 0007593676000001
  • Figure 0007593676000002
    Figure 0007593676000002
  • Figure 0007593676000003
    Figure 0007593676000003
Patent Text Reader

Abstract

To solve technical problems that a control handle is required which is generally used for a VR head-mounted display device and a game portable terminal so as to reduce wastes of resources of prior arts by providing a positioning method and locator for a combination handle, a combination handle and a virtual system.SOLUTION: Refer to the flowchart.EFFECT: A positioning method for a combination handle improves the flexibility of a combination handle when not only when the combination handle can constitute a game device together with a game portable terminal, but also when the combination handle can constitute a virtual system together with a heat-mounted display device. When the control handle is fitted to a handle case again, the control handle and handle case need to be calibrated, and when a user uses the combination to experience virtual functions, positioning at the combination handle can be made more accurate to improve the smoothness of machine interaction and further improve user's virtual experience.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to the technical field of virtual reality, and in particular to a positioning method and locator for a combination handle, a combination handle and a virtual system. [Background technology]

[0002] A VR (Virtual Reality) device includes a head-mounted display and a control handle. When a user experiences a VR head-mounted display device, the user wears the head-mounted display device on his / her head and operates the control handle with his / her hands. In the prior art, the control handles are generally designed to correspond to the head-mounted display device, and therefore, the control handles applied to the head-mounted display device are generally only applicable to the VR head-mounted display device, but cannot be applied to the game portable terminal with similar game functions. The control handle of the game portable terminal has many commonalities with the control handle of the VR head-mounted display device, and if the user needs to purchase the control handles of the VR head-mounted display device and the game portable terminal respectively, it will cause a waste of resources.

[0003] In the process of use, the spatial position information of the control handle of the VR head-mounted display device is closely related to the VR experience, but the game handle does not require spatial position information, so the control handle of the game portable terminal and the control handle of the VR head-mounted display device cannot be easily exchanged or shared. At present, there is a need for a control handle that is universally used for the VR head-mounted display device and the game portable terminal to reduce resource waste. Summary of the Invention [Problem to be solved by the invention]

[0004] In view of this, the present invention provides a positioning method and locator for a combination handle, a combination handle and a virtual system, and solves the technical problem of needing a control handle that is commonly used in VR head-mounted display devices and game portable terminals, so as to reduce resource waste in the prior art. [Means for solving the problem]

[0005] According to one aspect of the present invention, the present invention provides a positioning method for a combination handle, the combination handle comprising a control handle and a handle case, the positioning method for the combination handle comprising: After the control handle is communicatively connected to the head-mounted display device, the method includes obtaining initial positioning information of the control handle and obtaining initial positioning information of the handle case; obtaining a positioning calibration value of the control handle and a positioning calibration value of the handle case; and calibrating the initial positioning information of the control handle and the initial positioning information of the handle case based on the positioning calibration value of the control handle and the positioning calibration value of the handle case to generate spatial positioning information of the combination handle.

[0006] In one embodiment of the present invention, calibrating the initial positioning information of the control handle and the initial positioning information of the handle case based on the positioning calibration value of the control handle and the positioning calibration value of the handle case to generate spatial positioning information of the combined handle includes: calibrating the initial positioning information of the control handle based on the positioning calibration value of the control handle to generate positioning information of the control handle; calibrating the initial positioning information of the handle case based on the positioning calibration value of the handle case to generate positioning information of the handle case; and aligning the positioning information of the control handle and the positioning information of the handle case to generate spatial positioning information of the combined handle.

[0007] In one embodiment of the present invention, the handle case comprises an annular portion and a grip portion, and a recessed portion is provided at a central portion of the grip portion for receiving and fixing the control handle, and the combination handle further comprises a first infrared sensor installed on the control handle and a second infrared sensor installed on the annular portion, and acquiring initial positioning information of the control handle after the control handle is communicatively connected to a head-mounted display device, and acquiring initial positioning information of the handle case comprises acquiring positioning information of the head-mounted display device after the control handle is communicatively connected to the head-mounted display device; The method includes acquiring a first spot image of the first infrared sensor and IMU data of the control handle, and calculating initial positioning information of the control handle based on the first spot image, the IMU data of the control handle, and the positioning information of the head-mounted display device; and acquiring a second spot image of the second infrared sensor and IMU data of the handle case, and calculating initial positioning information of the handle case based on the second spot image, the IMU data of the handle case, and the positioning information of the head-mounted display device.

[0008] In one embodiment of the present invention, acquiring positioning information of the head mounted display device includes acquiring a spatial image in which the head mounted display device is located, acquiring IMU data of the head mounted display device, and calculating the spatial image in which the head mounted display device is located and the IMU data of the head mounted display device to generate positioning information of the head mounted display device.

[0009] In one embodiment of the present invention, after the control handle is communicatively connected to a head-mounted display device, before obtaining initial positioning information of the control handle and before obtaining the initial positioning information of the handle case, the positioning method for the combined handle further includes performing shipping calibration on the test positioning information of the control handle and the test positioning information of the handle case in the combined handle before the combined handle is shipped, to generate a positioning calibration value of the control handle and a positioning calibration value of the handle case.

[0010] In one embodiment of the present invention, performing factory calibration on the test positioning information of the control handle and the test positioning information of the handle case in the combined handle to generate a positioning calibration value of the control handle and a positioning calibration value of the handle case includes: obtaining a first test spot image of the first infrared sensor and test IMU data of the control handle, a second test spot image of the second infrared sensor and test IMU data of the handle case, obtaining test positioning information of the control handle based on the first test spot image and the test IMU data of the control handle, obtaining test positioning information of the handle case based on the second test spot image and the test IMU data of the handle case, comparing the test positioning information of the control handle with the standard positioning information of the control handle to generate a positioning calibration value of the control handle, and comparing the test positioning information of the handle case with the standard positioning information of the handle case to generate a positioning calibration value of the handle case.

[0011] In one embodiment of the present invention, the combination handle further includes a first memory installed in the control handle and a second memory installed in the handle case, and after generating the positioning calibration value of the control handle and the positioning calibration value of the handle case, the positioning method of the combination handle further includes storing the positioning calibration value of the control handle and the positioning calibration value of the handle case in the first memory and the second memory, respectively, and obtaining the positioning calibration value of the control handle and the positioning calibration value of the handle case includes retrieving the positioning calibration value of the control handle and the positioning calibration value of the handle case from the first memory and the second memory, respectively.

[0012] In one embodiment of the present invention, the positioning calibration value of the control handle includes a position offset value of the control handle and an angle offset value of the control handle, and the positioning calibration value of the handle case includes a position offset value of the handle case and an angle offset value of the handle case.

[0013] As a second aspect of the present invention, the present invention provides a positioning controller for a combination handle, comprising: a connection status determination unit for determining a connection status between a control handle and a head-mounted display device; an information determination unit for acquiring initial positioning information of the control handle, and acquiring initial positioning information of the handle case, and acquiring a positioning calibration value of the control handle and a positioning calibration value of the handle case; and a spatial positioning unit for calibrating the initial positioning information of the control handle and the initial positioning information of the handle case based on the positioning calibration value of the control handle and the positioning calibration value of the handle case, to generate spatial positioning information of the combination handle.

[0014] In one embodiment of the present invention, the spatial positioning unit includes a first calibration module for calibrating initial positioning information of the control handle based on the positioning calibration value of the control handle to generate positioning information of the control handle, a second calibration module for calibrating initial positioning information of the handle case based on the positioning calibration value of the handle case to generate positioning information of the handle case, and a positioning module for integrating the positioning information of the control handle and the positioning information of the handle case to generate spatial positioning information of the combined handle.

[0015] In one embodiment of the present invention, the information determination unit includes: a head-mounted display information determination module for acquiring positioning information of the head-mounted display device; a first positioning information determination module for acquiring a first spot image of the first infrared sensor and IMU data of the control handle, and calculating initial positioning information of the control handle based on the first spot image, the IMU data of the control handle, and the positioning information of the head-mounted display device; and a second positioning information determination module for acquiring a second spot image of the second infrared sensor and IMU data of the handle case, and calculating initial positioning information of the handle case based on the second spot image, the IMU data of the handle case, and the positioning information of the head-mounted display device.

[0016] In one embodiment of the present invention, the head-mounted display information determination module includes an image acquisition module for acquiring a spatial image in which the head-mounted display device is located, a degree of freedom data acquisition module for acquiring IMU data of the head-mounted display device, and a calculation module for calculating the spatial image in which the head-mounted display device is located and the IMU data of the head-mounted display device to generate positioning information of the head-mounted display device.

[0017] In one embodiment of the present invention, the spatial positioning unit further includes a shipping calibration module for performing shipping calibration on the test positioning information of the control handle and the test positioning information of the handle case in the combined handle before the combined handle is shipped, to generate a positioning calibration value of the control handle and a positioning calibration value of the handle case.

[0018] As a third aspect of the present invention, the present invention further provides a combination handle, comprising: a control handle; a first infrared sensor mounted on the control handle; a handle case having an annular portion and a gripping portion, the handle case having a recessed portion at a central location for receiving and fixing the control handle; and a second infrared sensor mounted on the annular portion.

[0019] In one embodiment of the present invention, the combination handle further includes a first inertial sensor mounted on the control handle for measuring IMU data of the control handle, and a second inertial sensor mounted on the handle case for measuring IMU data of the handle case.

[0020] In one embodiment of the present invention, the combination handle further includes a first memory installed in the control handle for storing a positioning calibration value of the control handle, and a second memory installed in the handle case for storing a positioning calibration value of the handle case.

[0021] In one embodiment of the present invention, the combination handle further includes a main control chip installed in the control handle, the main control chip is respectively connected to the first memory and the second memory, and the main control chip is for retrieving the positioning calibration value of the control handle from the first memory, and retrieving the positioning calibration value of the handle case from the second memory.

[0022] As a fourth aspect of the present invention, the present invention further provides a virtual system, comprising: a combination handle as described above; a positioning controller of the combination handle as described above; a head-mounted display device; and an imaging device mounted on the head-mounted display device for capturing a first spot image of the first infrared sensor, capturing a second spot image of the second infrared sensor and an image of a space in which the head-mounted display device is located, wherein the positioning controller of the combination handle is mounted on the head-mounted display device, and the imaging device and the control handle are both communicatively connected to the positioning controller of the combination handle.

[0023] In one embodiment of the present invention, the virtual system further comprises a third inertial sensor mounted on the head-mounted display device for measuring IMU data of the head-mounted display device.

[0024] In one embodiment of the invention, the virtual system comprises a virtual reality system or an augmented reality system. Effect of the Invention

[0025] In the combination handle positioning method of the present invention, if the combination handle can not only form a game device together with the game portable terminal, but also form a virtual system together with the head-mounted display device, the versatility of the combination handle is improved; and when the control handle is reattached to the handle case, the control handle and the handle case need to be re-calibrated after reattachment; when the user uses the combination to experience virtual functions, the positioning of the combination handle can be more accurate, thereby improving the fluency of man-machine interaction and further improving the user's sense of virtual experience. [Brief description of the drawings]

[0026] The above and other objects, features and advantages of the present invention will become more apparent when the embodiments of the present invention are described in more detail with reference to the drawings. The drawings are intended to provide a further understanding of the embodiments of the present invention, and are a part of the specification for interpreting the present invention together with the embodiments of the present invention, and are not intended to limit the present invention. In the drawings, the same reference numerals generally represent the same elements or steps. [Figure 1] 1 is a structural schematic diagram of a combination handle according to an embodiment of the present invention; [Diagram 2] 4 is a flowchart of a position calibration method for a combination handle according to an embodiment of the present invention. [Diagram 3] 11 is a flowchart of a position calibration method for a combination handle according to another embodiment of the present invention. [Figure 4] 11 is a flowchart of a position calibration method for a combination handle according to another embodiment of the present invention. [Diagram 5] 11 is a flowchart of a position calibration method for a combination handle according to another embodiment of the present invention. [Figure 6] 11 is a flowchart of a position calibration method for a combination handle according to another embodiment of the present invention. [Figure 7] 11 is a flowchart of a position calibration method for a combination handle according to another embodiment of the present invention. [Figure 8] 11 is a flowchart of a position calibration method for a combination handle according to another embodiment of the present invention. [Figure 9] FIG. 2 is a diagram illustrating the operation principle of the positioning controller of the combination handle according to an embodiment of the present invention. [Figure 10] FIG. 11 is a diagram illustrating the operation principle of a positioning controller of a combination handle according to another embodiment of the present invention. [Figure 11] FIG. 11 is a diagram illustrating the operation principle of a positioning controller of a combination handle according to another embodiment of the present invention. [Figure 12] FIG. 11 is a diagram illustrating the operation principle of a positioning controller of a combination handle according to another embodiment of the present invention. [Figure 13] FIG. 11 is a diagram illustrating the operation principle of a positioning controller of a combination handle according to another embodiment of the present invention. [Figure 14] FIG. 11 is a diagram illustrating the operation principle of a positioning controller of a combination handle according to another embodiment of the present invention. [Figure 15] FIG. 11 is a diagram illustrating the operation principle of a positioning controller of a combination handle according to another embodiment of the present invention. [Figure 16] 1 is a structural schematic diagram of an electronic device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] In the description of the present invention, unless otherwise clearly and specifically limited, the meaning of "plurality" is at least two, for example, two, three, etc. All directional indications (e.g., up, down, left, right, front, back, top, bottom, etc.) in the embodiments of the present invention are intended to interpret only the relative positional relationship, movement status, etc. of each member in a certain position (shown in the drawings), and when the specific position changes, the directional indication also changes correspondingly. In addition, the terms "include", "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or equipment including a series of steps or units is not limited to the recited steps or units, and preferably further includes steps or units not recited, or further includes other steps or units specific to these processes, methods, products, or equipment.

[0028] In addition, the term "embodiment" as referred to in this specification means that a particular feature, structure, or characteristic described with reference to the embodiment may be included in at least one embodiment of the present invention. The appearance of the term in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is exclusive of other embodiments. As will be understood by those skilled in the art, both explicitly and implicitly, the embodiment described in this specification may be combined with other embodiments.

[0029] Hereinafter, the technical solutions of the embodiments of the present invention will be described clearly and completely with reference to the drawings of the embodiments of the present invention, and obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, any other embodiments obtained by a person skilled in the art without inventive efforts are all within the protection scope of the present invention.

[0030] Unless a head-mounted display device is used in combination with a control handle, a better experience will not be achieved. For example, in the case of a virtual reality game user, the controller of the VR head-mounted display device receives operation commands and spatial position information uploaded from the control handle to realize functions such as positioning tracking of the control handle and game control.

[0031] A portable terminal game device generally realizes control of the main body by buttons on a control handle, and the control handles are generally attached to both sides of the main body.

[0032] Although the head-mounted display device has many things in common with the control handle of a mobile terminal game device in terms of functions and button placement, the handle of the current head-mounted display device is only used in combination with a virtual head-mounted display handle, and the control handle of the mobile terminal device is only used in combination with the main body of the mobile terminal device, and both cannot be used in general purpose, that is, users need to use two different handles when playing games on the head-mounted display device and the mobile terminal device. Moreover, currently, most control handles are one whole, that is, each part of the handle cannot be disassembled, and it is difficult to realize general purpose.

[0033] Exemplary Combination Handle The present invention provides a positioning method for a combination handle, which can be simultaneously applied to a head-mounted display device and a mobile terminal device. Specifically, referring to Fig. 1, Fig. 1 is a structural schematic diagram of a combination handle according to an embodiment of the present invention, in which the combination handle 32 includes a control handle 33, a first infrared sensor 331 installed on the control handle 33, a first inertial sensor 332 installed on the control handle 33 for measuring IMU data of the control handle, a handle case 36 having an annular part 361 and a grip part 362, and a recessed part for receiving and fixing the control handle 33 at the center of the grip part 362, a second inertial sensor 364 installed on the handle case 36 for measuring IMU data of the handle case 36, and a second infrared sensor 363 installed on the annular part 361.

[0034] Specifically, the first infrared sensor 331 and the second infrared sensor 363 are provided with ultra-small infrared lamps that emit infrared rays, and when the ultra-small infrared lamps are in an on state, the camera can photograph the ultra-small infrared lamps and capture the spots of the ultra-small infrared lamps, so that the positions of the spots can be determined as the positions corresponding to the infrared emitters. Generally, each of the first infrared sensor 331 and the second infrared sensor 363 is provided with a number of ultra-small infrared lamps, and the lamps of the first infrared sensor 331 and the second infrared sensor 363 form a lamp ring. That is, the camera can photograph the first spot image of the first infrared sensor 331 and the second spot image of the second infrared sensor 363, so that the position information of the control handle and the handle case can be determined respectively by the first spot image and the second spot image, for example, the translational degree of freedom information of the control handle and the handle case in space can be determined by the first spot image and the second spot image.

[0035] The first inertial sensor 332 can detect IMU data of the control handle 33, i.e., three rotational degrees of freedom data of the control handle 33. The second inertial sensor 364 can detect IMU data of the handle case 36, i.e., three rotational degrees of freedom data of the handle case 36.

[0036] In the combination handle of the embodiment of the present invention, the first spot image can be taken by controlling the first infrared sensor 331 installed on the control handle 33 to emit light, and the position information of the control handle 33 can be determined from the first spot image, and the first inertial sensor 332 installed on the control handle 33 can detect three rotational degrees of freedom data of the control handle 33, and the spatial positioning information of the control handle, i.e., six degrees of freedom data, can be determined based on the first spot image and the three rotational degrees of freedom data of the control handle 33. Similarly, the second infrared sensor 363 on the handle case 36 can be controlled to emit light, and the second spot image can be taken by controlling the second infrared sensor 363 on the handle case 36 to emit light, and the position information of the handle case 36 can be determined from the second spot image, and the second inertial sensor 364 installed on the handle case 36 can detect three rotational degrees of freedom data of the handle case 36, and the spatial positioning information of the handle case 36, i.e., six degrees of freedom data, can be determined based on the second spot image and the three rotational degrees of freedom data of the handle case 36.

[0037] As an option, the combined handle further includes a first memory installed in the control handle 33 for storing the positioning calibration value of the control handle 33, and a second memory installed in the handle case 36 for storing the positioning calibration value of the handle case 36. Before the combined handle is shipped, the positioning calibration value of the control handle and the positioning calibration value of the handle case are obtained after calibration, so that after the control handle in the combined handle is reattached to the handle case, the control handle and the handle case are recalibrated, and the calibrated positioning calibration value of the control handle 33 is stored in the first memory integrated into the control handle 33. The calibrated positioning calibration value of the handle case 36 is stored in the second memory integrated into the handle case 36.

[0038] As an option, the combination handle further includes a main control chip installed in the control handle 33, the main control chip being respectively connected to a first memory and a second memory, and the main control chip is for retrieving the positioning calibration value of the control handle from the first memory, and retrieving the positioning calibration value of the handle case from the second memory.

[0039] After the combined handle is calibrated before shipment, when the control handle 33 is re-attached to the handle case 36 and the positions of the control handle 33 and the handle case 36 are calibrated again, the main control chip is controlled to directly retrieve the positioning calibration value of the control handle from the first memory and directly retrieve the positioning calibration value of the handle case from the second memory.

[0040] Exemplary Combination Handle Positioning Method Since the control handle in the combination handle described above is removably attached to the recessed portion of the grip, when the control handle is used in combination with a game device, the control handle is removed from the recessed portion of the grip and then attached to the game device so as to allow the user to experience the game. When the control handle and the head-mounted display device in the combination handle form a virtual system, the control handle can be attached to the recessed portion of the grip.

[0041] At this time, the control handle is removed and then reattached to the recessed part of the grip part, and thus the combination handle and the head-mounted display device form a virtual system, whereby the user experiences a virtual scene through the virtual system, and when the user uses the virtual system to experience a virtual scene, it is necessary to position the combination handle, therefore the present invention provides a positioning method for a combination handle, and FIG. 2 is a flowchart of a positioning method for a combination handle according to one embodiment of the present invention, as shown in FIG. 2, the positioning method for a combination handle includes the following steps:

[0042] Step S1: after the control handle is communicatively connected to the head-mounted display device, obtain initial positioning information of the control handle and obtain initial positioning information of the handle case; Specifically, the head-mounted display device includes a virtual helmet and a display attached to the virtual helmet. The communication connection between the control handle and the head-mounted display device refers to the communication connection between the control handle and a device that needs to transmit information such as control commands and images on the head-mounted display device, for example, the communication connection between the display on the head-mounted display device and the control handle, and further, for example, the head-mounted display device has a sensor, and when the user wears the virtual helmet, the sensor transmits a sensing signal to the control handle, and the control handle determines to perform a virtual function according to the sensing signal, that is, the control handle and the head-mounted display device form a virtual system, and at this time, the control handle realizes the communication connection with the sensor.

[0043] As will be understood, regardless of which device on the head-mounted display device the control handle realizes a communication connection with, any of them can indicate that the control handle needs to perform a virtual function, i.e., that the control handle indicates that it configures a virtual system in cooperation with the head-mounted display device; therefore, the specific manner of determining the communication connection between the control handle and the head-mounted display device is not limited, as long as it indicates that the control handle configures a virtual system in cooperation with the head-mounted display device to allow the user to experience a virtual scene.

[0044] After determining that the control handle is communicatively connected to the head-mounted display device, the control handle and the head-mounted display device form a virtual system, so that the user can experience the virtual scene by wearing the head-mounted display device and operating the control handle. When the user experiences the virtual scene, the user's actions in the real environment are mapped in the virtual system, and at this time, the spatial positioning information of the combined handle needs to be obtained, and since the combined handle includes a control handle and a handle case, the control handle and the handle case need to be position-tracked, and therefore the initial positioning information of the control handle and the initial positioning information of the handle case need to be obtained.

[0045] Step S2, obtain the position calibration value of the control handle and the position calibration value of the handle case; The control handle and the game portable terminal in the combination handle can form a game device, so that the combination handle and the head-mounted display device can form a virtual system, so as to allow the user to experience a virtual scene, so that the control handle in the combination handle is used together with the game portable terminal to form a game device, when forming the virtual system with the head-mounted display device, the control handle is removed from the handle case and then reattached, so that after the control handle is reattached to the handle case, the relative position between the control handle and the handle case after reattachment will be slightly deviated, so that it is necessary to obtain the positioning calibration value of the control handle and the positioning calibration value of the handle case after reattachment.

[0046] Optionally, the positioning calibration value of the control handle includes a position offset value of the control handle and an angular offset value of the control handle, and the positioning calibration value of the handle case includes a position offset value of the handle case and an angular offset value of the handle case.

[0047] Step S3: calibrate the initial positioning information of the control handle and the initial positioning information of the handle case according to the positioning calibration value of the control handle and the positioning calibration value of the handle case to generate spatial positioning information of the combined handle.

[0048] Based on the positioning calibration value of the control handle and the positioning calibration device of the handle case obtained in step S2, the initial positioning information of the control handle and the initial positioning information of the handle case obtained in step S1 are calibrated to obtain the spatial positioning information of the combined handle.

[0049] In the combination handle positioning method of the present invention, if the combination handle can not only form a game device together with the game portable terminal, but also form a virtual system together with the head-mounted display device, the versatility of the combination handle is improved; and when the control handle is reattached to the handle case, the control handle and the handle case need to be re-calibrated after reattachment; when the user uses the combination to experience virtual functions, the positioning of the combination handle can be more accurate, thereby improving the fluency of man-machine interaction and further improving the user's sense of virtual experience.

[0050] In one embodiment of the present invention, FIG. 3 is a flowchart of a positioning method for a combined handle according to another embodiment of the present invention. As shown in FIG. 3, step S3 in the positioning method for the combined handle (calibrating the initial positioning information of the control handle and the initial positioning information of the handle case based on the positioning calibration value of the control handle and the positioning calibration value of the handle case, to generate spatial positioning information of the combined handle) specifically includes the following steps:

[0051] Step S31: calibrating initial positioning information of a control handle according to a positioning calibration value of the control handle to generate positioning information of the control handle; That is, the initial positioning information of the control handle is calibrated based on the positioning calibration value of the control handle, thereby improving the accuracy of the positioning information of the control handle.

[0052] Specifically, the positioning calibration value of the control handle can be calibrated and stored in advance. For example, before the combined handle is shipped, shipping calibration is performed on the control handle and the handle case, and the positioning calibration value of the control handle and the positioning calibration value of the handle case before and after shipping are stored.

[0053] Step S32: calibrating the initial positioning information of the handle case according to the positioning calibration value of the handle case to generate positioning information of the handle case; That is, the initial positioning information of the handle case is calibrated based on the positioning calibration value of the handle case, thereby improving the accuracy of the positioning information of the handle case.

[0054] Step S33: matching the positioning information of the control handle and the positioning information of the handle case to generate spatial positioning information of the combined handle.

[0055] The positioning information of the calibrated control handle and the positioning information of the handle case are matched to obtain the spatial positioning information of the combined handle, thereby improving the positioning accuracy of the combined handle.

[0056] In one embodiment of the present invention, FIG. 4 is a flowchart of a positioning method for a combination handle according to another embodiment of the present invention. As shown in FIG. 4, step S1 (obtaining initial positioning information of the control handle after the control handle is communicatively connected to the head-mounted display device, and obtaining initial positioning information of the handle case) specifically includes the following steps: Step S11: after the control handle is communicatively connected to the head-mounted display device, obtain positioning information of the head-mounted display device; Step S12: obtaining a first spot image of a first infrared sensor and IMU data of a control handle, and calculating initial positioning information of the control handle according to the first spot image, the IMU data of the control handle and the positioning information of the head-mounted display device; Specifically, a micro infrared lamp is installed in the first infrared sensor to emit infrared light, and when the micro infrared lamp is turned on, the camera can take a picture of it and capture the spot of the micro infrared lamp, so that the position of the spot can be determined as the position corresponding to the infrared emitter. Generally, each first infrared sensor is provided with a number of micro infrared lamps, and the multiple lamps of the first infrared sensor form a lamp ring. That is, the camera can take the first spot image of the first infrared sensor, so that the first spot image of the first infrared sensor can be obtained from the camera.

[0057] After the control handle that has been removed is reinstalled, the micro infrared lamp of the first infrared emitter is controlled to be turned on, and a first spot image including the spot of the micro infrared lamp of the first infrared emitter is obtained by photographing with a camera.

[0058] Specifically, an inertial sensor (abbreviated as Inertial Measurement Unit, IMU) is a device that measures the three-axis attitude angle (or angular velocity) and acceleration of an object. Generally, one IMU has three single-axis accelerometers and three single-axis gyros, the accelerometers detect independent three-axis acceleration signals in the medium coordinate system of the object, while the gyros detect angular velocity signals relative to the navigation coordinate system of the medium. Thus, the inertial sensor measures the angular velocity and acceleration of the object in three-dimensional space, and calculates the attitude of the object based on this, for example, the rotational degrees of freedom of the object, where the rotational degrees of freedom refer to the degrees of freedom related to the three positions of up, down, front, back, left, and right. IMU data is the result data of the inertial sensor detecting an object, that is, the angular velocity and acceleration data of an object in three-dimensional space detected by the inertial sensor. Thus, a first inertial sensor installed on the control handle can detect the IMU data of the control handle, and the attitude of the control handle, for example, the rotational degrees of freedom of the control handle can be calculated based on the IMU data of the control handle, where the rotational degrees of freedom refer to the degrees of freedom related to the three positions of up, down, front, back, left, and right.

[0059] In short, the six degree of freedom (6DOF) data of the control handle, ie, the 6DOF data of the control handle, can be determined based on the first spot image and the IMU data of the control handle detected by the first inertial sensor.

[0060] The initial positioning information of the control handle can be determined based on the 6DOF data of the control handle and the positioning information of the head-mounted display device.

[0061] Step S13: obtain a second spot image of the second infrared sensor and IMU data of the steering wheel case, and calculate initial positioning information of the steering wheel case according to the second spot image, the IMU data of the steering wheel case and the positioning information of the head-mounted display device.

[0062] Similarly, a micro infrared lamp is installed in the second infrared sensor to emit infrared light, and when the micro infrared lamp is turned on, the camera can take a picture of it and capture the spot of the micro infrared lamp, so that the position of the spot can be determined as the position corresponding to the infrared emitter. Generally, each second infrared sensor is provided with a number of micro infrared lamps, and the multiple lamps of the second infrared sensor form a lamp ring. That is, the camera can take the second spot image of the second infrared sensor, so that the second spot image of the second infrared sensor can be obtained from the camera.

[0063] After the control handle is removed and reinstalled, the micro infrared lamp of the second infrared emitter is controlled to be turned on, and a second spot image including the spot of the micro infrared lamp of the second infrared emitter is obtained by photographing with a camera, and the translation IMU data of the handle case can be determined from the second spot image; The second inertial sensor installed on the handle case can detect the IMU data of the handle case, and thus the IMU data of the handle case can be obtained from the second inertial sensor. The IMU data of the handle case can be used to calculate the attitude of the handle case, for example, to obtain the rotational degrees of freedom of the handle case, where the rotational degrees of freedom refer to the degrees of freedom related to the three positions of up and down, front and back, and left and right. The six degrees of freedom data of the handle case, i.e., the 6DOF data of the handle case, can be determined based on the second spot image and the IMU data of the handle case detected by the second inertial sensor.

[0064] The initial positioning information of the handle case can be determined based on the 6DOF data of the handle case and the positioning information of the head-mounted display device.

[0065] When a user uses the virtual reality device, the user wears the head-mounted display device on his / her head to view the game interface through the display on the head-mounted display device, and uses both hands to operate the buttons on the control handle to generate motion control commands. The motion control commands include forward, backward, moving left, moving right, rotating, jumping, etc. When the user's head or hand moves, the corresponding position information changes, and the 6-DOF data of the combination handle and the 6-DOF data of the head-mounted display device are determined and recorded, the motion trajectory of the combination handle is determined according to the change in the 6-DOF data of the combination handle, and the motion trajectory of the head-mounted display device is determined according to the change in the 6-DOF data of the head-mounted display device. Corresponding man-machine interaction is generated according to the motion trajectory, for example, a scene of switching the display of the game interface is realized according to the motion trajectory. Therefore, the motion trajectory is determined, and then the display page is controlled to be switched, and the corresponding game motion is generated according to the motion control command sent by the combination handle.

[0066] As described, the camera that captures the first spot image of the first infrared sensor on the control handle and the camera that captures the second spot image of the second infrared sensor on the handle case may be the same camera or different cameras. For example, the camera that captures the first infrared sensor and the second infrared sensor may be the same camera and installed on the head-mounted display device, and when the head-mounted display device includes a helmet and a mobile terminal, the camera may be installed on the helmet or the mobile terminal. Furthermore, for example, the camera that captures the first infrared sensor and the camera that captures the second infrared camera are not the same camera, and for example, the camera that captures the first infrared sensor may be one fixed camera installed in a space, and the camera that captures the second infrared camera may be one camera installed on the head-mounted display device.

[0067] In one embodiment of the present invention, FIG. 5 is a flowchart of a positioning method for a combination handle according to another embodiment of the present invention. As shown in FIG. 5, step S11 (obtaining positioning information of the head-mounted display device after the control handle is communicatively connected to the head-mounted display device) specifically includes the following steps: Step S110: acquiring a spatial image where the head-mounted display device is located; Specifically, the spatial image where the head-mounted display device is located may be captured by a camera, for example, a camera may be installed on the head-mounted display device, and the camera may capture the spatial image where the head-mounted display device is located. When the head-mounted display device includes a helmet and a mobile terminal, the camera may be installed on the helmet, or may be a rear camera of the mobile terminal.

[0068] Based on the spatial image in which the head mounted display device is positioned, position information in space, i.e., translational degree of freedom data, of the head mounted display device can be determined.

[0069] Step S111: Acquire IMU data of a head-mounted display device; Specifically, the IMU data of the head-mounted display device can be detected by a third inertial sensor installed on the head-mounted display device, that is, the third inertial sensor can detect the IMU data of the head-mounted display device.

[0070] Step S112: Calculate the spatial image where the head mounted display device is located and the IMU data of the head mounted display device to generate positioning information of the head mounted display device.

[0071] The six degrees of freedom data, i.e., 6DOF data, of the head mounted display device can be determined based on the spatial image of the head mounted display device and the IMU data of the head mounted display device, and the positioning information of the head mounted display device is the six degrees of freedom data of the head mounted display device.

[0072] In one embodiment of the present invention, FIG. 6 is a flowchart of a positioning method for a combination handle according to another embodiment of the present invention. As shown in FIG. 6, before step S1 (obtaining initial positioning information of the control handle after the control handle is communicatively connected to the head-mounted display device, and obtaining initial positioning information of the handle case), the positioning method for the combination handle further includes the following steps:

[0073] Step S10: before the combined handle is shipped, a shipping calibration is performed on the test positioning information of the control handle and the test positioning information of the handle case in the combined handle to generate a positioning calibration value of the control handle and a positioning calibration value of the handle case.

[0074] That is, the positioning calibration value of the control handle and the positioning calibration value of the handle case in step S2 are both calibration values ​​that have been calibrated before the combination handle is shipped.

[0075] Specifically, as shown in FIG. 7, step S10 (performing factory calibration on the test positioning information of the control handle and the test positioning information of the handle case in the combined handle, to generate positioning calibration values ​​of the control handle and positioning calibration values ​​of the handle case) specifically includes the following steps: Step S101, obtain a first test spot image of a first infrared sensor and a test IMU data of a control handle, and a second test spot image of a second infrared sensor and a test IMU data of a handle case; Before the combined handle is shipped, a camera captures a first test spot image of a first infrared sensor installed on the control handle and a second test spot image of a second infrared sensor installed on the handle case, a first inertial sensor installed on the control handle detects test IMU data of the control handle, and a second inertial sensor installed on the handle case detects test IMU data of the handle case.

[0076] Step S102, obtaining test positioning information of the control handle according to the first test spot image and the test IMU data of the control handle; The first test spot image can determine the test position information of the control handle, and the IMU data of the control handle can determine the test angle information of the control handle.

[0077] Step S103, obtain test positioning information of the steering wheel case according to the second test spot image and the test IMU data of the steering wheel case; The second test spot image can determine the test position information of the handle case, and the IMU data of the handle case can determine the test angle information of the handle case.

[0078] Step S104, comparing the test positioning information of the control handle with the standard positioning information of the control handle to generate a positioning calibration value of the control handle; Specifically, the positioning calibration value of the control handle includes a first position offset value and a first angle offset value, and the standard positioning information of the control handle includes a first standard position value and a first standard angle value.

[0079] Determine test position information of the control handle based on the first test spot image, and determine a first position offset value based on the test position information and a first standard position value in the standard positioning information.

[0080] Determine test angle information of the control handle based on the test IMU data of the control handle, and determine a first angle offset value based on the test angle information and a first standard angle value in the standard positioning information.

[0081] In step S105, the test positioning information of the handle case is compared with the standard positioning information of the handle case to generate a positioning calibration value of the handle case.

[0082] Specifically, the positioning calibration value of the steering wheel case includes a second position offset value and a second angle offset value, and the standard positioning information of the steering wheel case includes a second standard position value and a second standard angle value.

[0083] Determine test position information of the handle case based on the second test spot image, and determine a second position offset value based on the test position information and a second standard position value in the standard positioning information.

[0084] Determine test angle information of the handle case based on the test IMU data of the handle case, and determine a second angle offset value based on the test angle information and a second standard angle value in the standard positioning information.

[0085] After the control handle is attached to the hand-held part and before it is shipped, the control handle and the handle case need to be positioned to obtain a calibration value. During subsequent use, when the control handle of the combined handle is disassembled and then reassembled, the positions of the control handle and the handle case need to be calibrated based on the calibration value, and then the movement trajectory of the combined handle is tracked.

[0086] In one embodiment of the present invention, the combination handle further includes a first memory installed in the control handle and a second memory installed in the handle case, and as shown in FIG. 8, after step S105 and before step S2, the positioning method for the combination handle further includes: Step S106: storing the position calibration value of the control handle and the position calibration value of the handle case in the first memory and the second memory, respectively; That is, before the combined handle is shipped, the positioning calibration value of the control handle and the positioning calibration value of the handle case are determined, and then the positioning calibration value of the control handle is stored in a first memory installed in the control handle, and the positioning calibration value of the handle case is stored in a second memory installed in the handle case. The present invention stores the positioning calibration value of the control handle and the positioning calibration value of the handle case in different memories, which makes it easy to accurately extract the positioning calibration value and further reduces the probability of data loss.

[0087] When the positioning calibration values ​​are stored respectively, obtaining the positioning calibration value of the control handle and the positioning calibration value of the handle case is respectively retrieving from the first memory and the second memory. That is, step S2 (obtaining the positioning calibration value of the control handle and the positioning calibration value of the handle case) specifically includes:

[0088] In step S21, the positioning calibration value of the control handle and the positioning calibration value of the handle case are retrieved from the first memory and the second memory, respectively.

[0089] Exemplary Positioning Controller As a third aspect of the present invention, the present invention further provides a combination handle positioning controller, which may be integrated into a head-mounted display device or into a control handle of the combination handle. Figure 9 is an operation principle diagram of a combination handle positioning controller according to an embodiment of the present invention. As shown in Figure 9 and taken together with Figure 1, the combination handle positioning controller 1 includes a connection state determination unit 11, an information determination unit 12 and a spatial positioning unit 13. the connection state determination unit 11 is for determining a connection state between the control handle and the head-mounted display device; Specifically, the head-mounted display device includes a virtual helmet and a display attached to the virtual helmet. The communication connection between the control handle and the head-mounted display device refers to the communication connection between the control handle and a device that needs to transmit information such as control commands and images on the head-mounted display device, for example, the communication connection between the display on the head-mounted display device and the control handle, and further for example, the head-mounted display device has a sensor, in which the sensor is communicatively connected to the connection state determination unit 11, when the user wears the virtual helmet, the sensor transmits a sensing signal to the connection state determination unit 11, and the connection state determination unit 11 determines that the control handle performs a virtual function according to the sensing signal, that is, the control handle and the head-mounted display device form a virtual system, and the control handle realizes the communication connection with the sensor.

[0090] The information determination unit 12 is for obtaining initial positioning information of the control handle and obtaining initial positioning information of the handle case, and obtaining a positioning calibration value of the control handle and a positioning calibration value of the handle case; That is, the information determining unit 12 executes steps S1 and S2 in the combination handle positioning method described above.

[0091] Specifically, as shown in Fig. 9, the information determination unit 12 is communicatively connected to a first camera 333 for capturing a first spot image of the first infrared sensor 331 on the control handle 33, a second camera 365 for capturing a second spot image of the second infrared sensor 363 on the ring-shaped part 361 on the handle case, a first inertial sensor 332 installed on the control handle 33, and a second inertial sensor 364 on the ring-shaped part 361 installed on the handle case. The information determination unit 12 receives the first spot image of the first infrared sensor 331 transmitted from the first camera 333, receives the IMU data of the control handle 33 transmitted from the first inertial sensor 332, and determines the initial positioning information of the control handle 33 according to the first spot image and the IMU data of the control handle 33. The information determination unit 12 receives the second spot image of the second infrared sensor 363 transmitted from the second camera 365 and the IMU data of the handle case 36 transmitted from the second inertial sensor 364, and determines the initial positioning information of the handle case 36 based on the second spot image and the IMU data of the handle case 36.

[0092] The spatial positioning unit 13 is for calibrating the initial positioning information of the control handle and the initial positioning information of the handle case based on the positioning calibration value of the control handle and the positioning calibration value of the handle case, and generating spatial positioning information of the combined handle.

[0093] The spatial positioning unit 13 is for executing step S3 in the combination handle positioning method described above.

[0094] Specifically, the spatial positioning unit 13 is communicatively connected to the information determination unit 12, and the spatial positioning unit 13 receives the initial positioning data of the control handle and the initial positioning data of the handle case sent from the information determination unit 12, and calibrates the initial positioning information of the control handle and the initial positioning information of the handle case according to the positioning calibration value of the control handle and the positioning calibration value of the handle case, thereby generating spatial positioning information of the combined handle.

[0095] Specifically, as shown in FIG. 10, the positioning calibration value of the control handle and the positioning calibration value of the handle case are stored in the first memory 333 and the second memory 366, respectively, the first memory 333 is integrated into the control handle 33, the second memory 336 is integrated into the handle case 36, the spatial positioning unit 13 is communicatively connected to the first memory 333 and the second memory 336, respectively, and the spatial positioning unit 13 calls up the positioning calibration value of the control handle from the first memory 333 and the positioning calibration value of the handle case from the second memory 336.

[0096] The positioning controller of the combination handle of the present invention improves the versatility of the combination handle, since the combination handle can not only form a game device together with the game portable terminal, but also form a virtual system together with the head-mounted display device; in addition, when the control handle is reattached to the handle case, the control handle and the handle case need to be recalibrated after reattachment; when the user uses the combination to experience virtual functions, the positioning of the combination handle can be more accurate, thereby improving the fluency of man-machine interaction and further improving the user's sense of virtual experience.

[0097] In one embodiment of the present invention, FIG. 11 is an operating principle diagram of a positioning controller of a combination handle according to one embodiment of the present invention. As shown in FIG. 11, the spatial positioning unit 13 includes a first calibration module 131, a second calibration module 132 and a positioning module 133.

[0098] The first calibration module 131 is for calibrating the initial positioning information of the control handle according to the positioning calibration value of the control handle to generate the positioning information of the control handle; Specifically, the first calibration module 131 is communicatively connected to the information determination unit 12, the first calibration module 131 receives the positioning calibration value of the control handle transmitted from the information determination unit 12, the first calibration module 131 is communicatively connected to a first memory 333 in which the positioning calibration value of the control handle is stored, and can retrieve the positioning calibration value of the control handle from the first memory, and after receiving the initial positioning information of the control handle and the positioning calibration value of the control handle, the first calibration module 131 calibrates the initial positioning information of the control handle according to the positioning calibration value of the control handle to generate the positioning information of the control handle.

[0099] The second calibration module 132 is for calibrating the initial positioning information of the handle case according to the positioning calibration value of the handle case to generate the positioning information of the handle case; Specifically, the second calibration module 132 is communicatively connected to the information determination unit 12, and the second calibration module 132 receives the positioning calibration value of the handle case transmitted from the information determination unit 12. The first calibration module 131 is communicatively connected to a second memory 366 in which the positioning calibration value of the handle case is stored, and can retrieve the positioning calibration value of the handle case from the second memory. After receiving the initial positioning information of the handle case and the positioning calibration value of the handle case, the second calibration module 132 calibrates the initial positioning information of the handle case based on the positioning calibration value of the handle case to generate the positioning information of the handle case.

[0100] The positioning module 133 is for matching the positioning information of the control handle and the positioning information of the handle case to generate spatial positioning information of the combined handle.

[0101] The positioning module 133 is communicatively connected to the first calibration module 131 and the second calibration module 132, respectively, and receives the positioning information of the control handle transmitted from the first calibration module, and receives the positioning information of the handle case transmitted from the second calibration module, and matches the positioning information of the control handle and the positioning information of the handle case to generate spatial positioning information of the combined handle.

[0102] In one embodiment of the present invention, FIG. 12 and FIG. 13 are diagrams showing the operation principle of the positioning controller of the combination handle according to one embodiment of the present invention. As shown in FIG. 12 and FIG. 13, the spatial positioning unit 13 includes: The combined handle further includes a shipping calibration module 134 for performing shipping calibration on the test positioning information of the control handle and the test positioning information of the handle case in the combined handle before the combined handle is shipped, to generate a positioning calibration value of the control handle and a positioning calibration value of the handle case.

[0103] 12 , after the factory calibration module 134 completes the calibration of the combination handle, it can self-store the positioning calibration value of the control handle and the positioning calibration value of the handle case, that is, the positioning calibration value of the control handle and the positioning calibration value of the handle case are both stored in the factory calibration module 134. The factory calibration module 134 is communicatively connected to the first calibration module 131 and the second calibration module 132, and both the first calibration module 131 and the second calibration module 132 can call up the positioning calibration value of the control handle and the positioning calibration value of the handle case from the factory calibration module 134.

[0104] In addition, as shown in FIG. 13, after the factory calibration module 134 completes the calibration of the combination handle, it can store the positioning calibration value of the control handle in the first memory 333 integrated in the control handle, and store the positioning calibration value of the handle case in the second memory 366 integrated in the handle case. At this time, the factory calibration module 134 is communicatively connected to the first memory 333 and the second memory 366, and stores the positioning calibration value of the control handle in the first memory 333 integrated in the control handle, and stores the positioning calibration value of the handle case in the second memory 366 integrated in the handle case. The first calibration module 131 is communicatively connected to the first memory 333, and the first calibration module 131 can call the positioning calibration value of the control handle from the first memory 333. The second calibration module 132 is communicatively connected to the second memory 366, and the second calibration module 132 can call the positioning calibration value of the handle case from the second memory 366.

[0105] In one embodiment of the present invention, Figure 14 is an operating principle diagram of the positioning controller of the combination handle according to one embodiment of the present invention. As shown in Figure 14, the information determination unit 12 specifically includes a head-mounted display information determination module 121, a first positioning information determination module 122 and a second positioning information determination module 123.

[0106] The head-mounted display information determination module 121 is for acquiring positioning information of the head-mounted display device, The first positioning information determination module 122 is for obtaining a first spot image of the first infrared sensor and the IMU data of the control handle, and calculating initial positioning information of the control handle according to the first spot image, the IMU data of the control handle and the positioning information of the head-mounted display device; Specifically, the first positioning information determination module 122 is communicatively connected to the head-mounted display information determination module 121, the first calibration module 131, the first inertial sensor 332 and the first camera 333, the first positioning information determination module 122 receives the IMU data of the control handle transmitted from the first inertial sensor 332, and receives the first spot image of the first infrared sensor 332 captured by the first camera 333, and calculates the IMU data of the control handle, the first spot image and the positioning information of the head-mounted display device to determine initial positioning information of the control handle, and after the first positioning information determination module 122 determines the initial positioning information of the control handle, it sends the initial positioning information of the control handle to the first calibration module 131.

[0107] The second positioning information determination module 123 is for acquiring the second spot image of the second infrared sensor 364 and the IMU data of the handle case, and calculating the initial positioning information of the handle case based on the second spot image, the IMU data of the handle case, and the positioning information of the head-mounted display device.

[0108] Specifically, the second positioning information determination module 123 is communicatively connected to the head-mounted display information determination module 121, the second calibration module 132, the second inertial sensor 364 and the second camera 365, respectively. The second positioning information determination module 123 receives the IMU data of the handle case transmitted from the second inertial sensor 364, and receives the second spot image of the second infrared sensor 364 captured by the second camera 365, and calculates the IMU data of the handle case, the first spot image and the positioning information of the head-mounted display device to determine initial positioning information of the handle case. After the second positioning information determination module 123 determines the initial positioning information of the handle case, it sends the initial positioning information of the handle case to the second calibration module 132.

[0109] In one embodiment of the present invention, FIG. 15 is an operating principle diagram of a positioning controller of a combination handle according to one embodiment of the present invention. As shown in FIG. 15, the head-mounted display information determination module 121 specifically includes an image acquisition module 1211, a degree of freedom data acquisition module 1212 and a calculation module 1213.

[0110] The image acquisition module 1211 is for acquiring a spatial image in which the head-mounted display device is located; Specifically, when a third camera 40 is attached to the head-mounted display device, the third camera 40 can capture a spatial image of the space in which the head-mounted display device is located, and at this time, the image acquisition module 1211 is communicatively connected to the third camera 40 and receives the spatial image in which the head-mounted display device is located transmitted from the third camera 40.

[0111] The degree of freedom data acquisition module 1212 is for acquiring IMU data of the head mounted display device; Specifically, the IMU data of the head-mounted display device can be detected and acquired using a third inertial sensor 41 attached to the head-mounted display device, and at this time, the degree of freedom data acquisition module 1212 is communicatively connected to the third inertial sensor 41 attached to the head-mounted display device, and receives the IMU data of the head-mounted display device detected by the third inertial sensor 41.

[0112] The calculation module 1213 is for calculating the spatial image where the head mounted display device is located and the IMU data of the head mounted display device, and generating positioning information of the head mounted display device.

[0113] The calculation module 1213 is communicatively connected to the image acquisition module 1211, the degree of freedom data acquisition module 1212, the first positioning information determination module 122, and the second positioning information determination module 123. The calculation module 1213 receives the spatial image where the head mounted display device is located transmitted from the image acquisition module 1211, and receives the IMU data of the head mounted display device transmitted from the degree of freedom data acquisition module 1212, and calculates the spatial image where the head mounted display device is located and the IMU data of the head mounted display device to determine positioning information of the head mounted display device, and transmits the positioning information of the head mounted display device to the first positioning information determination module 122 and the second positioning information determination module 123.

[0114] Example Virtual System As a fourth aspect of the present invention, the present invention further provides a virtual system, comprising: a combination handle as described above; a positioning controller of the combination handle as described above; a head-mounted display device; and a photographing device mounted on the head-mounted display device for photographing a first spot image of the first infrared sensor, a second spot image of the second infrared sensor, and an image of a space in which the head-mounted display device is located, wherein the positioning controller of the combination handle is mounted on the head-mounted display device, and the photographing device and the control handle are both communicatively connected to the positioning controller of the combination handle.

[0115] Specifically, the photographing device may be equipped with the first camera, second camera and third camera described above, where the first camera is for photographing a first spot image of the first infrared sensor, the second camera is for photographing a second spot image of the second infrared sensor, and the third camera is for photographing an image of the space in which the head-mounted display device is located.

[0116] The operating principle of the virtual system is described above, and a detailed description thereof will be omitted here.

[0117] Optionally, the virtual system further comprises a third inertial sensor mounted on the head mounted display device for measuring IMU data of the head mounted display device.

[0118] Optionally, the virtual system comprises a virtual reality system or an augmented reality system.

[0119] Exemplary Electronic Devices An electronic device according to an embodiment of the present invention will now be described with reference to Fig. 16. Fig. 16 is a structural schematic diagram of an electronic device according to an embodiment of the present invention.

[0120] As shown in FIG. 16, an electronic device 600 includes one or more processors 601 and a memory 602 .

[0121] The processor 601 may be a central processing unit (CPU) or other type of processing unit having data processing and / or information execution capabilities and may control the other components of the electronic device 600 to perform desired functions.

[0122] The memory 601 may include one or more computer program products, which may include various types of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache). The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program information may be stored in the computer-readable storage medium, and the processor 601 may execute the program information to realize the combination handle positioning method or other desired functions of each embodiment of the present invention described above.

[0123] In one example, electronic device 600 may further include input devices 603 and output devices 604, these components being connected together by a bus system and / or other type of connection mechanism (not shown).

[0124] The input device 603 may include, for example, a keyboard, a mouse, and the like.

[0125] The output device 604 can output various information to the outside, and may include, for example, a display, a communication network, and a remote output device connected thereto.

[0126] Of course, for the sake of simplicity, Fig. 16 only shows some of the components related to the present invention in the electronic device 600, and omits components such as buses, input / output interfaces, etc. In addition, the electronic device 600 may further include any other suitable components according to specific application situations.

[0127] Besides the above methods and apparatus, an embodiment of the present invention may also be a computer program product including computer program information which, when executed by a processor, causes the processor to perform steps in the combination handle positioning methods according to various embodiments of the present invention as described herein.

[0128] The computer program product may produce program code for carrying out operations of embodiments of the present invention in any combination of one or more program design languages, including object oriented program design languages ​​such as Java, C++, etc., and further including conventional procedural program design languages ​​such as "C" or similar program design languages. The program code may run entirely on the user's computing device, partially on the user's device, as a separate software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or a server.

[0129] Furthermore, an embodiment of the present invention may further be a computer readable storage medium having computer program information stored thereon, which when executed by a processor causes the processor to perform steps in the combination handle positioning methods according to various embodiments of the present invention herein.

[0130] The computer readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include an electrical connection having one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0131] Although the basic principle of the present invention has been described above with reference to specific embodiments, it should be noted that the advantages, advantages, effects, etc. mentioned in the present invention are illustrative and not limiting, and these advantages, advantages, effects, etc. should not be considered as something that must be possessed by each embodiment of the present invention. In addition, the specific details disclosed above are merely illustrative and easy to understand, and are not limiting, and the details should not be construed as limiting the present invention so that it must be realized using the specific details.

[0132] Block diagrams of devices, apparatus, facilities, and systems mentioned in the present invention are merely illustrative examples, and are not intended to require or suggest that they must be connected, laid out, or arranged in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatus, facilities, and systems can be connected, laid out, or arranged in any manner. For example, words such as "include", "includes", and "having" are open class vocabulary terms and can be used interchangeably to mean "including, but not limited to, XXX". Unless otherwise clearly indicated by the context, the terms "or" and "and" used herein refer to, and can be used interchangeably with, the term "and / or". The term "for example" used herein refers to, and can be used interchangeably with, the collocation "including, but not limited to, XXX".

[0133] The above description is merely a preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent replacements, etc. made within the spirit and principle of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for determining a position of a combination handle, comprising the steps of: the combination handle comprises a control handle and a handle case; After the control handle is communicatively connected to the head-mounted display device, obtaining initial positioning information of the control handle and obtaining initial positioning information of the handle case; Obtaining a positioning calibration value of the control handle and a positioning calibration value of the handle case; calibrating initial positioning information of the control handle and the initial positioning information of the handle case based on the positioning calibration value of the control handle and the positioning calibration value of the handle case, thereby generating spatial positioning information of the combination handle.

2. Calibrating the initial positioning information of the control handle and the initial positioning information of the handle case based on the positioning calibration value of the control handle and the positioning calibration value of the handle case to generate spatial positioning information of the combined handle; calibrating initial positioning information of the control handle based on the positioning calibration value of the control handle to generate positioning information of the control handle; calibrating initial positioning information of the handle case based on the positioning calibration value of the handle case to generate positioning information of the handle case; 2. The method of claim 1, further comprising: matching the positioning information of the control handle and the positioning information of the handle case to generate spatial positioning information of the combination handle.

3. The handle case has an annular portion and a gripping portion, and a recess is provided at the center of the gripping portion for receiving and fixing the control handle. The combination handle further includes a first infrared sensor installed on the control handle and a second infrared sensor installed on the annular portion. After the control handle is communicatively connected to the head-mounted display device, obtaining initial positioning information of the control handle and obtaining initial positioning information of the handle case are After the control handle is communicatively connected to a head-mounted display device, acquiring positioning information of the head-mounted display device; Obtaining a first spot image of the first infrared sensor and IMU data of the control handle, and calculating initial positioning information of the control handle according to the first spot image, the IMU data of the control handle and the positioning information of the head-mounted display device; 2. The positioning method for a combination handle as described in claim 1, further comprising: acquiring a second spot image of the second infrared sensor and IMU data of the handle case; and calculating initial positioning information of the handle case based on the second spot image, the IMU data of the handle case, and the positioning information of the head-mounted display device.

4. Acquiring positioning information of the head mounted display device includes: acquiring an image of a space in which the head mounted display device is located; acquiring IMU data of the head mounted display device; 4. The method for determining the position of a combination handle according to claim 3, further comprising: calculating a spatial image in which the head-mounted display device is located and IMU data of the head-mounted display device to generate positioning information of the head-mounted display device.

5. After the control handle is communicatively connected to the head-mounted display device, obtaining initial positioning information of the control handle, and before obtaining initial positioning information of the handle case, the combination handle positioning method further includes: The positioning method for a combination handle as described in claim 3, further comprising: before the combination handle is shipped, performing shipping calibration on the test positioning information of the control handle and the test positioning information of the handle case in the combination handle to generate a positioning calibration value of the control handle and a positioning calibration value of the handle case.

6. performing factory calibration on the test positioning information of the control handle and the test positioning information of the handle case in the combined handle to generate a positioning calibration value of the control handle and a positioning calibration value of the handle case; Obtaining a first test spot image of the first infrared sensor and test IMU data of the control handle, and a second test spot image of the second infrared sensor and test IMU data of the handle case; Obtaining test positioning information of the control handle according to the first test spot image and test IMU data of the control handle; Obtaining test positioning information of the handle case according to the second test spot image and test IMU data of the handle case; Comparing the test positioning information of the control handle with the standard positioning information of the control handle to generate a positioning calibration value of the control handle; 6. The method for positioning a combination handle as claimed in claim 5, further comprising: comparing the test positioning information of the handle case with the standard positioning information of the handle case to generate a positioning calibration value of the handle case.

7. The combination handle further comprises a first memory mounted on the control handle and a second memory mounted on the handle case; After generating the positioning calibration value of the control handle and the positioning calibration value of the handle case, the combined handle positioning method further includes: storing a positioning calibration value of the control handle and a positioning calibration value of the handle case in the first memory and the second memory, respectively; Obtaining the positioning calibration value of the control handle and the positioning calibration value of the handle case includes:

6. The method for determining the position of a combination handle as claimed in claim 5, further comprising: retrieving the position calibration value of the control handle and the position calibration value of the handle case from the first memory and the second memory, respectively.

8. The positioning calibration value of the control handle includes a position offset value of the control handle and an angle offset value of the control handle; 2. The method for determining the position of a combination handle as claimed in claim 1, wherein the positioning calibration value of the handle case includes a position offset value of the handle case and an angle offset value of the handle case.

9. A combination handle positioning controller comprising a control handle and a handle case, a connection state determination unit for determining a connection state between the control handle and a head-mounted display device; An information determining unit is for obtaining initial positioning information of the control handle, obtaining initial positioning information of the handle case, and obtaining a positioning calibration value of the control handle and a positioning calibration value of the handle case; a spatial positioning unit for calibrating initial positioning information of the control handle and initial positioning information of the handle case based on the positioning calibration value of the control handle and the positioning calibration value of the handle case, thereby generating spatial positioning information of the combination handle.

10. The spatial positioning unit includes: a first calibration module for calibrating initial positioning information of the control handle according to a positioning calibration value of the control handle to generate positioning information of the control handle; a second calibration module for calibrating the initial positioning information of the handle case according to the positioning calibration value of the handle case to generate positioning information of the handle case; The positioning controller of claim 9, further comprising: a positioning module for integrating the positioning information of the control handle and the positioning information of the handle case to generate spatial positioning information of the combination handle.

11. The information determining unit comprises: a head-mounted display information determination module for acquiring positioning information of the head-mounted display device; a first positioning information determination module for obtaining a first spot image of a first infrared sensor and an IMU data of the control handle, and calculating initial positioning information of the control handle according to the first spot image, the IMU data of the control handle and the positioning information of the head-mounted display device; The positioning controller of the combination handle as described in claim 9, further comprising: a second positioning information determination module for acquiring a second spot image of a second infrared sensor and an IMU data of the handle case, and calculating initial positioning information of the handle case according to the second spot image, the IMU data of the handle case and the positioning information of the head-mounted display device.

12. The head mounted display information determination module includes: an image capture module for capturing an image of a space in which the head mounted display device is located; a degree of freedom data acquisition module for acquiring IMU data of the head mounted display device; and a calculation module for calculating a spatial image of the head mounted display device and IMU data of the head mounted display device to generate positioning information of the head mounted display device.

13. The spatial positioning unit further comprises: The positioning controller of the combination handle as described in claim 10, further comprising a shipping calibration module for performing shipping calibration on the test positioning information of the control handle and the test positioning information of the handle case in the combination handle before the combination handle is shipped, to generate a positioning calibration value of the control handle and a positioning calibration value of the handle case.

14. 1. A virtual system comprising: a combination handle including a control handle, a first infrared sensor installed on the control handle, a handle case having an annular portion and a grip portion, the grip portion having a recessed portion at a central location thereof for receiving and fixing the control handle, and a second infrared sensor installed on the annular portion; A positioning controller for a combination handle according to any one of claims 9 to 13; A head mounted display device; an imaging device that is installed on the head-mounted display device and captures a first spot image of the first infrared sensor, a second spot image of the second infrared sensor, and an image of a space in which the head-mounted display device is located; A virtual system, characterized in that a positioning controller of the combination handle is installed on the head-mounted display device, and the shooting device and the control handle are both communicatively connected to the positioning controller of the combination handle.

15. The combination handle is a first inertial sensor mounted on the control handle for measuring IMU data of the control handle; The virtual system of claim 14, further comprising: a second inertial sensor mounted on the handle case for measuring IMU data of the handle case.

16. The combination handle is a first memory disposed on the control handle for storing a positioning calibration value of the control handle; The virtual system according to claim 14, further comprising: a second memory installed in the handle case for storing a positioning calibration value of the handle case.

17. The combination handle is The virtual system of claim 16, further comprising a main control chip installed in the control handle, the main control chip being respectively connected to the first memory and the second memory, the main control chip being for retrieving the positioning calibration value of the control handle from the first memory, and retrieving the positioning calibration value of the handle case from the second memory.

18. 15. The virtual system of claim 14, further comprising a third inertial sensor mounted on the head-mounted display device for measuring IMU data of the head-mounted display device.

19. 15. The virtual system of claim 14, wherein the virtual system comprises a virtual reality system or an augmented reality system.

Citation Information

Patent Citations

  • Program, information processing device, information processing system, information processing method, and head mount display

    JP2020091774A

  • Hand-held controller tracked by LED mounted under a concaved dome

    US10409392B1

  • Interactive exercise and training system and method

    US10705597B1

  • Seat cover

    WO2020000177A1