VR handle LED light spot matching method and hardware based on maximum likelihood estimation

Abstract

The invention relates to a VR handle LED light spot matching method and hardware based on maximum likelihood estimation. A VR handle is initialized, an inertial sensor and an LED for light spot matching are configured, and a VR wearable device is configured corresponding to the VR handle; obtaining VR handle information at the previous moment based on an inertial sensor, fusing the real value of the VR handle at the previous moment to obtain a predicted value of the VR handle at the current moment, and projecting the predicted value to a camera group of the VR wearable equipment to obtain a predicted light spot position at the current moment; performing Gaussian distribution probability calculation based on the predicted light spot position and the real light spot position at the current moment; calculating the posture of the LED on the VR handle according to the maximum likelihood estimation and matching; repeating until the Gaussian distribution probability reaches a threshold value; outputting and tracking a light spot matching result; hardware is implemented based on the method. According to the method, the light spots on the VR handle can be quickly and accurately matched, the stability of the handle when a user uses the VR handle is ensured, the VR experience feeling is strong, and the smoothness is relatively good.

Description

Technical Field

[0001] The present invention relates to the technical field of calculation, extrapolation or counting, and in particular to a VR handle LED light spot matching method and hardware based on maximum likelihood estimation. Background Art

[0002] A VR all-in-one is a VR device with independent computing, input and output functions. The current mainstream VR all-in-one devices generally include VR all-in-one glasses (helmets) and VR all-in-one dual handles.

[0003] Generally speaking, multiple cameras are set on the all-in-one glasses, and several LED light spots are distributed on the handle of the VR all-in-one machine. The camera observes these light spots and matches them with the design values ​​of the LED, which is the light spot matching algorithm. After matching, the position of the handle can be calculated by the PNP algorithm, and then the handle can be continuously tracked using algorithms such as visual inertial fusion, so that the camera can locate the glasses and handle of the VR all-in-one machine.

[0004] However, among the existing controller tracking algorithms, the mainstream matching algorithm is the brute force matching algorithm, which takes a long time and may cause mismatches, especially when the controller moves quickly. In some typical scenarios, such as when the controller needs to be swung quickly and vigorously in the game rhythm lightsaber, the brute force light spot matching algorithm often causes the controller to freeze, resulting in a poor gaming experience.

[0005] Therefore, it is necessary to provide a faster and more accurate controller light point matching algorithm to accelerate the matching of VR controller light points. Summary of the invention

[0006] The present invention solves the problems existing in the prior art and provides a VR handle LED light spot matching method and hardware based on maximum likelihood estimation.

[0007] The technical solution adopted by the present invention is a VR handle LED light spot matching method based on maximum likelihood estimation, and the method comprises the following steps:

[0008] S1 initializes the VR handle, configures the inertial sensor and the LED for light spot matching, and configures the VR wearable device corresponding to the VR handle;

[0009] S2 obtains the VR handle information of the previous moment based on the inertial sensor, integrates the real value of the VR handle of the previous moment, and obtains the predicted value of the VR handle at the current moment;

[0010] S3 projects the predicted value onto the camera group of the VR wearable device to obtain the predicted light spot position at the current moment;

[0011] S4 performs Gaussian distribution probability calculation based on the predicted light spot position and the actual light spot position at the current moment;

[0012] S5 calculates and matches the posture of the LED on the VR handle based on maximum likelihood estimation;

[0013] S6 repeats S5 until the Gaussian distribution probability reaches a preset value;

[0014] S7 outputs the light spot matching results and tracks them;

[0015] If the inertial sensor feeds back data, S8 repeats S2, otherwise it ends.

[0016] Preferably, in S1, the LED is arranged on the VR handle with its circumscribed spherical surface facing the center of the sphere, and the circumscribed spherical surface is the surface where the smallest circumscribed sphere of the VR handle is located.

[0017] Preferably, the VR handle information is the integration of data collected by the inertial sensor.

[0018] Preferably, the inertial sensor includes a gyroscope and an accelerometer; and the VR handle information includes an integral of position, velocity and posture.

[0019] Preferably, the position integral satisfies,

[0020]

[0021] Speed ​​integral meets,

[0022]

[0023] The posture score is satisfied.

[0024]

[0025] in, is the angular velocity data, b g is the gyroscope bias, η gd is the observation noise of the gyroscope, is the acceleration data, b a is the zero bias of the accelerometer, η a is the observation noise of the accelerometer, and g is the magnitude of gravity.

[0026] Preferably, in S3, the VR wearable device is a VR helmet, and the camera group includes four cameras arranged in a circle on the VR helmet.

[0027] Preferably, the predicted light spot position obtained on each camera satisfies,

[0028]

[0029] Among them, Π is the projection function, X ci =[X ci Y ci Z ci ], i corresponds to the light spot number [f u f v ] T is the focal length of the camera, [c u c v ] T The optical center of the camera.

[0030] Preferably, Represents the coordinates of each camera image, and the two-dimensional Gaussian distribution probability calculation satisfies,

[0031]

[0032] Among them, μ 4 and μ 5 For all The mean value, σ 4 and σ 5 is the corresponding standard deviation, and ρ is the relevant parameter of image pixels x and y.

[0033] A computer-readable storage medium stores a VR handle LED light spot matching program based on maximum likelihood estimation, and when the program is executed by a processor, the VR handle LED light spot matching method based on maximum likelihood estimation is implemented.

[0034] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned VR handle LED light spot matching method based on maximum likelihood estimation is implemented.

[0035] The present invention relates to a VR handle LED light spot matching method and hardware based on maximum likelihood estimation, which comprises the following steps: initializing a VR handle, configuring an inertial sensor and an LED for light spot matching, and configuring a VR wearable device corresponding to the VR handle; obtaining VR handle information at a previous moment based on the inertial sensor, fusing the real value of the VR handle at the previous moment, and obtaining a predicted value of the VR handle at the current moment; projecting the predicted value to a camera group of the VR wearable device to obtain a predicted light spot position at the current moment; performing Gaussian distribution probability calculation based on the predicted light spot position and the real light spot position at the current moment; calculating and matching the posture of the LED on the VR handle according to maximum likelihood estimation; repeating until the Gaussian distribution probability reaches a threshold; outputting the light spot matching result and tracking; and realizing hardware based on the method.

[0036] The beneficial effect of the present invention is that the light spots on the VR handle can be matched quickly and accurately, ensuring the stability of the handle when the user uses the VR handle, and the VR experience is strong and the fluency is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below in conjunction with embodiments, but the protection scope of the present invention is not limited thereto.

[0039] The present invention relates to a VR controller LED light spot matching method based on maximum likelihood estimation, which calculates the position of the controller in space according to the 2D point where the image light spot is located and the 3D point in the corresponding space. The following is an explanation of the specific method.

[0040] S1 initializes the VR handle, configures the inertial sensor and the LED for light spot matching, and configures the VR wearable device corresponding to the VR handle;

[0041] In S1, the LED is arranged on the VR handle with the circumscribed spherical surface of the VR handle facing the center of the sphere, and the circumscribed spherical surface is the surface where the smallest circumscribed sphere of the VR handle is located.

[0042] In the present invention, the regular arrangement of LEDs is beneficial to the deviation correction during the light spot matching process.

[0043] The VR handle information is the integration of data collected by the inertial sensor.

[0044] The inertial sensor includes a gyroscope and an accelerometer; the VR handle information includes the integral of position, velocity and attitude.

[0045] In the present invention, the acceleration and angular velocity data collected by the IMU installed on the handle are first integrated to obtain the position, velocity, and attitude integrals of the VR handle:

[0046] The position integral satisfies,

[0047]

[0048] Speed ​​integral meets,

[0049]

[0050] The posture score is satisfied.

[0051]

[0052] in, is the angular velocity data, b g is the gyroscope bias, ηgd is the observation noise of the gyroscope, is the acceleration data, b a is the zero bias of the accelerometer, η a is the observation noise of the accelerometer, and g is the magnitude of gravity.

[0053] S2 obtains the VR handle information of the previous moment based on the inertial sensor, integrates the real value of the VR handle of the previous moment, and obtains the predicted value of the VR handle at the current moment;

[0054] In the present invention, the fused content here is the result of the last observation and calculation of the VR handle and the IMU integrated data. Since the integrated data represents the movement tendency of the VR handle, it is fused with the result of the last observation and calculation to obtain the current predicted value.

[0055] S3 projects the predicted value onto the camera group of the VR wearable device to obtain the predicted light spot position at the current moment;

[0056] In S3, the VR wearable device is a VR helmet, and the camera group includes four cameras arranged around the VR helmet.

[0057] The predicted light spot position obtained on each camera satisfies,

[0058]

[0059] Among them, Π is the projection function, X ci =[X ci Y ci Z ci ], i corresponds to the light spot number, [f u f v ] T is the focal length of the camera, [c u c v ] T The optical center of the camera;

[0060] Based on this, the four cameras will obtain four corresponding sets of results. If four LEDs are set on the VR handle, there will be 16 predicted light spot positions.

[0061] S4 performs Gaussian distribution probability calculation based on the predicted light spot position and the actual light spot position at the current moment;

[0062] by Represents the coordinates of each camera image, and the two-dimensional Gaussian distribution probability calculation satisfies,

[0063]

[0064] Among them, μ 4 and μ 5 For all The mean value, σ 4 and σ 5 is the corresponding standard deviation, and ρ is the relevant parameter of image pixels x and y.

[0065] In general, ρ∈(0,1).

[0066] S5 calculates and matches the posture of the LED on the VR handle based on maximum likelihood estimation;

[0067] According to the maximum likelihood estimation calculation, the mean and variance can be estimated based on the sample data to obtain the estimated values ​​of the parameters, and finally the most likely position, speed and posture of the LED can be obtained and matched;

[0068] S6 repeats S5, iteratively optimizing the maximum likelihood estimation function until the Gaussian distribution probability reaches a preset value;

[0069] S7 outputs the light spot matching results and tracks them;

[0070] This completes single tracking.

[0071] If the inertial sensor feeds back data in S8, it means that the need for light spot matching still exists, so S2 is repeated, otherwise it ends; of course, in the actual operation process, the light spot matching and tracking can be terminated directly by stopping the machine.

[0072] The present invention also relates to a computer-readable storage medium in its application, on which is stored a VR handle LED light spot matching program based on maximum likelihood estimation. When the program is executed by a processor, the above-mentioned VR handle LED light spot matching method based on maximum likelihood estimation is implemented.

[0073] The present invention also relates to a computer device in its application, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned VR handle LED light spot matching method based on maximum likelihood estimation is implemented.

[0074] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0078] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0079] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A VR handle LED light spot matching method based on maximum likelihood estimation, characterized by: The method comprises the following steps: S1 initializes the VR handle, configures the inertial sensor and the LED for light spot matching, and configures the VR wearable device corresponding to the VR handle; S2 obtains the VR handle information of the previous moment based on the inertial sensor, integrates the real value of the VR handle of the previous moment, and obtains the predicted value of the VR handle at the current moment; S3 projects the predicted value onto the camera group of the VR wearable device to obtain the predicted light spot position at the current moment; S4 performs Gaussian distribution probability calculation based on the predicted light spot position and the actual light spot position at the current moment; S5 calculates and matches the posture of the LED on the VR handle based on maximum likelihood estimation; S6 repeats S5 until the Gaussian distribution probability reaches a preset value; S7 outputs the light spot matching results and tracks them; If the inertial sensor feeds back data, S8 repeats S2, otherwise it ends.

2. The VR handle LED light spot matching method based on maximum likelihood estimation according to claim 1, characterized in that: In S1, the LED is arranged on the VR handle with the circumscribed spherical surface of the VR handle facing the center of the sphere, and the circumscribed spherical surface is the surface where the smallest circumscribed sphere of the VR handle is located.

3. The VR handle LED light spot matching method based on maximum likelihood estimation according to claim 1, characterized in that: The VR handle information is the integration of data collected by the inertial sensor.

4. The VR handle LED light spot matching method based on maximum likelihood estimation according to claim 3, characterized in that: Inertial sensors include gyroscopes and accelerometers; The VR handle information includes the integral of position, velocity and posture.

5. The VR handle LED light spot matching method based on maximum likelihood estimation according to claim 3, characterized in that: The position integral satisfies, Speed ​​integral meets, The posture score is satisfied. in, is the angular velocity data, b g is the gyroscope bias, η gd is the observation noise of the gyroscope, is the acceleration data, b a is the zero bias of the accelerometer, η a is the observation noise of the accelerometer, and g is the magnitude of gravity.

6. The VR handle LED light spot matching method based on maximum likelihood estimation according to claim 1, characterized in that: In S3, the VR wearable device is a VR helmet, and the camera group includes four cameras arranged around the VR helmet.

7. The VR handle LED light spot matching method based on maximum likelihood estimation according to claim 6, characterized in that: The predicted light spot position obtained on each camera satisfies, Among them, Π is the projection function, X ci =[X ci Y ci Z ci ], i corresponds to the light spot number, [f u f v ] T is the focal length of the camera, [c u c v ] T The optical center of the camera.

8. The VR handle LED light spot matching method based on maximum likelihood estimation according to claim 7, characterized in that: by Represents the coordinates of each camera image, and the two-dimensional Gaussian distribution probability calculation satisfies, Among them, μ6 and μ7 are corresponding to all The mean value, σ 6 and σ 7 is the corresponding standard deviation, and ρ is the relevant parameter of image pixels x and y.

9. A computer-readable storage medium, characterized in that: A VR handle LED light spot matching program based on maximum likelihood estimation is stored thereon, and when the program is executed by the processor, the VR handle LED light spot matching method based on maximum likelihood estimation described in one of claims 1 to 8 is implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the VR handle LED light spot matching method based on maximum likelihood estimation described in any one of claims 1 to 8 is implemented.

Images