Light spot identification method of VR handle and VR equipment

Abstract

The invention provides a light spot identification method of a VR handle and VR equipment. The light spot identification accuracy of the handle is improved. Comprising the following steps: performing handle light spot identification on a first image at any moment by using a light spot identification algorithm to obtain actual position coordinates of each handle light spot to be authenticated in the first image; the first image is a short-exposure image which contains the VR handle and an LED lamp in the VR handle is in an on state; based on the actual position coordinates of the to-be-authenticated handle light spot, the to-be-authenticated handle light spot is projected in a second image, the projection position coordinates of the to-be-authenticated handle light spot are obtained, and the second image is a long-exposure image which comprises a VR handle and in which an LED lamp in the VR handle is in an off state; obtaining a first target area and a second target area of the to-be-authenticated handle light spot in the second image according to the projection position coordinates of the to-be-authenticated handle light spot, and obtaining the confidence coefficient of the light spot; and identifying each to-be-authenticated handle light spot based on the confidence of each to-be-authenticated handle light spot to obtain a target handle light spot.

Description

Background Art

[0002] VR (Virtual Reality) helmet display applications are widely used in various industries. For example, in education and training, fire drills, virtual driving, and real estate projects. While bringing users an immersive visual feast, natural and smooth interaction is the most important manifestation of user satisfaction in user experience, so handle control is an indispensable interaction method.

[0003] VR devices have two types of positioning and tracking for handles: 3DOF (3 Degrees of Freedom) and 6DOF (6 Degrees of Freedom). Among them, 6DOF is the current mainstream control interaction method, so the application scenario of this solution is mainly in 6DOF. The positioning of the handle posture is mainly determined based on the handle light spot captured by the VR device. Therefore, the accuracy of the handle light spot recognition is crucial for the handle positioning.

[0004] At present, because the handle itself is equipped with infrared LEDs (light-emitting diodes), but there are usually a lot of lights on the ceiling in scenes such as exhibitions, so for handles positioned with infrared light rings, the light arrangements of both sides are likely to be similar, and it is easy to mistake the ambient light for the light in the handle light ring. Figure 1 As shown, Figure 1 The light in the box is the light in the handle light ring, and the other lights are ambient lights. Therefore, the ambient lights may be mistakenly identified as handle lights, resulting in a low accuracy rate in identifying the handle's light spot. Summary of the invention

[0005] The present application provides a light spot recognition method for a VR handle and a VR device, which are used to improve the accuracy of recognizing the light spot of the handle.

[0006] In a first aspect, an embodiment of the present application provides a light spot recognition method for a VR controller, the method comprising:

[0007] At any moment, the first image corresponding to the moment is subjected to handle light spot recognition using a light spot recognition algorithm to obtain the actual position coordinates of each handle light spot to be authenticated in the first image; wherein the first image is a short exposure image containing a VR handle, and the LED light in the VR handle is turned on; and,

[0008] For any handle light spot to be authenticated, based on the actual position coordinates of the handle light spot to be authenticated, project the handle light spot to be authenticated in a second image corresponding to the moment to obtain the projection position coordinates of the handle light spot to be authenticated in the second image, wherein the second image is a long exposure image including the VR handle and the LED light in the VR handle is in the off state; and,

[0009] According to the projection position coordinates of the light spot of the handle to be authenticated, a first target area of ​​the light spot of the handle to be authenticated in the second image is obtained; and,

[0010] The confidence of the light spot is obtained through a first target area and a second target area of ​​the light spot of the handle to be authenticated, wherein the second target area is an area corresponding to the light spot of the handle to be authenticated in the first image obtained according to the actual position coordinates of the light spot of the handle to be authenticated;

[0011] The light spots of the handles to be authenticated are identified based on the confidence levels of the light spots of the handles to be authenticated, so as to obtain the target light spots of the handles.

[0012] A second aspect of the present application provides a VR device, including a processor and a memory, wherein the processor and the memory are connected via a bus;

[0013] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0014] At any moment, the first image corresponding to the moment is subjected to handle light spot recognition using a light spot recognition algorithm to obtain the actual position coordinates of each handle light spot to be authenticated in the first image; wherein the first image is a short exposure image containing a VR handle, and the LED light in the VR handle is turned on; and,

[0015] For any handle light spot to be authenticated, based on the actual position coordinates of the handle light spot to be authenticated, project the handle light spot to be authenticated in a second image corresponding to the moment to obtain the projection position coordinates of the handle light spot to be authenticated in the second image, wherein the second image is a long exposure image including the VR handle and the LED light in the VR handle is in the off state; and,

[0016] According to the projection position coordinates of the light spot of the handle to be authenticated, a first target area of ​​the light spot of the handle to be authenticated in the second image is obtained; and,

[0017] The confidence of the light spot is obtained through a first target area and a second target area of ​​the light spot of the handle to be authenticated, wherein the second target area is an area corresponding to the light spot of the handle to be authenticated in the first image obtained according to the actual position coordinates of the light spot of the handle to be authenticated;

[0018] The light spots of the handles to be authenticated are identified based on the confidence levels of the light spots of the handles to be authenticated, so as to obtain the target light spots of the handles.

[0019] According to a third aspect provided by an embodiment of the present invention, a computer storage medium is provided, wherein the computer storage medium stores a computer program, and the computer program is used to execute the method as described in the first aspect.

[0020] In the above-mentioned embodiment of the present application, the actual position coordinates of each handle light spot to be authenticated obtained in the short exposure image are projected in the long exposure image to obtain the projection position coordinates of each handle light spot to be authenticated, and the confidence of each handle light spot to be authenticated is determined by using the first target area obtained by the projection position coordinates of each handle light spot to be authenticated and the second target area obtained based on the actual position coordinates of each handle light spot to be authenticated. Then, based on the confidence of each handle light spot to be authenticated, each handle light spot to be authenticated is identified to obtain the target handle light spot. Therefore, in the embodiment of the present application, the light spot corresponding to the background light can be filtered out in the scene with background light, and the light spot corresponding to the VR handle light can be identified, thereby improving the accuracy of the handle light spot identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0022] Figure 1 A schematic diagram of a handle light spot in the prior art provided by an embodiment of the present application is exemplarily shown;

[0023] Figure 2 One of the flow charts of the light spot recognition method of the VR controller provided in the embodiment of the present application is exemplarily shown;

[0024] Figure 3 The schematic diagram of the process of determining the projection position coordinates of the light spot of the handle to be authenticated in the second image provided by the embodiment of the present application is exemplarily shown;

[0025] Figure 4 A schematic diagram for determining a first target area provided in an embodiment of the present application is exemplarily shown;

[0026] Figure 5 A schematic diagram of a first middle area provided in an embodiment of the present application is exemplarily shown;

[0027] Figure 6 The following is a schematic diagram of a process for determining a second target area provided in an embodiment of the present application;

[0028] Figure 7 The following is a schematic diagram of a process for determining the area of ​​a common area provided in an embodiment of the present application;

[0029] Figure 8 The second flowchart of the light spot recognition method of the VR controller provided in the embodiment of the present application is exemplarily shown;

[0030] Fig. 9 A schematic diagram of a light spot recognition device for a VR controller provided in an embodiment of the present application is exemplarily shown;

[0031] Fig.10 The hardware structure diagram of the VR device provided in the embodiment of the present application is exemplified. DETAILED DESCRIPTION

[0032] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0033] Based on the exemplary embodiments described in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the claims attached to this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of the disclosure can also constitute a complete implementation method separately.

[0034] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0035] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a series of components is not necessarily limited to those components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0036] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0037] The idea of ​​the embodiments of the present application is summarized below.

[0038] At present, because the handle itself is equipped with infrared LEDs (light-emitting diodes), but there are usually a lot of lights on the ceiling in scenes such as exhibitions, so for handles positioned with infrared light rings, the light arrangements of both sides are likely to be similar, and it is easy to mistake the ambient light for the light in the handle light ring. Figure 1 As shown, Figure 1 The light in the box is the light in the handle light ring, and the other lights are ambient lights. Therefore, the ambient lights may be mistakenly identified as handle lights, resulting in a low accuracy rate in identifying the handle's light spot.

[0039] Based on the problem of low accuracy of handle light spot recognition in the prior art. The embodiment of the present application provides a light spot recognition method for a VR handle, which projects the actual position coordinates of each handle light spot to be authenticated obtained in the short exposure image into the long exposure image to obtain the projection position coordinates of each handle light spot to be authenticated, and determines the confidence of each handle light spot to be authenticated by using the first target area obtained by the projection position coordinates of each handle light spot to be authenticated and the second target area obtained based on the actual position coordinates of each handle light spot to be authenticated. Then, based on the confidence of each handle light spot to be authenticated, each handle light spot to be authenticated is identified to obtain the target handle light spot. Therefore, in the embodiment of the present application, the light spot corresponding to the background light can be filtered out in the scene with background light, and the light spot corresponding to the VR handle light can be identified, thereby improving the accuracy of handle light spot recognition. Below, the light spot recognition method for the VR handle in the embodiment of the present application is described in detail in conjunction with the accompanying drawings.

[0040] like Figure 2 FIG. 1 is a flow chart of a method for identifying a light spot of a VR controller, which may include the following steps:

[0041] Step 201: for any moment, use a light spot recognition algorithm to perform handle light spot recognition on a first image corresponding to the moment, and obtain the actual position coordinates of each handle light spot to be authenticated in the first image; wherein the first image is a short exposure image containing a VR handle, and the LED light in the VR handle is turned on;

[0042] It should be noted that the light spot recognition algorithm in the embodiment of the present application can be set according to actual conditions, and the embodiment of the present application does not limit the light spot recognition algorithm.

[0043] Step 202: for any handle light spot to be authenticated, based on the actual position coordinates of the handle light spot to be authenticated, project the handle light spot to be authenticated in a second image corresponding to the moment to obtain the projection position coordinates of the handle light spot to be authenticated in the second image, wherein the second image is a long exposure image including the VR handle and the LED light in the VR handle is in the off state;

[0044] Next, a method for determining the projection position coordinates of the light spot of the handle to be authenticated in the second image is introduced. Figure 3 As shown, it is a schematic diagram of a process for determining the projection position coordinates of the light spot of the handle to be authenticated in the second image, which may include the following steps:

[0045] Step 301: obtaining a preset light spot distance range, wherein the light spot distance range includes a maximum distance and a minimum distance between a camera in a VR device and the light spot;

[0046] Step 302: According to the maximum distance and the actual position coordinates, the position coordinates of the first projection point of the light spot of the handle to be authenticated in the second image are obtained, and according to the minimum distance and the actual position coordinates, the position coordinates of the second projection point of the light spot of the handle to be authenticated in the second image are obtained; wherein the position coordinates of the first projection point of the light spot of the handle to be authenticated in the second image can be obtained by formula (1):

[0047]

[0048] Among them, q1 is the position coordinate of the first projection point, d1 is the maximum distance, p k is the actual position coordinate of the light spot k, f c () is a preset function that converts image position coordinates into normalized focal plane position coordinates, It is the conversion matrix between the inertial sensor and the VR camera in the pre-set VR device. is the world position coordinate matrix of the VR device output by the inertial sensor in the VR device at time t1, It is the inverse matrix of the world position coordinates of the VR device output by the inertial sensor in the VR device at time t0;

[0049] And, the position coordinates of the second projection point of the light spot of the handle to be authenticated in the second image can be obtained by formula (2):

[0050]

[0051] Among them, q2 is the second projection position coordinate, and d2 is the shortest distance.

[0052] Among them, f in the embodiment of the present application c () function is a function in the prior art, and The function is also a function in the prior art, and will not be described in detail in the embodiments of the present application.

[0053] Step 303: Based on the position coordinates of the first projection point and the position coordinates of the second projection point, obtain the distance between the first projection point and the second projection point; wherein the distance between the first projection point and the second projection point can be determined by formula (3):

[0054]

[0055] Among them, d is the distance between the first projection point and the second projection point, u1 is the horizontal position coordinate of the first projection point, u2 is the horizontal position coordinate of the second projection point, v1 is the vertical position coordinate of the first projection point, and v2 is the vertical position coordinate of the second projection point.

[0056] Step 304: If the distance is less than a first specified threshold, the position coordinates of the first projection point are determined as the projection position coordinates of the light spot of the handle to be authenticated in the second image.

[0057] It should be noted that the first specified threshold in the embodiment of the present application can be set according to actual conditions, and the embodiment of the present application does not limit the first specified threshold.

[0058] Step 203: obtaining a first target area of ​​the light spot of the handle to be authenticated in the second image according to the projection position coordinates of the light spot of the handle to be authenticated;

[0059] like Figure 4 As shown, it is a schematic diagram of a process for determining a first target area, which may include the following steps:

[0060] Step 401: taking the position point corresponding to the projection position coordinates of the handle light spot as the center point to obtain a first middle area of ​​a specified size;

[0061] The specified size in the embodiment of the present application is 21 pixels * 21 pixels, but the embodiment of the present application does not limit the specific value of the specified size, and the specific value of the specified size can be set according to actual conditions. Figure 5 As shown in FIG. 1 , it is a schematic diagram of the first intermediate region. It can be seen from the figure that point a is the center point, and region ABCD is the first intermediate region with point a as the center point. However, the shape of the first intermediate region in the embodiment of the present application can be set according to actual conditions, and the embodiment of the present application does not limit the first intermediate region.

[0062] Step 402: binarizing the first middle region according to the gray value of the center point to obtain the binarized first middle region;

[0063] In one embodiment, step 402 can be specifically implemented as follows: multiplying the grayscale value of the center point by a specified value to obtain a binarization processing threshold, setting the pixel value of each pixel point in the first intermediate area whose pixel value is less than the binarization processing threshold to a first specified pixel value, and setting the pixel value of each pixel point in the first intermediate area whose pixel value is greater than the binarization processing threshold to a second specified pixel value, to obtain the first intermediate area after binarization processing.

[0064] It should be noted that the specified value in the embodiment of the present application is a real number, and the specified value in the embodiment of the present application is 0.8, but the embodiment of the present application does not limit the specified value here, and the specific value of the specified value can be set according to the actual situation. In the embodiment of the present application, the first specified pixel value is 0, and the second specified pixel value is 255.

[0065] Step 403: Determine the region consisting of the pixel points whose pixel values ​​are greater than the second specified threshold in the first intermediate region after the binarization process as the first target region.

[0066] The second specified threshold in the embodiment of the present application is equal to the binarization processing threshold mentioned above.

[0067] Step 204: obtaining the confidence of the light spot of the handle to be authenticated through the first target area and the second target area, wherein the second target area is the area corresponding to the light spot of the handle to be authenticated in the first image obtained according to the actual position coordinates of the light spot of the handle to be authenticated;

[0068] First, the method for determining the second target area in the embodiment of the present application is introduced. Figure 6As shown, it is a schematic diagram of a process for determining the second target area, which may include the following steps:

[0069] Step 601: taking the position point corresponding to the actual position coordinates of the light spot of the handle to be authenticated as the center point to obtain a second middle area of ​​a specified size;

[0070] The method for determining the second middle area in the embodiment of the present application is the same as that of the first middle area described above, and the embodiment of the present application does not elaborate on the method for determining the second middle area here.

[0071] Step 602: binarizing the second middle region according to the gray value of the center point to obtain the binarized second middle region;

[0072] In one embodiment, step 602 can be specifically implemented as follows: setting the pixel value of each pixel point in the second intermediate area whose pixel value is less than the binarization processing threshold to a first specified pixel value, and setting the pixel value of each pixel point in the second intermediate area whose pixel value is greater than the binarization processing threshold to a second specified pixel value, to obtain the second intermediate area after binarization processing.

[0073] It should be noted that the specific method for determining the binarization processing threshold mentioned above is the determination method in step 402, and the implementation of this application will not be repeated here.

[0074] Step 603: Determine the region consisting of the pixel points whose pixel values ​​are greater than the second specified threshold in the second intermediate region after the binarization process as the second target region.

[0075] The second specified threshold in the embodiment of the present application is equal to the binarization processing threshold mentioned above.

[0076] Next, the method of determining the confidence of the light spot in step 204 is described. Specifically, the confidence of the light spot can be determined in the following two ways:

[0077] Method 1: If the number of pixels in the first target area is greater than the third specified threshold, the area of ​​the common area between the first target area and the second target area is divided by the area of ​​the second target area to obtain the confidence of the light spot. Specifically, the confidence of the light spot can be determined by formula (4):

[0078]

[0079] Among them, μ is the confidence of the light spot, S 共 is the area of ​​the common area, and S2 is the area of ​​the second target area.

[0080] It should be noted that the third designated threshold in the embodiment of the present application is 4, but the embodiment of the present application does not limit the third designated threshold, and the specific value of the third designated threshold can be set according to actual conditions.

[0081] Next, a method for determining the area of ​​the common area between the first target area and the second target area is introduced. Figure 7 As shown, it is a schematic diagram of a process for determining the area of ​​a common area, which may include the following steps:

[0082] Step 701: traverse each pixel point in the first target area according to a preset order;

[0083] The preset order in the embodiment of the present application may be from left to right, from top to bottom. However, the specific preset order is not limited in the embodiment of the present application, and the preset order may be limited according to actual conditions.

[0084] Step 702: for any traversed pixel point, based on the position of the pixel point in the first target area, determine whether there is a target pixel point in the second target area with the same position as the traversed pixel point; if so, execute step 703; if not, execute step 704;

[0085] Step 703: increasing the area of ​​the common area between the first target area and the second target area by a specified value to obtain an intermediate area;

[0086] The specified value in the embodiment of the present application is 1, but the specific specified value can be set according to actual conditions. The embodiment of the present application does not set the specified value here.

[0087] Step 704: Determine whether the traversal of each pixel point in the first target area is completed, if yes, execute step 705, if no, return to execute step 701;

[0088] Step 705: Determine the middle area as the area of ​​the common area between the first target area and the second target area.

[0089] In one embodiment, the area of ​​the second target region is obtained by:

[0090] The number of each pixel point in the second target area is determined as the area of ​​the second target area.

[0091] Step 205: Identify each handle light spot to be authenticated based on the confidence level of each handle light spot to be authenticated to obtain a target handle light spot.

[0092] It should be noted that: in the embodiment of the present application, the handle spot to be authenticated with a confidence greater than the specified confidence is determined as the target handle spot. However, the specified confidence in the embodiment of the present application can be set according to actual conditions, and the embodiment of the present application does not limit the specified confidence.

[0093] In one embodiment, step 205 can be specifically implemented as follows: for any handle light spot to be authenticated whose confidence is greater than a specified confidence among the handle light spots to be authenticated, based on the actual position coordinates of the handle light spot to be authenticated and the predicted position coordinates of the VR handle, the distance between the handle light spot and the VR handle is obtained, and if the distance is less than the specified distance, the handle light spot to be authenticated is determined as the target handle light spot; if the distance is not less than the specified distance, the process ends. The predicted position coordinates of the VR handle are predicted based on the actual position of the VR handle in the short exposure image corresponding to the previous moment.

[0094] The predicted position of the VR handle in the embodiment of the present application is obtained by inputting the actual position of the VR handle in the short exposure image corresponding to the previous moment and the IMU data in the short exposure image of the current frame into the pose estimation algorithm. However, the pose estimation algorithm in the embodiment of the present application can be set according to the actual situation, and the embodiment of the present application does not limit the pose estimation algorithm here. The specified distance in the embodiment of the present application is 5 pixels, but the embodiment of the present application does not limit the specified distance here, and the specific value of the specified distance can be set according to the actual situation.

[0095] Method 2: If the number of pixels in the first target area is not greater than the third specified threshold, the fourth specified threshold is determined as the confidence level of the light spot.

[0096] It should be noted that the fourth specified threshold in the embodiment of the present application is 0, but the embodiment of the present application does not limit the fourth specified threshold here. The specific value of the fourth specified threshold in the embodiment of the present application can be limited according to actual conditions.

[0097] In order to further connect the methods in this application, such as Figure 8 FIG. 1 is a flow chart of a method for generating a hand data set in the present application, which may include the following steps:

[0098] Step 801: for any moment, use a light spot recognition algorithm to perform handle light spot recognition on a first image corresponding to the moment, and obtain the actual position coordinates of each handle light spot to be authenticated in the first image; wherein the first image is a short exposure image including a VR handle, and the LED light in the VR handle is turned on;

[0099] Step 802: obtaining a preset light spot distance range, wherein the light spot distance range includes a maximum distance and a minimum distance between a camera in a VR device and the light spot;

[0100] Step 803: Obtaining the position coordinates of the first projection point of the light spot of the to-be-authenticated handle in the second image according to the maximum distance and the actual position coordinates, wherein the second image is a long exposure image including the VR handle and the LED light in the VR handle is in the off state;

[0101] Step 804: Obtaining the position coordinates of the second projection point of the light spot of the handle to be authenticated in the second image according to the minimum distance and the actual position coordinates;

[0102] Step 805: taking the position point corresponding to the projection position coordinates of the handle light spot as the center point to obtain a first middle area of ​​a specified size;

[0103] Step 806: binarizing the first middle region according to the gray value of the center point to obtain the binarized first middle region;

[0104] Step 807: determining the region consisting of the pixel points whose pixel values ​​are greater than the second specified threshold in the first intermediate region after the binarization process as the first target region;

[0105] Step 808: Determine whether the number of pixels in the first target area is greater than a third specified threshold, if so, execute step 809, if not, execute step 810;

[0106] Step 809: Dividing the area of ​​the common area between the first target area and the second target area by the area of ​​the second target area to obtain the confidence of the light spot, wherein the second target area is an area corresponding to the light spot of the handle to be authenticated in the first image obtained according to the actual position coordinates of the light spot of the handle to be authenticated;

[0107] Step 810: Determine a fourth specified threshold as the confidence level of the light spot;

[0108] Step 811: Identify each handle light spot to be authenticated based on the confidence level of each handle light spot to be authenticated to obtain a target handle light spot.

[0109] Based on the same inventive concept, the light spot recognition method of the VR handle disclosed above can also be implemented by a light spot recognition device of the VR handle. The effect of the light spot recognition device of the VR handle is similar to that of the aforementioned method, and will not be repeated here.

[0110] Fig. 9Schematic diagram of the structure of a light spot recognition device of a VR handle according to an embodiment of the present disclosure.

[0111] like Fig. 9 As shown, the light spot recognition device 900 of the VR handle of the present disclosure may include a light spot recognition module 910, a projection module 920, a first target area determination module 930, a confidence determination module 940 and a target handle light spot determination module 950.

[0112] The light spot recognition module 910 is used to perform handle light spot recognition on the first image corresponding to any moment using a light spot recognition algorithm to obtain the actual position coordinates of each handle light spot to be authenticated in the first image; wherein the first image is a short exposure image including a VR handle, and the LED light in the VR handle is turned on; and,

[0113] The projection module 920 is used to project any handle light spot to be authenticated in a second image corresponding to the moment based on the actual position coordinates of the handle light spot to be authenticated, so as to obtain the projection position coordinates of the handle light spot to be authenticated in the second image, wherein the second image is a long exposure image including the VR handle and the LED light in the VR handle is turned off; and

[0114] The first target region determining module 930 is used to obtain the first target region of the light spot of the handle to be authenticated in the second image according to the projection position coordinates of the light spot of the handle to be authenticated; and

[0115] The confidence determination module 940 is used to obtain the confidence of the light spot of the handle to be authenticated through the first target area and the second target area of ​​the light spot of the handle to be authenticated, wherein the second target area is the area corresponding to the light spot of the handle to be authenticated in the first image obtained according to the actual position coordinates of the light spot of the handle to be authenticated;

[0116] The target handle light spot determination module 950 is used to identify each handle light spot to be authenticated based on the confidence level of each handle light spot to be authenticated, so as to obtain a target handle light spot.

[0117] In one embodiment, the projection module 920 is specifically used for:

[0118] Obtaining a preset light spot distance range, wherein the light spot distance range includes a maximum distance and a minimum distance between a camera in a VR device and the light spot;

[0119] According to the maximum distance and the actual position coordinates, the position coordinates of a first projection point of the light spot of the handle to be authenticated in the second image are obtained, and according to the minimum distance and the actual position coordinates, the position coordinates of a second projection point of the light spot of the handle to be authenticated in the second image are obtained;

[0120] Based on the position coordinates of the first projection point and the position coordinates of the second projection point, obtaining a distance between the first projection point and the second projection point;

[0121] If the distance is less than a first specified threshold, the position coordinates of the first projection point are determined as the projection position coordinates of the light spot of the handle to be authenticated in the second image.

[0122] In one embodiment, the projection module 920 is further used for:

[0123] The position coordinates of the first projection point are obtained by the following formula:

[0124]

[0125] Among them, q1 is the position coordinate of the first projection point, d1 is the maximum distance, p k is the actual position coordinate of the light spot k, f c The function is a preset function that converts the image position coordinates into the normalized focal plane position coordinates. It is the conversion matrix between the inertial sensor and the VR camera in the pre-set VR device. is the world position coordinate matrix of the inertial sensor in the VR device at time t1;

[0126] The position coordinates of the second projection point are obtained by the following formula:

[0127]

[0128] Among them, q2 is the second projection position coordinate, and d2 is the shortest distance.

[0129] In one embodiment, the first target area determination module 930 is specifically configured to:

[0130] Taking the position point corresponding to the projection position coordinates of the handle light spot as the center point, obtaining a first middle area of ​​a specified size;

[0131] Binarize the first middle area according to the gray value of the center point to obtain the binarized first middle area;

[0132] The area consisting of pixel points whose pixel values ​​in the first intermediate area after the binarization process are greater than the second specified threshold is determined as the first target area.

[0133] In one embodiment, the apparatus further comprises:

[0134] The second target area determination module 960 is configured to obtain the second target area by:

[0135] Taking the position point corresponding to the actual position coordinates of the light spot of the handle to be authenticated as the center point, a second middle area of ​​a specified size is obtained;

[0136] Binarize the second middle area according to the gray value of the center point to obtain the second middle area after the binarization process;

[0137] The area consisting of pixel points whose pixel values ​​are greater than the second specified threshold in the second intermediate area after the binarization process is determined as the second target area.

[0138] In one embodiment, the confidence determination module 940 is specifically configured to:

[0139] If the number of pixels in the first target area is greater than a third specified threshold, the area of ​​the common area between the first target area and the second target area is divided by the area of ​​the second target area to obtain the confidence of the light spot; or,

[0140] If the number of pixels in the first target area is not greater than the third specified threshold, the fourth specified threshold is determined as the confidence level of the light spot.

[0141] In one embodiment, the apparatus further comprises:

[0142] The shared area determination module 970 is used to obtain the area of ​​the shared area by:

[0143] Traverse each pixel point in the first target area according to a preset order; for any traversed pixel point, determine whether there is a target pixel point in the second target area with the same position as the traversed pixel point based on the position of the pixel point in the first target area; if there is, increase the area of ​​the common area between the first target area and the second target area by a specified value, and return to the step of traversing each pixel point in the first target area according to the preset order until the traversal of each pixel point in the first target area is completed, and the area of ​​the common area is obtained; if there is not, return to the step of traversing each pixel point in the first target area according to the preset order until the traversal of each pixel point in the first target area is completed, and the area of ​​the common area is obtained;

[0144] The area determination module 980 is configured to obtain the area of ​​the second target area by:

[0145] The number of each pixel point in the second target area is determined as the area of ​​the second target area.

[0146] In one embodiment, the target handle light spot determination module 950 is specifically used to:

[0147] For any handle light spot to be authenticated whose confidence is greater than a specified confidence among the handle light spots to be authenticated, the distance between the handle light spot and the VR handle is obtained based on the actual position coordinates of the handle light spot to be authenticated and the predicted position coordinates of the VR handle, wherein the predicted position coordinates of the VR handle are predicted based on the actual position of the VR handle in the short exposure image corresponding to the previous moment; and

[0148] If the distance is less than the specified distance, the handle light spot to be considered is determined as the target handle light spot.

[0149] After introducing a light spot recognition method and device for a VR handle according to an exemplary embodiment of the present invention, next, a VR device according to another exemplary embodiment of the present invention is introduced.

[0150] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0151] In some possible implementations, the VR device according to the present invention may include at least one processor and at least one computer storage medium. The computer storage medium stores program code, and when the program code is executed by the processor, the processor executes the steps of the light spot recognition method of the VR handle according to various exemplary embodiments of the present invention described above in this specification. For example, the processor may execute the following steps: Figure 2 Steps 201-205 shown in .

[0152] Refer to the following Fig.10 The VR device 1000 according to this embodiment of the present invention is described. Fig.10 The VR device 1000 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0153] like Fig.10 As shown, the VR device 1000 is presented in the form of a general VR device. The components of the VR device 1000 may include, but are not limited to: the at least one processor 1001, the at least one computer storage medium 1002, and a bus 1003 connecting different system components (including the computer storage medium 1002 and the processor 1001).

[0154] Bus 1003 represents one or more of several types of bus structures, including a computer storage media bus or computer storage media controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0155] The computer storage medium 1002 may include readable media in the form of volatile computer storage media, such as random access computer storage media (RAM) 1021 and / or cache storage media 1022 , and may further include read-only computer storage media (ROM) 1023 .

[0156] The computer storage medium 1002 may also include a program / utility 1025 having a set (at least one) of program modules 1024, such program modules 1024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0157] The VR device 1000 may also communicate with one or more external devices 1004 (e.g., keyboards, pointing devices, etc.), and may also communicate with one or more devices that enable a user to interact with the VR device 1000 and / or any device that enables the VR device 1000 to communicate with one or more other VR devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1005. Furthermore, the VR device 1000 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 1006. As shown, the network adapter 1006 communicates with other modules for the VR device 1000 via a bus 1003. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the VR device 1000, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0158] In some possible implementations, various aspects of the light spot recognition method for a VR handle provided by the present invention may also be implemented in the form of a program product, which includes a program code. When the program product runs on a computer device, the program code is used to enable the computer device to execute the steps of the light spot recognition method for a VR handle according to various exemplary embodiments of the present invention described above in this specification.

[0159] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A light spot recognition method for a VR handle, characterized in that: The method comprises: At any moment, the first image corresponding to the moment is subjected to handle light spot recognition using a light spot recognition algorithm to obtain the actual position coordinates of each handle light spot to be authenticated in the first image; wherein the first image is a short exposure image containing a VR handle, and the LED light in the VR handle is turned on; and, For any handle light spot to be authenticated, based on the actual position coordinates of the handle light spot to be authenticated, project the handle light spot to be authenticated in a second image corresponding to the moment to obtain the projection position coordinates of the handle light spot to be authenticated in the second image, wherein the second image is a long exposure image including the VR handle and the LED light in the VR handle is in the off state; and, According to the projection position coordinates of the light spot of the handle to be authenticated, a first target area of ​​the light spot of the handle to be authenticated in the second image is obtained; and, The confidence of the light spot is obtained through a first target area and a second target area of ​​the light spot of the handle to be authenticated, wherein the second target area is an area corresponding to the light spot of the handle to be authenticated in the first image obtained according to the actual position coordinates of the light spot of the handle to be authenticated; The light spots of the handles to be authenticated are identified based on the confidence levels of the light spots of the handles to be authenticated, so as to obtain the target light spots of the handles.

2. The method according to claim 1, characterized in that: The step of projecting the light spot of the handle to be authenticated in a second image corresponding to the moment based on the actual position coordinates of the light spot of the handle to be authenticated, and obtaining the projection position coordinates of the light spot of the handle to be authenticated in the second image, comprises: Obtaining a preset light spot distance range, wherein the light spot distance range includes a maximum distance and a minimum distance between a camera in a VR device and the light spot; According to the maximum distance and the actual position coordinates, the position coordinates of a first projection point of the light spot of the handle to be authenticated in the second image are obtained, and according to the minimum distance and the actual position coordinates, the position coordinates of a second projection point of the light spot of the handle to be authenticated in the second image are obtained; Based on the position coordinates of the first projection point and the position coordinates of the second projection point, obtaining a distance between the first projection point and the second projection point; If the distance is less than a first specified threshold, the position coordinates of the first projection point are determined as the projection position coordinates of the light spot of the handle to be authenticated in the second image.

3. The method according to claim 2, characterized in that The step of obtaining the position coordinates of the first projection point of the light spot of the handle to be authenticated in the second image according to the maximum distance and the actual position coordinates includes: The position coordinates of the first projection point are obtained by the following formula: Among them, q1 is the position coordinate of the first projection point, d1 is the maximum distance, p k is the actual position coordinate of the light spot k, f c () is a preset function that converts image position coordinates into normalized focal plane position coordinates. It is the conversion matrix between the inertial sensor and the VR camera in the pre-set VR device. It is the world position coordinate matrix of the VR device output by the inertial sensor in the VR device at time t1; The step of obtaining the position coordinates of the second projection point of the light spot of the handle to be authenticated in the second image according to the minimum distance and the actual position coordinates includes: The position coordinates of the second projection point are obtained by the following formula: Among them, q2 is the second projection position coordinate, and d2 is the shortest distance.

4. The method according to claim 1, characterized in that: The step of obtaining the first target area of ​​the light spot of the handle to be authenticated in the second image according to the projection position coordinates of the light spot of the handle to be authenticated includes: Taking the position point corresponding to the projection position coordinates of the handle light spot as the center point, obtaining a first middle area of ​​a specified size; Binarize the first middle area according to the gray value of the center point to obtain the binarized first middle area; The area consisting of pixel points whose pixel values ​​in the first intermediate area after the binarization process are greater than the second specified threshold is determined as the first target area.

5. The method according to claim 1, characterized in that The second target area is obtained by: Taking the position point corresponding to the actual position coordinates of the light spot of the handle to be authenticated as the center point, a second middle area of ​​a specified size is obtained; Binarize the second middle area according to the gray value of the center point to obtain the second middle area after the binarization process; The area consisting of pixel points whose pixel values ​​are greater than the second specified threshold in the second intermediate area after the binarization process is determined as the second target area.

6. The method according to claim 1, characterized in that The step of obtaining the confidence of the light spot through the first target area and the second target area of ​​the light spot of the handle to be authenticated includes: If the number of pixels in the first target area is greater than a third specified threshold, the area of ​​the common area between the first target area and the second target area is divided by the area of ​​the second target area to obtain the confidence of the light spot; or, If the number of pixels in the first target area is not greater than the third specified threshold, the fourth specified threshold is determined as the confidence level of the light spot.

7. The method according to claim 6, characterized in that The area of ​​the common area is obtained by: Traversing each pixel point in the first target area in a preset order; for any traversed pixel point, determining whether there is a target pixel point in the second target area with the same position as the traversed pixel point based on the position of the pixel point in the first target area; If it exists, then after increasing the area of ​​the common area between the first target area and the second target area by a specified value, return to the step of traversing each pixel point in the first target area in a preset order until the traversal of each pixel point in the first target area is completed, and the area of ​​the common area is obtained; if it does not exist, return to the step of traversing each pixel point in the first target area in a preset order until the traversal of each pixel point in the first target area is completed, and the area of ​​the common area is obtained; The area of ​​the second target region is obtained by: The number of each pixel point in the second target area is determined as the area of ​​the second target area.

8. The method according to claim 1, characterized in that: The identifying each handle light spot to be authenticated based on the confidence of each handle light spot to be authenticated to obtain a target handle light spot includes: For any handle light spot to be authenticated whose confidence is greater than a specified confidence among the handle light spots to be authenticated, the distance between the handle light spot and the VR handle is obtained based on the actual position coordinates of the handle light spot to be authenticated and the predicted position coordinates of the VR handle, wherein the predicted position coordinates of the VR handle are predicted based on the actual position of the VR handle in the short exposure image corresponding to the previous moment; and If the distance is less than the specified distance, the handle light spot to be considered is determined as the target handle light spot.

9. A VR device, characterized in that: comprising a processor and a memory, wherein the processor and the memory are connected via a bus; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: At any moment, the first image corresponding to the moment is subjected to handle light spot recognition using a light spot recognition algorithm to obtain the actual position coordinates of each handle light spot to be authenticated in the first image; wherein the first image is a short exposure image containing a VR handle, and the LED light in the VR handle is turned on; and, For any handle light spot to be authenticated, based on the actual position coordinates of the handle light spot to be authenticated, project the handle light spot to be authenticated in a second image corresponding to the moment to obtain the projection position coordinates of the handle light spot to be authenticated in the second image, wherein the second image is a long exposure image including the VR handle and the LED light in the VR handle is in the off state; and, According to the projection position coordinates of the light spot of the handle to be authenticated, a first target area of ​​the light spot of the handle to be authenticated in the second image is obtained; and, The confidence of the light spot is obtained through a first target area and a second target area of ​​the light spot of the handle to be authenticated, wherein the second target area is an area corresponding to the light spot of the handle to be authenticated in the first image obtained according to the actual position coordinates of the light spot of the handle to be authenticated; The light spots of the handles to be authenticated are identified based on the confidence levels of the light spots of the handles to be authenticated, so as to obtain the target light spots of the handles.

10. The VR device according to claim 9, characterized in that: The processing is executed based on the actual position coordinates of the light spot of the handle to be authenticated, projecting the light spot of the handle to be authenticated in the second image corresponding to the moment, and obtaining the projection position coordinates of the light spot of the handle to be authenticated in the second image, and is specifically configured as follows: Obtaining a preset light spot distance range, wherein the light spot distance range includes a maximum distance and a minimum distance between a camera in a VR device and the light spot; According to the maximum distance and the actual position coordinates, the position coordinates of a first projection point of the light spot of the handle to be authenticated in the second image are obtained, and according to the minimum distance and the actual position coordinates, the position coordinates of a second projection point of the light spot of the handle to be authenticated in the second image are obtained; Based on the position coordinates of the first projection point and the position coordinates of the second projection point, obtaining a distance between the first projection point and the second projection point; If the distance is less than a first specified threshold, the position coordinates of the first projection point are determined as the projection position coordinates of the light spot of the handle to be authenticated in the second image.

Images