A VR handle light spot identification method and a VR device

Abstract

The application provides a VR handle light spot identification method and a VR device, and improves the handle light spot identification accuracy. The method comprises the following steps: performing handle light spot identification on a first image at any moment by using a light spot identification algorithm to obtain actual position coordinates of each to-be-authenticated handle light spot in the first image; the first image is a short exposure image containing a VR handle and a LED lamp in the VR handle being in an open state; projecting the to-be-authenticated handle light spot in a second image based on the actual position coordinates of the to-be-authenticated handle light spot to obtain projection position coordinates of the to-be-authenticated handle light spot, the second image being a long exposure image containing the VR handle and the LED lamp in the VR handle being in a closed state; obtaining a confidence of the light spot through a first target area and a second target area of the to-be-authenticated handle light spot in the second image obtained according to the projection position coordinates of the to-be-authenticated handle light spot; and identifying each to-be-authenticated handle light spot based on the confidence of each to-be-authenticated handle light spot to obtain a target handle light spot.

Description

TECHNICAL FIELD

[0001] The present application relates to the virtual reality technical field, and particularly relates to a VR handle light spot identification method and a VR device. BACKGROUND

[0002] VR (Virtual Reality) headgear display is applied in various industries and has a wide range of applications. For example, in education and training, fire drill, virtual driving, and real estate projects. While providing users with an immersive visual feast, natural and smooth interaction is the most important manifestation of user satisfaction in user experience, so the control of the handle is an indispensable interaction method.

[0003] The VR device has 3DOF (3Degree of Freedom) and 6DOF (6Degree of Freedom) for positioning tracking of the handle. Among them, 6DOF is the current mainstream control interaction method, so the application scenario of the present scheme is mainly in 6DOF. The positioning of the handle pose is mainly determined according to the handle light spot captured by the VR device. Therefore, the accuracy of identifying the handle light spot is crucial for the positioning of the handle.

[0004] At present, because the handle itself is installed with an infrared LED (light-emitting diode), but there are usually a large number of lights on the ceiling in exhibition and other scenes, so for the handle with an infrared light ring, the arrangement of the lights on both sides is prone to similar conditions, and the environment light is prone to being misidentified as the light in the handle light ring. As shown in FIG. 1, Figure 1 Figure 1 The light in the box in FIG. 1 is the light in the handle light ring, and the other lights are environment lights, so the environment light is misidentified as the handle light, resulting in a low accuracy of identifying the light spot of the handle. SUMMARY

[0005] The present application provides a VR handle light spot identification method and a VR device, which is used to improve the accuracy of identifying the light spot of the handle.

[0006] In a first aspect, the present application provides a VR handle light spot identification method, which comprises the following steps:

[0007] For any moment, a handle light spot is identified in a first image corresponding to the moment by using a light spot identification algorithm to obtain the actual position coordinates of each to-be-authenticated handle light spot in the first image; wherein the first image is a short exposure image containing a VR handle, and the LED light in the VR handle is in an open state; and ​

[0008] For any one to be authenticated handle light spot, based on the actual position coordinates of the to be authenticated handle light spot, the to be authenticated handle light spot is projected in the second image corresponding to the time, to obtain the projection position coordinates of the to be authenticated handle light spot in the second image, wherein the second image is a long exposure image including the VR handle, and the LED lamp in the VR handle is in the closed state; and,

[0009] According to the projection position coordinates of the to be authenticated handle light spot, the first target area of the to be authenticated handle light spot in the second image is obtained; and,

[0010] The confidence of the light spot is obtained through the first target area and the second target area of the to be authenticated handle light spot, wherein the second target area is the corresponding area of the to be authenticated handle light spot in the first image obtained according to the actual position coordinates of the to be authenticated handle light spot;

[0011] Based on the confidence of each to be authenticated handle light spot, the to be authenticated handle light spot is identified, and a target handle light spot is obtained.

[0012] The second aspect of the application provides a VR device, comprising a processor and a memory, the processor and the memory are connected through a bus;

[0013] The memory stores a computer program, and the processor is configured to execute the following operations based on the computer program:

[0014] For any one time, a handle light spot recognition algorithm is used to recognize the handle light spot of a first image corresponding to the time, to obtain the actual position coordinates of each to be authenticated handle light spot in the first image; wherein the first image is a short exposure image containing a VR handle, and the LED lamp in the VR handle is in the open state; and,

[0015] For any one to be authenticated handle light spot, based on the actual position coordinates of the to be authenticated handle light spot, the to be authenticated handle light spot is projected in the second image corresponding to the time, to obtain the projection position coordinates of the to be authenticated handle light spot in the second image, wherein the second image is a long exposure image including the VR handle, and the LED lamp in the VR handle is in the closed state; and,

[0016] According to the projection position coordinates of the to be authenticated handle light spot, the first target area of the to be authenticated handle light spot in the second image is obtained; and,

[0017] obtain the confidence of the to-be-authenticated handle light spot through the first target region and the second target region of the to-be-authenticated handle light spot, wherein the second target region is a region corresponding to the to-be-authenticated handle light spot in the first image according to the actual position coordinates of the to-be-authenticated handle light spot;

[0018] identify the to-be-authenticated handle light spot based on the confidence of each to-be-authenticated handle light spot, and obtain a target handle light spot.

[0019] According to a third aspect provided in an embodiment of the present application, a computer storage medium is provided, which stores a computer program for executing the method according to the first aspect.

[0020] In the above embodiments of the present application, the actual position coordinates of each to-be-authenticated handle light spot obtained in the short-exposure image are used to project each to-be-authenticated handle light spot in the long-exposure image, to obtain the projection position coordinates of each to-be-authenticated handle light spot. The first target region obtained by using the projection position coordinates of each to-be-authenticated handle light spot and the second target region obtained based on the actual position coordinates of each to-be-authenticated handle light spot are used to determine the confidence of each to-be-authenticated handle light spot, respectively. Then, each to-be-authenticated handle light spot is identified based on the confidence of each to-be-authenticated handle light spot, to obtain a target handle light spot. Thus, in the scene with background light, the light spot corresponding to the background light can be filtered out, and the light spot corresponding to the VR handle light can be identified, thereby improving the accuracy of the identification of the handle light spot. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative labor.

[0022] Figure 1 An exemplary schematic diagram of the handle light spot in the prior art provided by the embodiments of the present application is shown;

[0023] Figure 2 An exemplary flowchart of the light spot identification method of the VR handle provided by the embodiments of the present application is shown;

[0024] Figure 3 An exemplary flowchart of determining the projection position coordinates of the to-be-authenticated handle light spot in the second image provided by the embodiments of the present application is shown;

[0025] Figure 4 An exemplary diagram for determining the first target region provided by the embodiments of the present application is shown;

[0026] Figure 5 Fig. 1 shows a schematic diagram of a first intermediate region according to an example embodiment of the present application;

[0027] Figure 6 Fig. 2 shows a schematic diagram of a process of determining a second target region according to an example embodiment of the present application;

[0028] Figure 7 Fig. 3 shows a schematic diagram of a process of determining an area of a common region according to an example embodiment of the present application;

[0029] Figure 8 Fig. 4 shows a schematic diagram of a process of a light spot recognition method of a VR handle according to an example embodiment of the present application;

[0030] Figure 9 Fig. 5 shows a schematic diagram of a light spot recognition device of a VR handle according to an example embodiment of the present application;

[0031] Figure 10 Fig. 6 shows a hardware structure diagram of a VR device according to an example embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the objects, implementation manners and advantages of the present application clearer, the following will combine the drawings in the example embodiments of the present application to clearly and completely describe the example implementation manners of the present application. Obviously, the described example embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application.

[0033] Based on the example embodiments described in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the claims of the present application. In addition, although the disclosure in the present application is introduced according to one or several examples, it should be understood that each aspect of the disclosure can also constitute a complete embodiment independently.

[0034] It should be noted that the brief description of the terms in the present application is only for the convenience of understanding the following described embodiments, and is not intended to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0035] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to be omnipresent but not exclusive; for example, a product or device comprising a series of components is not necessarily limited to those explicitly listed, but may include other components not explicitly listed or inherent to such product or device.

[0036] As used in this application, the term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0037] The following is an overview of the ideas behind the embodiments of this application.

[0038] Currently, because the controller itself is equipped with infrared LEDs (light-emitting diodes), but in scenarios such as exhibitions, there are usually many lights on the ceiling, so for controllers with infrared LED ring positioning, the light arrangements of both sides can easily become similar, leading to misidentification of ambient light as the light from the controller's LED ring. For example... Figure 1 As shown, Figure 1 The light in the box is the light from the controller's light ring, while the other lights are ambient lights. Therefore, there is a possibility that the ambient light may be mistaken for the controller light, resulting in a low accuracy rate in recognizing the controller's light spot.

[0039] Existing technologies suffer from low accuracy in identifying the light spots of VR controllers. This application provides a method for identifying the light spots of VR controllers. The method involves projecting the actual position coordinates of each controller light spot to be certified, obtained from a short-exposure image, onto a long-exposure image to obtain the projected position coordinates. A first target region derived from these coordinates and a second target region derived from the actual position coordinates are used to determine the confidence level of each controller light spot. Then, based on these confidence levels, each controller light spot is identified to obtain the target controller light spot. Therefore, in scenes with background lighting, this application can filter out light spots corresponding to the background lighting and identify the light spots corresponding to the VR controller light, improving the accuracy of controller light spot identification. The VR controller light spot identification method of this application will be described in detail below with reference to the accompanying drawings.

[0040] likeFigure 2 As shown in FIG. 1, it is a flowchart of the spot recognition method of the VR handle, which can include the following steps:

[0041] Step 201: For any moment, use the spot recognition algorithm to perform handle spot recognition on a first image corresponding to the moment to obtain actual position coordinates of each to-be-authenticated handle spot in the first image; wherein the first image is a short-exposure image containing a VR handle and LED lights in the VR handle are in an open state;

[0042] It should be noted that the spot recognition algorithm in the embodiments of the present application can be set according to actual conditions, and the embodiments of the present application do not limit the spot recognition algorithm here.

[0043] Step 202: For any to-be-authenticated handle spot, based on the actual position coordinates of the to-be-authenticated handle spot, project the to-be-authenticated handle spot in a second image corresponding to the moment to obtain projection position coordinates of the to-be-authenticated handle spot in the second image, wherein the second image is a long-exposure image including the VR handle and the LED lights in the VR handle are in a closed state;

[0044] Next, the method for determining the projection position coordinates of the to-be-authenticated handle spot in the second image will be introduced, as shown in FIG. 2, which is a flowchart for determining the projection position coordinates of the to-be-authenticated handle spot in the second image, which can include the following steps: Figure 3

[0045] Step 301: Obtain a pre-set spot distance range, wherein the spot distance range includes a maximum distance and a minimum distance between a camera in a VR device and a spot;

[0046] Step 302: According to the maximum distance and the actual position coordinates, obtain the position coordinates of a first projection point of the to-be-authenticated handle spot in the second image, and according to the minimum distance and the actual position coordinates, obtain the position coordinates of a second projection point of the to-be-authenticated handle spot in the second image; wherein the position coordinates of the first projection point of the to-be-authenticated handle spot in the second image can be obtained by formula (1):

[0047]

[0048] Wherein q1 is the position coordinates of the first projection point, d1 is the maximum distance, p k is the actual position coordinates of the spot k, f c is a preset function for converting image position coordinates into normalized focal plane position coordinates, ​a conversion matrix between an inertial sensor and a VR camera in a preset VR device, a world position coordinate matrix of the VR device output by the inertial sensor in the VR device at time t1, an inverse matrix of the world position coordinate of the VR device output by the inertial sensor in the VR device at time t0;

[0049] and the position coordinates of the second projection point of the to-be-authenticated handle light spot in the second image can be obtained through formula (2):

[0050]

[0051] wherein q2 is the second projection position coordinates, and d2 is the shortest distance.

[0052] wherein f c () is a function in the prior art, and () is also a function in the prior art, which will not be described herein.

[0053] Step 303: obtaining the distance between the first projection point and the second projection point based on the position coordinates of the first projection point and the position coordinates of the second projection point; wherein the distance between the first projection point and the second projection point can be determined through formula (3):

[0054]

[0055] wherein d is the distance between the first projection point and the second projection point, u1 is the horizontal position coordinates of the first projection point, u2 is the horizontal position coordinates of the second projection point, v1 is the vertical position coordinates of the first projection point, and v2 is the vertical position coordinates of the second projection point.

[0056] Step 304: if the distance is less than a first specified threshold, determining the position coordinates of the first projection point as the projection position coordinates of the to-be-authenticated handle light spot in the second image.

[0057] It should be noted that the first specified threshold in the embodiments of the present application can be set according to actual conditions, and the first specified threshold is not limited herein.

[0058] Step 203: obtaining a first target region of the to-be-authenticated handle light spot in the second image according to the projection position coordinates of the to-be-authenticated handle light spot;

[0059] As shown in FIG. 2, a flowchart for determining the first target region can include the following steps: Figure 4

[0060] ​Step 401: taking a position point corresponding to a projection position coordinate of the handle light spot as a center point to obtain a first intermediate region of a specified size;

[0061] The specified size in the embodiment of the present application is 21 pixels*21 pixels, but the present application does not limit the specific value of the specified size here, and the specific value of the specified size can be set according to actual conditions. The first intermediate region in the embodiment of the present application is a square, as shown in FIG. 5, which is a schematic diagram of the first intermediate region. As can be seen from the figure, point a is the center point, and region ABCD is the first intermediate region with point a as the center point. However, the shape of the first intermediate region in the embodiment of the present application can be set according to actual conditions, and the present application does not limit the first intermediate region here. Figure 5

[0062] Step 402: performing a binaryzation processing on the first intermediate region according to a gray value of the center point to obtain a first intermediate region after binaryzation processing;

[0063] In one embodiment, step 402 can be specifically implemented as: multiplying the gray value of the center point by a specified value to obtain a binaryzation processing threshold value, setting a pixel value of each pixel point in the first intermediate region whose pixel value is less than the binaryzation processing threshold value to a first specified pixel value, setting each pixel point in the first intermediate region whose pixel value is greater than the binaryzation processing threshold value to a second specified pixel value, to obtain the first intermediate region after binaryzation processing.

[0064] It should be noted that the specified value in the embodiment of the present application is a real number, and the specified value in the embodiment of the present application is 0.8, but the present application does not limit the specified value here, and the specific value of the specified value can be set according to actual conditions. The first specified pixel value in the embodiment of the present application is 0, and the second specified pixel value is 255.

[0065] Step 403: determining a region composed of pixel points whose pixel values are greater than the second specified threshold value in the first intermediate region after binaryzation processing as the first target region.

[0066] The second specified threshold value in the embodiment of the present application is equal to the binaryzation processing threshold value described above.

[0067] Step 204: obtaining a confidence degree of the light spot through the first target region and the second target region of the to-be-authenticated handle light spot, wherein the second target region is a region corresponding to the to-be-authenticated handle light spot in the first image obtained according to an actual position coordinate of the to-be-authenticated handle light spot;

[0068] First, the determination method of the second target region in the embodiment of the present application is introduced, as shown in FIG. 6, which is a schematic diagram of the second target region. As can be seen from the figure, point a is the center point, and region ABCD is the second target region with point a as the center point. Figure 6 ​As shown, the flowchart for determining the second target region can include the following steps:

[0069] Step 601: Taking a position point corresponding to an actual position coordinate of the handle light spot to be authenticated as a center point, a second intermediate region of a specified size is obtained.

[0070] The determination manner of the second intermediate region in the embodiment of the present application is the same as that of the first intermediate region described above, and the determination manner of the second intermediate region is not described herein.

[0071] Step 602: Performing a binaryzation processing on the second intermediate region according to a gray value of the center point, to obtain a second intermediate region after binaryzation processing.

[0072] In one embodiment, step 602 can be specifically implemented as: setting a pixel value of each pixel point in the second intermediate region whose pixel value is less than the binaryzation processing threshold value as a first specified pixel value, and setting a pixel value of each pixel point in the second intermediate region whose pixel value is greater than the binaryzation processing threshold value as a second specified pixel value, to obtain the second intermediate region after binaryzation processing.

[0073] It should be noted that the specific determination manner of the binaryzation processing threshold value described above is the determination manner in step 402, and the determination manner of the binaryzation processing threshold value is not described herein.

[0074] Step 603: Determining a region composed of pixel points whose pixel value is greater than a second specified threshold value in the second intermediate region after binaryzation processing as the second target region.

[0075] The second specified threshold value in the embodiment of the present application is equal to the binaryzation processing threshold value described above.

[0076] Next, the determination manner of the confidence of the light spot in step 204 is described, and the confidence of the light spot can be determined by the following two manners:

[0077] Manner one: if the number of pixel points in the first target region is greater than a third specified threshold value, then dividing an area of a common region between the first target region and the second target region by an area of the second target region to obtain the confidence of the light spot. Specifically, the confidence of the light spot can be determined by formula (4):

[0078]

[0079] wherein μ is the confidence of the light spot, S 共 is the area of the common region, and S2 is the area of the second target region.

[0080] It should be noted that the third specified threshold in the embodiments of the present application is 4, but the embodiments of the present application do not limit the third specified threshold here, and the specific value of the third specified threshold can be set according to actual conditions.

[0081] Next, the way to determine the area of the common region between the first target region and the second target region will be introduced, as shown in the flowchart for determining the area of the common region, which can include the following steps: Figure 7

[0082] Step 701: Traverse each pixel point in the first target region in a preset order;

[0083] The preset order in the embodiments of the present application can be from left to right and from top to bottom. However, the specific preset order is not limited here, and the preset order can be limited according to actual conditions.

[0084] Step 702: For any one of the traversed pixel points, based on the position of the pixel point in the first target region, determine whether there is a target pixel point with the same position as the traversed pixel point in the second target region; if yes, execute step 703, if not, execute step 704;

[0085] Step 703: Increase the area of the common region between the first target region and the second target region by a specified value to obtain an intermediate area;

[0086] The specified value in the embodiments of the present application is 1, but the specific specified value can be set according to actual conditions, and the present application does not set the specified value here.

[0087] Step 704: Determine whether the traversal of each pixel point in the first target region is completed, if yes, execute step 705, if not, return to execute step 701;

[0088] Step 705: Determine the intermediate area as the area of the common region between the first target region and the second target region.

[0089] In one embodiment, the area of the second target region is obtained in the following way:

[0090] The number of each pixel point in the second target region is determined as the area of the second target region.

[0091] Step 205: Identify each of the to-be-authenticated handle light spots based on the confidence of each of the to-be-authenticated handle light spots to obtain a target handle light spot.

[0092] ​It should be noted that: in the embodiments of the present application, the confidence of the to-be-authenticated handle light spot greater than the specified confidence is determined as the target handle light spot. However, the specified confidence in the embodiments of the present application can be set according to actual conditions, and the present application does not limit the specified confidence.

[0093] In one embodiment, step 205 can be specifically implemented as: for any one of the to-be-authenticated handle light spots with a confidence greater than a specified confidence, based on the actual position coordinates of the to-be-authenticated handle light spot and the predicted position coordinates of the VR handle, obtaining the distance between the handle light spot and the VR handle, and if the distance is less than a specified distance, determining the to-be-authenticated handle light spot as the target handle light spot; if the distance is not less than the specified distance, ending. Wherein, the predicted position coordinates of the VR handle are obtained based on the actual position of the VR handle in the short exposure image corresponding to the last time.

[0094] The predicted position of the VR handle in the embodiments of the present application is obtained by inputting the actual position of the VR handle in the short exposure image corresponding to the last time and the IMU data in the current frame short exposure image into the pose estimation algorithm. However, the pose estimation algorithm in the embodiments of the present application can be set according to actual conditions, and the present application does not limit the pose estimation algorithm. The specified distance in the embodiments of the present application is 5 pixels, but the present application does not limit the specified distance, and the specific value of the specified distance can be set according to actual conditions.

[0095] Method two: if the number of pixel points in the first target area is not greater than the third specified threshold, the fourth specified threshold is determined as the confidence of the light spot.

[0096] It should be noted that: the fourth specified threshold in the embodiments of the present application is 0, but the present application does not limit the fourth specified threshold, and the specific value of the fourth specified threshold in the embodiments of the present application can be limited according to actual conditions.

[0097] In order to further connect the method in the present application, as shown in Figure 8 The flowchart of the generation method of the hand data set in the present application can include the following steps:

[0098] Step 801: for any one time, using a light spot recognition algorithm to recognize the handle light spot in a first image corresponding to the time, to obtain the actual position coordinates of each to-be-authenticated handle light spot in the first image; wherein the first image is a short exposure image containing a VR handle, and the LED light in the VR handle is in an open state;

[0099] Step 802: Obtain a preset spot distance range, wherein the spot distance range includes a maximum distance and a minimum distance between a camera in the VR device and a spot;

[0100] Step 803: Obtain a position coordinate of a first projection point of the to-be-authenticated handle spot in the second image according to the maximum distance and the actual position coordinate, wherein the second image is a long-exposure image including the VR handle and the LED light in the VR handle is in an off state;

[0101] Step 804: Obtain a position coordinate of a second projection point of the to-be-authenticated handle spot in the second image according to the minimum distance and the actual position coordinate;

[0102] Step 805: Obtain a first intermediate region of a specified size with a position point corresponding to a projection position coordinate of the handle spot as a center point;

[0103] Step 806: Perform a binaryzation processing on the first intermediate region according to a gray value of the center point to obtain a binaryzation-processed first intermediate region;

[0104] Step 807: Determine a region composed of pixel points with a pixel value greater than a second specified threshold in the binaryzation-processed first intermediate region as the first target region;

[0105] Step 808: Determine whether a number of pixel points in the first target region is greater than a third specified threshold, if yes, execute Step 809, and if no, execute Step 810;

[0106] Step 809: Divide an area of a common region between the first target region and a second target region by an area of the second target region to obtain a confidence degree of the spot, wherein the second target region is a region corresponding to the to-be-authenticated handle spot in the first image obtained according to an actual position coordinate of the to-be-authenticated handle spot;

[0107] Step 810: Determine a fourth specified threshold as the confidence degree of the spot;

[0108] Step 811: Identify the to-be-authenticated handle spots based on the confidence degrees of the to-be-authenticated handle spots to obtain a target handle spot.

[0109] Based on the same inventive concept, the VR handle spot identification method as described above can also be implemented by a VR handle spot identification device. The VR handle spot identification device has similar effects to the foregoing method, and thus will not be described here again.

[0110] Figure 9A structural schematic diagram of a spot recognition device of a VR handle according to one embodiment of the present disclosure.

[0111] As shown in Figure 9 The spot recognition device 900 of the VR handle of the present disclosure can include a spot recognition module 910, a projection module 920, a first target area determination module 930, a confidence determination module 940, and a target handle spot determination module 950.

[0112] The spot recognition module 910 is configured to, for any one time, perform handle spot recognition on a first image corresponding to the time by using a spot recognition algorithm, to obtain actual position coordinates of each to-be-authenticated handle spot in the first image; wherein the first image is a short-exposure image containing a VR handle and LED lights in the VR handle are in an open state; and

[0113] The projection module 920 is configured to, for any one to-be-authenticated handle spot, project the to-be-authenticated handle spot in a second image corresponding to the time based on actual position coordinates of the to-be-authenticated handle spot, to obtain projection position coordinates of the to-be-authenticated handle spot in the second image, wherein the second image is a long-exposure image including the VR handle and the LED lights in the VR handle are in a closed state; and

[0114] The first target area determination module 930 is configured to obtain a first target area of the to-be-authenticated handle spot in the second image according to the projection position coordinates of the to-be-authenticated handle spot; and

[0115] The confidence determination module 940 is configured to obtain a confidence of the spot by using a first target area and a second target area of the to-be-authenticated handle spot, wherein the second target area is a corresponding area of the to-be-authenticated handle spot in the first image obtained according to the actual position coordinates of the to-be-authenticated handle spot;

[0116] The target handle spot determination module 950 is configured to identify the to-be-authenticated handle spots based on the confidence of each to-be-authenticated handle spot, to obtain a target handle spot.

[0117] In one embodiment, the projection module 920 is specifically configured to:

[0118] obtain a pre-set spot distance range, wherein the spot distance range includes a maximum distance and a minimum distance between a camera in a VR device and a spot;

[0119] According to the maximum distance and the actual position coordinate, a position coordinate of a first projection point of the handle light spot to be authenticated in the second image is obtained, and according to the minimum distance and the actual position coordinate, a position coordinate of a second projection point of the handle light spot to be authenticated in the second image is obtained;

[0120] Based on the position coordinate of the first projection point and the position coordinate of the second projection point, a distance between the first projection point and the second projection point is obtained.

[0121] If the distance is less than a first specified threshold, the position coordinate of the first projection point is determined as a projection position coordinate of the handle light spot to be authenticated in the second image.

[0122] In one embodiment, the projection module 920 is further configured to:

[0123] The position coordinate of the first projection point is obtained by the following formula:

[0124]

[0125] wherein q1 is the position coordinate of the first projection point, d1 is the maximum distance, p k is an actual position coordinate of the light spot k, f c is a preset function of converting an image position coordinate into a normalized focal plane position coordinate, is a preset conversion matrix between an inertial sensor in the VR device and a VR camera, is a world position coordinate matrix of the inertial sensor in the VR device at the time t1;

[0126] The position coordinate of the second projection point is obtained by the following formula:

[0127]

[0128] wherein q2 is the second projection position coordinate, and d2 is the minimum distance.

[0129] In one embodiment, the first target region determination module 930 is specifically configured to:

[0130] A position point corresponding to the projection position coordinate of the handle light spot is taken as a center point to obtain a first intermediate region of a specified size;

[0131] The first intermediate region is subjected to a binaryzation processing according to a gray value of the center point to obtain a binaryzation-processed first intermediate region;

[0132] The region composed of the pixel points with pixel values greater than the second specified threshold in the first intermediate region after the binarization processing is determined as the first target region.

[0133] In one embodiment, the apparatus further includes:

[0134] The second target region determination module 960 is configured to obtain the second target region in the following manner:

[0135] The second intermediate region of a specified size is obtained with the position point corresponding to the actual position coordinate of the handle light spot to be authenticated as the center point.

[0136] The second intermediate region is subjected to binarization processing according to the gray value of the center point, to obtain a second intermediate region after binarization processing.

[0137] The region composed of the pixel points with pixel values greater than the second specified threshold in the second intermediate region after the binarization processing is determined as the second target region.

[0138] In one embodiment, the confidence determination module 940 is specifically configured to:

[0139] If the number of pixel points in the first target region is greater than a third specified threshold, the area of the common region between the first target region and the second target region is divided by the area of the second target region to obtain the confidence of the light spot; or,

[0140] If the number of pixel points in the first target region is not greater than the third specified threshold, a fourth specified threshold is determined as the confidence of the light spot.

[0141] In one embodiment, the apparatus further includes:

[0142] The common region determination module 970 is configured to obtain the area of the common region in the following manner:

[0143] The first target region is traversed according to a preset order; for any pixel point traversed, whether there is a target pixel point with the same position as the traversed pixel point in the second target region is determined based on the position of the pixel point in the first target region; if there is, the area of the common region between the first target region and the second target region is increased by a specified value, and then the step of traversing each pixel point in the first target region according to a preset order is returned until the traversal of each pixel point in the first target region is completed, and the area of the common region is obtained; if there is not, the step of traversing each pixel point in the first target region according to a preset order is returned until the traversal of each pixel point in the first target region is completed, and the area of the common region is obtained.

[0144] The area determination module 980 is configured to obtain the area of the second target region by the following method:

[0145] The number of each pixel point in the second target region is determined as the area of the second target region.

[0146] In one embodiment, the target handle light spot determination module 950 is specifically configured to:

[0147] For any one of the to-be-authenticated handle light spots with a confidence greater than a specified confidence, a distance between the to-be-authenticated handle light spot and the VR handle is obtained based on the actual position coordinates of the to-be-authenticated handle light spot and the predicted position coordinates of the VR handle, wherein the predicted position coordinates of the VR handle are obtained based on the actual position of the VR handle in the short-exposure image corresponding to the last time; and

[0148] If the distance is less than a specified distance, the to-be-authenticated handle light spot is determined as the target handle light spot.

[0149] After introducing the light spot recognition method and device of the VR handle according to an example embodiment of the present application, next, a VR device according to another example embodiment of the present application is introduced.

[0150] Those skilled in the art can understand that each aspect of the present application can be implemented as a system, a method or a program product. Therefore, each aspect of the present application can be specifically implemented as follows: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system".

[0151] In some possible implementation manners, the VR device according to the present application can at least include at least one processor and at least one computer storage medium. The computer storage medium stores program codes, when the program codes are executed by the processor, the processor executes the steps in the light spot identification method of the VR handle according to various exemplary embodiments of the present application described above in the specification. For example, the processor can execute the steps 201-205 as shown in Figure 2

[0152] The VR device 1000 according to this implementation manner of the present application will be described below with reference to Figure 10 Figure 10 The VR device 1000 shown is only an example, and should not bring any limitation to the function and use range of the embodiments of the present application.

[0153] As shown in Figure 10 , the VR device 1000 is in the form of a general VR device. The components of the VR device 1000 can include but are not limited to the at least one processor 1001 described above, the at least one computer storage medium 1002 described above, and the bus 1003 connecting different system components including the computer storage medium 1002 and the processor 1001.

[0154] The bus 1003 represents one or more of several types of bus structures, including a computer storage medium bus or a computer storage medium controller, a peripheral bus, a processor bus, or a local bus using any of a variety of bus structures.

[0155] The computer storage medium 1002 can include readable media in the form of volatile computer storage medium, such as random access computer storage medium (RAM) 1021 and / or cache storage medium 1022, and can further include read-only computer storage medium (ROM) 1023.

[0156] The computer storage medium 1002 can further include programs / utilities 1025 with a set of (at least one) program modules 1024, such as but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof can include the implementation of a network environment.

[0157] ​​The VR device 1000 can also communicate with one or more external devices 1004 such as a keyboard, a pointing device, etc.; and / or one or more devices that enable a user to interact with the VR device 1000 and / or one or more devices that enable the VR device 1000 to communicate with one or more other VR devices. Such communication can occur via Input / Output (I / O) interface 1005. Still yet, the VR device 1000 can communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or the Internet) through a network adapter 1006. As depicted, the network adapter 1006 communicates with the other components of the VR device 1000 via bus 1003. It should be appreciated that although not shown, other hardware and / or software modules could be used in connection with the VR device 1000. Such modules include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0158] In some possible implementation, each aspect of the method for identifying a light spot of a VR handle provided by the present application can also be implemented in the form of a program product, which includes program codes for causing a computer device to execute the steps of the method for identifying a light spot of a VR handle according to various exemplary embodiments of the present application described above in the specification when the program product is run on the computer device.

[0159] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A method for spot recognition of a VR handle, characterized in that, The method comprises: For any one time, a handle light spot recognition algorithm is used to recognize a first image corresponding to the time to obtain actual position coordinates of each to-be-authenticated handle light spot in the first image; wherein the first image is a short-exposure image containing a VR handle and LED lights in the VR handle are in an open state; and For any one to-be-authenticated handle light spot, the to-be-authenticated handle light spot is projected in a second image corresponding to the time based on actual position coordinates of the to-be-authenticated handle light spot to obtain projection position coordinates of the to-be-authenticated handle light spot in the second image, wherein the second image is a long-exposure image including the VR handle and the LED lights in the VR handle are in a closed state; and A first target region of the to-be-authenticated handle light spot in the second image is obtained according to the projection position coordinates of the to-be-authenticated handle light spot; and A confidence degree of the light spot is obtained through the first target region and a second target region of the to-be-authenticated handle light spot, wherein the second target region is a region corresponding to the to-be-authenticated handle light spot in the first image obtained according to the actual position coordinates of the to-be-authenticated handle light spot; Each to-be-authenticated handle light spot is identified based on the confidence degrees of the to-be-authenticated handle light spots to obtain a target handle light spot.

2. The method of claim 1, wherein, The method comprises: A preset light spot distance range is obtained, wherein the light spot distance range includes a maximum distance and a minimum distance between a camera in a VR device and a light spot; A position coordinate of a first projection point of the to-be-authenticated handle light spot in the second image is obtained according to the maximum distance and the actual position coordinates, and a position coordinate of a second projection point of the to-be-authenticated handle light spot in the second image is obtained according to the minimum distance and the actual position coordinates; A distance between the first projection point and the second projection point is obtained based on the position coordinate of the first projection point and the position coordinate of the second projection point; If the distance is less than a first specified threshold, the position coordinate of the first projection point is determined as the projection position coordinate of the to-be-authenticated handle light spot in the second image.

3. The method of claim 2, wherein, The method comprises: The position coordinate of the first projection point is obtained through the following formula: ； wherein, is a position coordinate of the first projection point, is the maximum distance, is an actual position coordinate of the light spot k, is a preset function of converting the image position coordinate into a normalized focal plane position coordinate, is a preset conversion matrix between the inertial sensor and the VR camera in the VR device, is a preset conversion matrix between the inertial sensor and the VR camera in the VR device, is a world position coordinate matrix of the VR device output by the inertial sensor in the VR device at the time t1, is a world position coordinate matrix of the VR device output by the inertial sensor in the VR device at the time t1, is an inverse matrix of the world position coordinate of the VR device output by the inertial sensor in the VR device at the time t1. The method comprises: The position coordinate of the second projection point is obtained through the following formula: ； wherein, is the position coordinate of the second projection point, is the minimum distance.

4. The method of claim 1, wherein, The method comprises: The first target region of the to-be-authenticated handle light spot in the second image is obtained according to the projection position coordinates of the to-be-authenticated handle light spot. A position point corresponding to a projection position coordinate of the handle light spot is taken as a center point to obtain a first intermediate region of a specified size; The first intermediate region is binarized according to a gray value of the center point to obtain a binarized first intermediate region; A region formed by pixel points with pixel values greater than a second specified threshold in the binarized first intermediate region is determined as the first target region.

5. The method of claim 1, wherein, The second target region is obtained in the following manner: A position point corresponding to an actual position coordinate of the to-be-authenticated handle light spot is taken as a center point to obtain a second intermediate region of a specified size; The second intermediate region is binarized according to a gray value of the center point to obtain a binarized second intermediate region; A region formed by pixel points with pixel values greater than a second specified threshold in the binarized second intermediate region is determined as the second target region.

6. The method of claim 1, wherein, The confidence of the light spot is obtained from the first target region and the second target region of the to-be-authenticated handle light spot, including: If the number of pixel points in the first target region is greater than a third specified threshold, the area of a common region between the first target region and the second target region is divided by the area of the second target region to obtain the confidence of the light spot; or, If the number of pixel points in the first target region is not greater than the third specified threshold, a fourth specified threshold is determined as the confidence of the light spot.

7. The method of claim 6, wherein, The area of the common region is obtained in the following manner: Each pixel point in the first target region is traversed in a preset order; for any one traversed pixel point, it is determined whether there is a target pixel point with the same position as the traversed pixel point in the second target region based on the position of the pixel point in the first target region; If there is, the area of the common region between the first target region and the second target region is increased by a specified value, and then the step of traversing each pixel point in the first target region in a preset order is returned until the traversal of each pixel point in the first target region is completed, and the area of the common region is obtained; if not, the step of traversing each pixel point in the first target region in a preset order is returned until the traversal of each pixel point in the first target region is completed, and the area of the common region is obtained; The area of the second target region is obtained in the following manner: The number of pixel points in the second target region is determined as the area of the second target region.

8. The method of claim 1, wherein, The to-be-authenticated handle light spots are identified based on the confidence of each to-be-authenticated handle light spot to obtain a target handle light spot, including: For any one of the to-be-authenticated handle light spots with a confidence greater than a specified confidence, based on the actual position coordinates of the to-be-authenticated handle light spot and the predicted position coordinates of the VR handle, a distance between the handle light spot and the VR handle is obtained, wherein the predicted position coordinates of the VR handle are obtained based on the actual position of the VR handle in a short-exposure image corresponding to a previous time; and If the distance is less than a specified distance, the to-be-authenticated handle light spot is determined as the target handle light spot.

9. A VR device, comprising: The device comprises a processor and a memory, which are connected through a bus; The memory stores a computer program, and the processor is configured to execute the following operations based on the computer program: For any one time, a handle light spot identification is performed on a first image corresponding to the time by using a light spot identification algorithm to obtain actual position coordinates of each to-be-authenticated handle light spot in the first image; wherein the first image is a short-exposure image containing a VR handle, and the LED light in the VR handle is in an open state; and For any one to-be-authenticated handle light spot, the to-be-authenticated handle light spot is projected in a second image corresponding to the time based on the actual position coordinates of the to-be-authenticated handle light spot to obtain projection position coordinates of the to-be-authenticated handle light spot in the second image, wherein the second image is a long-exposure image containing the VR handle, and the LED light in the VR handle is in a closed state; and According to the projection position coordinates of the to-be-authenticated handle light spot, a first target region of the to-be-authenticated handle light spot in the second image is obtained; and The confidence of the light spot is obtained through the first target region and a second target region of the to-be-authenticated handle light spot, wherein the second target region is a region corresponding to the to-be-authenticated handle light spot in the first image obtained according to the actual position coordinates of the to-be-authenticated handle light spot; Based on the confidence of each to-be-authenticated handle light spot, the to-be-authenticated handle light spots are identified to obtain a target handle light spot.

10. The VR device of claim 9, wherein, The processing of projecting the to-be-authenticated handle light spot in the second image based on the actual position coordinates of the to-be-authenticated handle light spot to obtain the projection position coordinates of the to-be-authenticated handle light spot in the second image is specifically configured as: A preset light spot distance range is obtained, wherein the light spot distance range includes a maximum distance and a minimum distance between a camera in a VR device and a light spot; According to the maximum distance and the actual position coordinates, a position coordinate of a first projection point of the to-be-authenticated handle light spot in the second image is obtained, and according to the minimum distance and the actual position coordinates, a position coordinate of a second projection point of the to-be-authenticated handle light spot in the second image is obtained; Based on the position coordinates of the first projection point and the second projection point, a distance between the first projection point and the second projection point is obtained; If the distance is less than a first specified threshold, then the position coordinates of the first projected point are determined as the projected position coordinates of the handle light spot in the second image.

Images