Vr handle positioning method and apparatus, electronic device, and readable storage medium

Abstract

The application discloses a VR handle positioning method and device, electronic equipment and a readable storage medium. The VR handle positioning method comprises the following steps: acquiring an infrared image of a VR handle, performing image processing on the infrared image, and obtaining the center of each light spot; based on the contour size of each light spot center, filtering a first number of light spots from each light spot center, and acquiring the image coordinate value of the first number of light spots; acquiring the actual environment coordinate value corresponding to a second number of light spots on the VR handle, and matching the one-to-one correspondence between the second number of light spots and the first number of light spots according to the image coordinate value and the actual environment coordinate value, wherein the second number of light spots includes the first number of light spots; and positioning the VR handle according to the correspondence, and obtaining the positioning information of the VR handle. The application solves the technical problem of low real-time performance of VR handle positioning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of VR (Virtual Reality), in particular to a VR handle positioning method and device, electronic equipment and a readable storage medium. BACKGROUND

[0002] In the existing VR device, the VR handle is still indispensable hardware, and the key to the successful use of the handle lies in whether the positioning of the VR handle in the VR space is accurate. Therefore, correctly and quickly tracking the handle is one of the core research contents in the VR field. The main means for positioning the VR handle at present includes positioning the VR handle by infrared light based on the input infrared image containing the VR handle. The advantage of the current infrared light positioning method is that it can ensure high accuracy, low cost and strong anti-interference ability, but the algorithm involved is difficult, and in the infrared light positioning algorithm, usually more than 8 points are involved in calculation, and then brute force matching is performed, which has large algorithm calculation amount and is difficult to ensure real-time, thereby causing low real-time of the VR handle positioning. SUMMARY

[0003] The main purpose of the present application is to provide a VR handle positioning method, device, electronic equipment and readable storage medium, which aims to solve the technical problem of low real-time of VR handle positioning.

[0004] To achieve the above purpose, the present application provides a VR handle positioning method applied to a VR device, which comprises:

[0005] acquiring an infrared image of a VR handle, performing image processing on the infrared image, and obtaining light point centers;

[0006] Based on the contour size of each light point center, a first number of light points are selected from each light point center, and image coordinate values of the first number of light points are obtained;

[0007] Obtaining actual environment coordinate values corresponding to a second number of light points on the VR handle, and matching a one-to-one correspondence relationship between the second number of light points and the first number of light points according to the image coordinate values and the actual environment coordinate values, wherein the second number of light points includes the first number of light points;

[0008] According to the corresponding relationship, the VR handle is positioned to obtain the positioning information of the VR handle.

[0009] Optionally, the step of performing image processing on the infrared image to obtain the light point center comprises:

[0010] contour extraction is performed on each light spot in the infrared image to obtain a contour corresponding to each light spot;

[0011] elliptical fitting is performed on the contour of each light spot to obtain a target contour shape of each light spot;

[0012] geometric centers of the contour shapes are extracted to obtain centers of the light spots.

[0013] Optionally, the step of positioning the VR handle according to the correspondence relationship to obtain positioning information of the VR handle comprises:

[0014] The light spots are sorted according to image coordinate values of the light spots to obtain a sorting relationship between the light spots;

[0015] A one-to-one correspondence relationship between the image coordinate values and actual environment coordinate values of the light spots is obtained, and a VR handle pose of each frame is determined according to the correspondence relationship;

[0016] The VR handle is pose predicted according to the VR handle poses of the previous two frames corresponding to the current frame to obtain a current pose;

[0017] It is judged whether the current pose is matched successfully according to the image coordinate values and actual environment coordinate values of the light spots in the sorting relationship, camera intrinsic parameters, and the current pose;

[0018] If the matching is successful, the current pose is set as the VR handle positioning information corresponding to the current frame, and pose prediction is continuously performed in subsequent frames to obtain the positioning information.

[0019] Optionally, the step of sorting the light spots according to the image coordinate values of the light spots to obtain a sorting relationship between the light spots comprises:

[0020] An external rectangle corresponding to each light spot is determined according to the image coordinate values of the light spots;

[0021] The external rectangle and the light spots are rotated to a preset direction, wherein the preset direction is a horizontal direction or a vertical direction;

[0022] When the external rectangle and the light spots have been rotated to the preset direction, coordinates of the light spots are extracted;

[0023] A sorting direction is determined according to a difference between the coordinates of the light spots in the horizontal direction and the vertical direction, wherein the sorting direction is the horizontal direction or the vertical direction;

[0024] A sorting relationship between the light spots is determined according to the sorting direction and the coordinates of the light spots.

[0025] Optionally, the step of judging whether the current pose is matched successfully according to the image coordinate values of the light points in the sorting relationship, the actual environment coordinate values, the camera intrinsic parameters and the current pose comprises:

[0026] determining a re-projection error according to the image coordinate values of the light points in the sorting relationship, the actual environment coordinate values, the camera intrinsic parameters and the current pose;

[0027] if the re-projection error is not greater than a preset threshold, determining that the current pose is matched successfully;

[0028] if the re-projection error is greater than the preset threshold, performing enumeration matching and positioning on the first number of light points and the second number of light points to obtain the positioning information.

[0029] Optionally, the step of performing enumeration matching and positioning on the first number of light points and the second number of light points to obtain the positioning information comprises:

[0030] performing enumeration matching on the first number of light points and the second number of light points to obtain each matching combination;

[0031] positioning the VR handle according to each matching combination to obtain a VR handle predicted pose corresponding to each matching combination;

[0032] evaluating positioning accuracy of each VR handle predicted pose according to the image coordinate values of the light points in the sorting relationship, the actual environment coordinate values, the camera intrinsic parameters and the current pose;

[0033] extracting the positioning information from each VR handle predicted pose based on the positioning accuracy of each VR handle predicted pose.

[0034] Optionally, the VR handle is positioned according to the corresponding relationship to obtain positioning information of the VR handle:

[0035] sorting each light point according to the image coordinate values of the light points to obtain a sorting relationship between the light points;

[0036] performing enumeration matching on the first number of light points and the second number of light points to obtain each matching combination;

[0037] positioning the VR handle according to each matching combination to obtain a VR handle predicted pose corresponding to each matching combination;

[0038] According to image coordinate values of each of the light points in the sorting relationship, actual environment coordinate values, camera intrinsic parameters, and the VR handle predicted poses, positioning accuracy of each of the VR handle predicted poses is evaluated;

[0039] Based on the positioning accuracy of each of the VR handle predicted poses, the positioning information is extracted from each of the VR handle predicted poses.

[0040] The application also provides a VR handle positioning device, which is applied to a VR handle positioning apparatus, and includes:

[0041] A light point extraction module is configured to acquire an infrared image of a VR handle, perform image processing on the infrared image, and obtain light point centers;

[0042] A light point screening module is configured to screen a first number of light points from the light point centers based on contour sizes of the light point centers, and acquire image coordinate values of the first number of light points;

[0043] A light point corresponding module is configured to acquire actual environment coordinate values corresponding to a second number of light points on the VR handle, and match a one-to-one correspondence between the second number of light points and the first number of light points according to the image coordinate values and the actual environment coordinate values, wherein the second number of light points includes the first number of light points;

[0044] A handle positioning module is configured to position the VR handle according to the correspondence, and obtain positioning information of the VR handle.

[0045] Optionally, the light point extraction module is further configured to:

[0046] Contour extraction is performed on each light point in the infrared image to obtain a contour corresponding to each of the light points;

[0047] Elliptical fitting is performed on the contour of each of the light points to obtain a target contour shape of each of the light points;

[0048] Geometric centers of each of the contour shapes are extracted to obtain the light point centers.

[0049] Optionally, the handle positioning module is further configured to:

[0050] Each of the light points is sorted according to image coordinate values of each of the light points to obtain a sorting relationship between each of the light points;

[0051] A one-to-one correspondence between the image coordinate values of each of the light points and actual environment coordinate values is acquired, and VR handle poses of each frame are determined according to the correspondence;

[0052] According to the VR handle poses corresponding to the previous two frames of the current frame, pose prediction is performed on the VR handle to obtain a current pose;

[0053] According to the image coordinate values and actual environment coordinate values of the light points in the sorting relationship, the camera intrinsic parameters, and the current pose, it is determined whether the current pose is matched successfully;

[0054] If the matching is successful, the current pose is set as the VR handle positioning information corresponding to the current frame, and pose prediction is continuously performed on the subsequent frames to obtain the positioning information.

[0055] Optionally, the handle positioning module is further configured to:

[0056] According to the image coordinate values of the light points, the circumscribed rectangles corresponding to the light points are determined;

[0057] The circumscribed rectangles and the light points are rotated to a preset direction, wherein the preset direction is a horizontal direction or a vertical direction;

[0058] When the circumscribed rectangles and the light points have been rotated to the preset direction, the coordinates of the light points are extracted;

[0059] According to the difference values of the coordinates of the light points in the horizontal direction and the vertical direction, a sorting direction is determined, wherein the sorting direction is a horizontal direction or a vertical direction;

[0060] According to the sorting direction and the coordinates of the light points, a sorting relationship between the light points is determined.

[0061] Optionally, the handle positioning module is further configured to:

[0062] According to the image coordinate values and actual environment coordinate values of the light points in the sorting relationship, the camera intrinsic parameters, and the current pose, a re-projection error is determined;

[0063] If the re-projection error is not greater than a preset threshold, it is determined that the current pose is matched successfully;

[0064] If the re-projection error is greater than the preset threshold, the first number of light points and the second number of light points are enumerated and matched to obtain the positioning information.

[0065] Optionally, the handle positioning module is further configured to:

[0066] The first number of light points and the second number of light points are enumerated and matched to obtain each matching combination;

[0067] According to each matching combination, the VR handle is positioned to obtain a VR handle predicted pose corresponding to each matching combination;

[0068] evaluate positioning accuracy of each of the VR handle predicted poses according to the image coordinate values of each of the light points in the sorting relationship, actual environment coordinate values, the camera intrinsic parameters and the VR handle predicted poses;

[0069] extract the positioning information from each of the VR handle predicted poses based on the positioning accuracy of each of the VR handle predicted poses.

[0070] Optionally, the handle positioning module is further configured to:

[0071] sort each of the light points according to the image coordinate values of each of the light points to obtain a sorting relationship between each of the light points;

[0072] perform enumeration matching between the first number of light points and the second number of light points to obtain each matching combination;

[0073] perform positioning of the VR handle according to each of the matching combinations to obtain a VR handle predicted pose corresponding to each of the matching combinations;

[0074] evaluate positioning accuracy of each of the VR handle predicted poses according to the image coordinate values of each of the light points in the sorting relationship, actual environment coordinate values, the camera intrinsic parameters and the VR handle predicted poses;

[0075] extract the positioning information from each of the VR handle predicted poses based on the positioning accuracy of each of the VR handle predicted poses.

[0076] The application further provides an electronic device, which is a physical device, and the electronic device comprises a memory, a processor and a program of the VR handle positioning method stored in the memory and executable on the processor, and the program of the VR handle positioning method can implement the steps of the VR handle positioning method when executed by the processor.

[0077] The application further provides a computer readable storage medium, which stores a program of a VR handle positioning method, and the program of the VR handle positioning method can implement the steps of the VR handle positioning method when executed by a processor.

[0078] The application further provides a computer program product, which comprises a computer program, and the computer program can implement the steps of the VR handle positioning method when executed by a processor.

[0079] The application provides a VR handle positioning method and device, electronic equipment and a readable storage medium. First, an infrared image of a VR handle is acquired, image processing is performed on the infrared image, the centers of each light point are obtained, then a first number of light points are selected from each light point center based on the contour size of each light point center, image coordinate values of the first number of light points are acquired, actual environment coordinate values corresponding to a second number of light points on the VR handle are acquired, and a one-to-one correspondence between the second number of light points and the first number of light points is matched according to the image coordinate values and the actual environment coordinate values, wherein the second number of light points contains the first number of light points, finally, the VR handle is positioned according to the correspondence, and positioning information of the VR handle is obtained. The application selects light points on the VR handle based on the contour size corresponding to each light point center, obtains a first number of light points, and calculates the positioning result of the VR handle based on the correspondence between the second number of light points and the first number of light points, wherein the first number can be set according to requirements, the positioning of the VR handle is realized by fewer point positions, the calculation amount is reduced, the positioning efficiency is improved, the technical defect of large calculation amount caused by brute force matching in the prior art is overcome, and the real-time performance of VR handle positioning is improved. BRIEF DESCRIPTION OF DRAWINGS

[0080] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate an embodiment consistent with the present application and, together with the description, serve to explain the principles of the application.

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings required to be used in the embodiments or prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0082] Figure 1 Flowchart of the first embodiment of the VR handle positioning method of the present application;

[0083] Figure 2 Infrared image corresponding to the VR handle in the first embodiment of the VR handle positioning method of the present application;

[0084] Figure 3 Each light point image that has completed sorting in the first embodiment of the VR handle positioning method of the present application;

[0085] Figure 4 Structure composition schematic diagram of the VR handle positioning device of the present application;

[0086] Figure 5A device structure schematic diagram of a hardware running environment involved in a VR handle positioning method in embodiments of the present application.

[0087] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments in combination with the accompanying drawings. DETAILED DESCRIPTION

[0088] In order to make the above-mentioned objects, features and advantages of the present application more apparent and easy to understand, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0089] Embodiment one

[0090] In today's rapidly changing VR technology, VR handle is still an indispensable part of VR equipment, and the key to successfully using the handle to assist VR display is whether the positioning of the VR handle in the VR space is accurate. Therefore, correct and rapid handle tracking is one of the core research contents in the VR field. The main means for positioning the VR handle at present includes infrared positioning of the VR handle based on input infrared images containing the VR handle. The advantage of the current infrared positioning method is that it can ensure high accuracy, low cost and strong anti-interference ability, but the algorithm involved is difficult, and usually more than 8 points are involved in the calculation of the infrared positioning algorithm. Then, the algorithm calculation amount is large, and it is difficult to ensure real-time, thereby leading to low real-time of VR handle positioning.

[0091] The present application provides a VR handle positioning method. In the first embodiment of the VR handle positioning method of the present application, referring to Figure 1 , the VR handle positioning method comprises:

[0092] Step S10, acquiring an infrared image of a VR handle, performing image processing on the infrared image to obtain a center of each light point;

[0093] Step S20, based on the contour size of each light point center, screening a first number of light points from each light point center, and acquiring image coordinate values of the first number of light points;

[0094] Step S30, acquiring actual environment coordinate values corresponding to a second number of light points on the VR handle, and matching a one-to-one correspondence relationship between the second number of light points and the first number of light points according to the image coordinate values and the actual environment coordinate values, wherein the second number of light points includes the first number of light points.

[0095] In step S40, the VR handle is positioned according to the correspondence relationship, and positioning information of the VR handle is obtained.

[0096] In the embodiments of the present application, it should be noted that, with reference to Figure 2 , the infrared image includes a VR handle with infrared LED (light-emitting diode) lights, and the target contour can be an ellipse, a circle or other polygons; the image processing process is a process of extracting the light point centers of each infrared LED light, which includes image binarization, Gaussian blur, image erosion, light point contour extraction and ellipse fitting; the light points are the LED light points on the VR handle, wherein the coordinate points of the light points in the image coordinate system can be understood as 2D light points, and the coordinate points of the light points in the actual environment can be understood as 3D light points, wherein the coordinate axes of the 3D light points are preset, so the coordinates of the 3D light points are known and can be directly obtained (since the coordinate axes are on the handle, the coordinate values of the handle do not change no matter how the handle moves); the first number is the number of selected 2D light points, and compared with the prior art which usually needs more than 8 points to participate in VR handle positioning, the first number can be less than 8, and as a preferred embodiment, the first number is 5, so as to realize VR handle positioning with the least number of light points, reduce the calculation amount and improve the positioning efficiency; the second number is the actual number of light points on the VR handle, which is different due to different specifications of the VR handle, and the second number is not less than the first number, and the second number of light points must include the first number of 2D light points.

[0097] As an example, the correspondence relationship between each of the 2D light points and each of the 3D light points on the VR handle can be only one kind of correspondence relationship, or a combination of multiple kinds of correspondence relationships, for example, the correspondence relationship between the 2D light point p1 and the 3D light point P M , and the matching combination of the 3D light points corresponding to the other 2D light points p2, p3, p4 and p5 can be [P M+1 , P M+2 , P M+3 , P M+4 , P M+2 , P M+3 , P M+4 , P M+6 and [P M+2 , P M+4 , P M+6 , P M+8 , etc.

[0098] As a preferred embodiment, the first number is at least less than 5, and specifically can be 5.

[0099] As an example, steps S10 to S40 include: acquiring an infrared image corresponding to a VR handle containing an infrared LED lamp; sequentially performing image binarization, Gaussian blur and image erosion on the infrared image to obtain a first infrared image; performing contour extraction on light points in the first infrared image to obtain a contour corresponding to each light point; extracting a light point center of a target ellipse corresponding to each light point by performing ellipse fitting on each contour; screening 5 largest contours based on the size of the contour in the first infrared image corresponding to each light point center to obtain image coordinate values of each light point; acquiring actual environment coordinate values of a second number of light points on the VR handle, matching a one-to-one correspondence between the second number of light points and the first number of light points according to the image coordinate values and the actual environment coordinate values, and obtaining a pose of the VR handle; and positioning the VR handle based on each pose to obtain VR handle positioning information.

[0100] The step of performing image processing on the infrared image to obtain each light point center includes:

[0101] Step S11, performing contour extraction on each light point in the infrared image to obtain a contour corresponding to each light point.

[0102] Step S12, performing ellipse fitting on the contour of each light point respectively to obtain each target contour shape.

[0103] Step S13, extracting a geometric center of each contour shape to obtain each light point center.

[0104] In the embodiments of the present application, it should be noted that the contour corresponding to each light point includes a contour line and a contour area, and the contour area is used to screen a first number of 2D light points from each light point center.

[0105] As an example, steps S11 to S13 include: sequentially performing image binarization, Gaussian blur and image erosion on the infrared image to obtain a first infrared image; performing contour extraction on light points in the first infrared image to obtain a contour line and a contour area corresponding to each light point; performing ellipse fitting on each contour based on the contour line corresponding to each light point to obtain each target contour shape; and extracting a geometric center of each target contour shape to obtain each light point center.

[0106] The step of positioning the VR handle according to the correspondence to obtain positioning information of the VR handle includes:

[0107] Step S41, sorting each light point according to the image coordinate values of each light point to obtain a sorting relationship between each light point.

[0108] Step S42, a one-to-one correspondence between the image coordinate value of each light point and the actual environment coordinate value is obtained, and the VR handle pose of each frame is determined according to the correspondence;

[0109] Step S43, the pose of the VR handle is predicted according to the corresponding VR handle poses of the previous two frames of the current frame, to obtain a current pose;

[0110] Step S44, whether the current pose is matched successfully is judged according to the image coordinate value and the actual environment coordinate value of each light point in the sorting relationship, the camera intrinsic parameter and the current pose;

[0111] Step S45, if the matching is successful, the current pose is set as the VR handle positioning information corresponding to the current frame, and the pose prediction is continued in the subsequent frame to obtain the positioning information.

[0112] In the embodiments of the present application, it should be noted that when the one-to-one correspondence between the image coordinate value of each light point and the actual environment coordinate value can be obtained, the VR handle can be positioned by steps S41 to S45 to obtain the positioning information, so as to realize infrared light positioning by fewer points and faster method, and the technical defects that the algorithm calculation amount is large and it is difficult to ensure real-time in the prior art infrared light positioning algorithm in which more than 8 points are usually involved in calculation and then matched by force can be overcome.

[0113] As an example, steps S41 to S45 include: sorting each light point according to the horizontal direction coordinate or the vertical direction image coordinate value of each light point, to obtain the sorting relationship between each light point; obtaining a one-to-one correspondence between each light point and each light point, and calculating the VR handle pose corresponding to each frame by PNP (Persective-n-Point) method according to the correspondence; predicting the current pose corresponding to the current frame based on the VR handle poses in the previous two continuous frames; calculating the re-projection error based on the image coordinate value and the actual environment coordinate value of each light point corresponding to the sorting relationship, the camera intrinsic parameter and the current pose; judging whether the current pose is matched successfully according to the re-projection error and a preset threshold; if the matching is successful, setting the current pose as the VR handle positioning information corresponding to the current frame, and continuing the pose prediction in the subsequent frame to obtain the positioning information.

[0114] As an example, refer to Figure 3, each of the light points can be numbered as p0, p1, p2, p3 and p4 after sorting, and the number of continuous light points on the VR handle is N. As a preferred, N is 14, and is distributed in two rings staggered, each of which is 7, numbered as P1, P2,..., P N . Wherein, the VR handle pose corresponding to the current frame is T Q , the VR handle poses corresponding to the previous two frames are T Q-1 and T Q-2 , then the calculation formula of the VR handle pose prediction corresponding to the current frame includes,

[0115] Wherein, the step of sorting each of the light points according to the image coordinate values of each of the light points to obtain the sorting relationship between each of the light points includes:

[0116] Step S411, determining the circumscribed rectangle corresponding to each of the light points according to the image coordinate values of each of the light points;

[0117] Step S412, rotating the circumscribed rectangle and each of the light points to a preset direction, wherein the preset direction is a horizontal direction or a vertical direction;

[0118] Step S413, extracting the coordinates of each of the light points when the circumscribed rectangle and each of the light points have been rotated to the preset direction;

[0119] Step S414, determining the sorting direction according to the difference in horizontal direction and vertical direction of the coordinates of each of the light points, wherein the sorting direction is a horizontal direction or a vertical direction;

[0120] Step S415, determining the sorting relationship between each of the light points according to the sorting direction and the coordinates of each of the light points.

[0121] In the embodiments of the present application, it should be noted that when rotating the circumscribed rectangle and each of the light points, the circumscribed rectangle and each of the light points are an integral whole, and the center of rotation is the geometric center of the circumscribed rectangle. Wherein, when determining the preset direction as a horizontal direction or a vertical direction, it can be determined according to the angle of rotation as needed. If the angle of rotation to the horizontal direction is less than that to the vertical direction, then the preset direction is determined as the horizontal direction. Determining the sorting direction as a horizontal direction or a vertical direction depends on the difference in horizontal direction and the difference in vertical direction of the coordinates of each of the light points. If the difference in horizontal direction is greater than the difference in vertical direction, then the sorting direction is determined as the horizontal direction.

[0122] As an example, steps S411 to S415 include: determining a minimum circumscribed rectangle corresponding to each of the light points according to the coordinates of the light points; determining a preset direction based on an angle of the minimum circumscribed rectangle rotated to a horizontal direction and a size of the minimum circumscribed rectangle rotated to a vertical direction; rotating the circumscribed rectangle and each of the light points as a whole based on a collective center of the circumscribed rectangle until the circumscribed rectangle and each of the light points are rotated to the preset direction, where the preset direction is the horizontal direction or the vertical direction; if a difference between the coordinates of each of the light points in the horizontal direction and the vertical direction is greater than zero, sorting each of the light points based on the coordinate values of each of the light points in the horizontal direction; and if the difference between the coordinates of each of the light points in the horizontal direction and the vertical direction is not greater than zero, sorting each of the light points based on the coordinate values of each of the light points in the vertical direction.

[0123] The step of determining, according to the image coordinate values and the actual environment coordinate values of each of the light points in the sorting relationship, the camera intrinsic parameters, and the current pose, whether the current pose is matched successfully includes:

[0124] Step S441, determining a re-projection error through the image coordinate values and the actual environment coordinate values of each of the light points in the sorting relationship, the camera intrinsic parameters, and the current pose.

[0125] Step S442, if the re-projection error is not greater than a preset threshold, determining that the current pose is matched successfully.

[0126] Step S443, if the re-projection error is greater than the preset threshold, performing enumeration matching and positioning on the first number of light points and the second number of light points to obtain the positioning information.

[0127] As an example, steps S441 to S443 include: calculating a product between the actual environment coordinate values of each of the sorted light points, the camera intrinsic parameters, and the current pose, and calculating a difference between the product and the image coordinate values of each of the light points to obtain a re-projection error; if the re-projection error is not greater than a preset threshold, determining that the current pose is matched successfully; and if the re-projection error is greater than the preset threshold, performing enumeration matching on the first number of light points and the second number of light points on the VR handle to obtain each matching combination; performing pose prediction on the VR handle according to each of the matching combinations to obtain a VR handle predicted pose corresponding to each frame in each of the matching combinations; and performing screening on each of the VR handle predicted poses by calculating a re-projection error of each of the VR handle predicted poses and a deviation from the nearest available frame positioning to obtain the positioning information.

[0128] As an example, a calculation formula of the re-projection error is: e pro = K * T * P i -p iWhere K is the camera intrinsic parameter, T is the predicted VR controller pose obtained through the PNP method, and e pro As a preferred method, the reprojection error is preset to a threshold of 3, meaning that when the reprojection error is no more than 3 pixels, the VR controller pose of the current frame is successfully matched.

[0129] The step of enumerating and matching the first number of light spots with the second number of light spots to obtain the positioning information includes:

[0130] Step A10: Enumerate and match the first number of light spots with the second number of light spots to obtain each matching combination;

[0131] Step A20: Position the VR controller according to each of the matching combinations to obtain the predicted pose of the VR controller corresponding to each of the matching combinations;

[0132] Step A30: Based on the image coordinates and actual environment coordinates of each light point in the sorting relationship, the camera intrinsic parameters, and the current pose, evaluate the positioning accuracy of the predicted pose of each VR controller.

[0133] Step A40: Based on the positioning accuracy of the predicted pose of each VR controller, extract the positioning information from the predicted pose of each VR controller.

[0134] In this embodiment of the application, it should be noted that there are a total of 32 matching combinations, and the accuracy is determined by the reprojection error and the deviation from the positioning of the nearest available frame.

[0135] As an example, when the first quantity is 5, the interval between the 3D point numbers of the 5 2D points is set to 1 or 2. Then let the 3D point corresponding to point p0 be P. M Then there are 32 possible combinations of 3D points corresponding to other 2D points. If the index of the 3D point is less than 1, then N needs to be added; if the index of the 3D point is greater than N, then N needs to be subtracted.

[0136] As an example, steps A10 to A40 include: enumerating and matching the five 2D light points with each 3D light point on the VR controller to obtain a total of 32 matching combinations; calculating the VR controller pose for each frame using the PNP method according to each matching combination; predicting the current pose for the current frame based on the VR controller poses in the previous two consecutive frames; determining the reprojection error by using the image coordinates of each light point in the sorting relationship, the actual environment coordinates, the camera intrinsics, and the current pose; calculating the angle difference and position difference of each current pose based on the VR controller pose for each frame and the current pose; if the angle difference is less than a first preset value and the position difference is less than a second preset value... If the angle difference and the position difference are combined, the current pose is set as the VR controller positioning information for the current frame, and pose prediction continues in subsequent frames to obtain the positioning information. If the angle difference is not less than a first preset value or the position difference is not less than a second preset value, the positioning accuracy of each VR controller predicted pose is determined based on the angle difference, the position difference, and the reprojection error. If the positioning accuracy is not greater than a third preset value, the VR controller predicted pose corresponding to the positioning accuracy is set as the VR controller positioning information for the current frame, and pose prediction continues in subsequent frames to obtain the positioning information. If the accuracy is greater than the third preset value, the VR controller positioning is determined to have failed.

[0137] As a preferred embodiment, the first preset value is 1 degree, the second preset value is 0.01m, and the third preset value is 3.15.

[0138] As an example, if the 3D point corresponding to point p0 is P M Then there are 32 possible combinations of 3D points corresponding to other 2D points, of which 16 matching combinations include: [P M+1 ,P M+2 ,P M+3 ,P M+4 ]、

[0139] [P M+2 ,P M+3 ,P M+4 ,P M+6 ][P M+2 ,P M+4 ,P M+6 ,P M+8 ]、[P M+2 ,P M+3 ,P M+5 ,P M+6 ]、

[0140] [P M+2 ,P M+3 ,P M+5 ,PM+7 ]、[P M+2 ,P M+3 ,P M+4 ,P M+5 ]、[P M+2 ,P M+4 ,P M+5 ,P M+6 ]、

[0141] [P M+2 ,P M+4 ,P M+5 ,P M+7 ]、[P M+2 ,P M+4 ,P M+6 ,P M+7 ]、[P M+1 ,P M+3 ,P M+4 ,P M+6 ]、

[0142] [P M+1 ,P M+3 ,P M+4 ,P M+6 ]、[P M+1 ,P M+3 ,P M+5 ,P M+6 ]、[P M+1 ,P M+3 ,P M+5 ,P M+7 ]、

[0143] [P M+1 ,P M+2 ,P M+3 ,P M+5 ]、[P M+1 ,P M+2 ,P M+4 ,P M+6 ]、[P M+1 ,P M+2 ,P M+4 ,P M+5 For the other 16 matching combinations, simply replace the plus sign with a minus sign.

[0144] As a preferred approach, the first three enumerated matching combinations (6 in total if including minus signs) need to be tested first. In this example, over 90% of the cases have already been covered. If a matching combination already meets the criteria, then testing the other 13 combinations (26 in total if including minus signs) is unnecessary. For the starting point P corresponding to p1... M Further filtering can be performed when there is information on the most recently available frame (if the handle positioning results exist in the previous two frames).

[0145] As an example, by filtering through reprojection error and the deviation of the VR controller's predicted pose from the most recent available frame, the final error is: e = e pro *r dis *t dis , where r dis The angular difference between the frame and the nearest available frame, in degrees, t dis The positional difference between the nearest available frame and the nearest available frame, in meters. When the nearest available frame r dis Not greater than 1 and t dis If the value is not greater than 0.01, the predicted pose T of the corresponding VR controller is used as the VR controller positioning information of the current frame; otherwise, it is determined whether e is not greater than 3.15. If e is not greater than 3.15, the corresponding pose T is set as the VR controller positioning information of the current frame; if e is greater than 3.15, the VR controller positioning is determined to have failed.

[0146] Specifically, based on the aforementioned correspondence, the VR controller is positioned to obtain its positioning information:

[0147] Step B10: Sort the light points according to their image coordinate values ​​to obtain the sorting relationship between them.

[0148] Step B20: Enumerate and match the first number of light spots with the second number of light spots to obtain each matching combination;

[0149] Step B30: Position the VR controller according to each of the matching combinations to obtain the predicted pose of the VR controller corresponding to each of the matching combinations;

[0150] Step B40: Based on the image coordinates and actual environment coordinates of each light point in the sorting relationship, the camera intrinsic parameters, and the current pose, evaluate the positioning accuracy of the predicted pose of each VR controller.

[0151] Step B50: Based on the positioning accuracy of the predicted pose of each VR controller, extract the positioning information from the predicted pose of each VR controller.

[0152] In this embodiment of the application, it should be noted that steps B10 to B50 are used to obtain the positioning information by enumerating and matching each 2D light point with each 3D light point on the VR controller when the one-to-one correspondence between each 2D light point and each 3D light point cannot be clearly obtained. This results in obtaining each set of corresponding VR controller predicted poses, and then filtering the VR controller predicted poses using preset conditions. The specific implementation steps of steps B10 to B50 have been described in steps S41 and A10 to A40, and will not be repeated here.

[0153] This application provides a VR controller positioning method. First, an infrared image of the VR controller is acquired. Image processing is performed on the infrared image to obtain the centers of each light spot. Then, based on the contour size of each light spot center, a first number of light spots are selected from the centers, and the image coordinate values ​​of the first number of light spots are obtained. Next, the actual environmental coordinate values ​​corresponding to a second number of light spots on the VR controller are obtained. A one-to-one correspondence is matched between the second number of light spots and the first number of light spots based on the image coordinate values ​​and the actual environmental coordinate values, wherein the second number of light spots includes the first number of light spots. Finally, the VR controller is positioned according to the correspondence to obtain the positioning information of the VR controller. In this application embodiment, the light spots on the VR controller are selected based on the contour size corresponding to each light spot center to obtain a first number of light spots. The positioning result of the VR controller is calculated based on the correspondence between the second number of light spots and the first number of light spots. The first number can be set according to requirements, enabling the positioning of the VR controller with fewer points, reducing the computational load, improving positioning efficiency, overcoming the technical defects of large computational load caused by brute-force matching in the prior art, and improving the real-time performance of VR controller positioning.

[0154] Example 2

[0155] This application also provides a VR controller positioning device, which is applied to a VR controller positioning device, as described above. Figure 4 The VR controller positioning device includes:

[0156] The light spot extraction module is used to acquire the infrared image of the VR controller, perform image processing on the infrared image, and obtain the center of each light spot;

[0157] A spot filtering module is used to filter a first number of spots from the centers of the spots based on the outline size of the centers of the spots, and to obtain the image coordinate values ​​of the first number of spots.

[0158] The light spot correspondence module is used to obtain the actual environmental coordinate values ​​corresponding to the second number of light spots on the VR controller, and match the one-to-one correspondence between the second number of light spots and the first number of light spots according to the image coordinate values ​​and the actual environmental coordinate values, wherein the second number of light spots includes the first number of light spots;

[0159] The controller positioning module is used to position the VR controller according to the correspondence to obtain the positioning information of the VR controller.

[0160] Optionally, the light spot extraction module is further used for:

[0161] Contour extraction is performed on each light spot in the infrared image to obtain the contour corresponding to each light spot;

[0162] Ellipse fitting is performed on the contours of each light point to obtain the shape of each target contour.

[0163] Extract the geometric center of each of the contour shapes to obtain the center of each of the light spots.

[0164] Optionally, the handle positioning module is further used for:

[0165] The light points are sorted according to their image coordinate values ​​to obtain the sorting relationship between them.

[0166] Obtain the one-to-one correspondence between the image coordinates of each light point and the actual environment coordinates, and determine the VR controller pose for each frame based on the correspondence.

[0167] Based on the VR controller poses corresponding to the two frames preceding the current frame, the pose of the VR controller is predicted to obtain the current pose.

[0168] Based on the image coordinates of each light point in the sorting relationship, the actual environment coordinates, camera intrinsics, and the current pose, determine whether the current pose is successfully matched.

[0169] If a match is successful, the current pose is set to the VR controller positioning information corresponding to the current frame, and pose prediction continues in subsequent frames to obtain the positioning information.

[0170] Optionally, the handle positioning module is further used for:

[0171] Based on the image coordinate values ​​of each light point, determine the bounding rectangle corresponding to each light point;

[0172] Rotate the circumscribed rectangle and each of the light spots to a preset direction, wherein the preset direction is either horizontal or vertical.

[0173] When the circumscribed rectangle and each of the light points have been rotated to the preset direction, the coordinates of each of the light points are extracted.

[0174] The sorting direction is determined based on the difference between the coordinates of each light point in the horizontal and vertical directions, wherein the sorting direction is either horizontal or vertical.

[0175] The sorting relationship between the light points is determined based on the sorting direction and the coordinates of each light point.

[0176] Optionally, the handle positioning module is further used for:

[0177] The reprojection error is determined by the image coordinates of each light point in the sorting relationship, the actual environment coordinates, the camera intrinsic parameters, and the current pose.

[0178] If the reprojection error is not greater than a preset threshold, then the current pose matching is determined to be successful;

[0179] If the reprojection error is greater than the preset threshold, the first number of light spots and the second number of light spots are enumerated, matched and located to obtain the location information.

[0180] Optionally, the handle positioning module is further used for:

[0181] The first number of light spots and the second number of light spots are enumerated and matched to obtain each matching combination;

[0182] The VR controller is positioned according to each of the matching combinations to obtain the predicted pose of the VR controller corresponding to each matching combination;

[0183] The positioning accuracy of each VR controller's predicted pose is evaluated based on the image coordinates and actual environment coordinates of each light point in the sorting relationship, the camera intrinsic parameters, and the current pose.

[0184] Based on the positioning accuracy of the predicted pose of each VR controller, the positioning information is extracted from the predicted pose of each VR controller.

[0185] Optionally, the handle positioning module is further used for:

[0186] The light points are sorted according to their image coordinate values ​​to obtain the sorting relationship between them.

[0187] The first number of light spots and the second number of light spots are enumerated and matched to obtain each matching combination;

[0188] The VR controller is positioned according to each of the matching combinations to obtain the predicted pose of the VR controller corresponding to each matching combination;

[0189] The positioning accuracy of each VR controller's predicted pose is evaluated based on the image coordinates and actual environment coordinates of each light point in the sorting relationship, the camera intrinsic parameters, and the current pose.

[0190] Based on the positioning accuracy of the predicted pose of each VR controller, the positioning information is extracted from the predicted pose of each VR controller.

[0191] The VR controller positioning device provided in this application adopts the VR controller positioning method in the above embodiments, solving the technical problem of low real-time performance of VR controller positioning. Compared with the prior art, the beneficial effects of the VR controller positioning device provided in this application are the same as those of the VR controller positioning method provided in the above embodiments, and other technical features in this VR controller positioning device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0192] Example 3

[0193] This application provides an electronic device, which includes: at least one processor; and a memory communicatively linked to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the VR controller positioning method in the first embodiment described above.

[0194] The following is for reference. Figure 5 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of the present disclosure. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0195] like Figure 5 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also linked to the bus.

[0196] Typically, the following systems can be linked to the I / O interface: input devices such as touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices such as liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices such as magnetic tapes, hard disks, etc.; and communication devices. Communication devices allow electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although electronic devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0197] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined above in the methods of embodiments of this disclosure.

[0198] The electronic device provided in this application employs the VR controller positioning method in the above embodiments, solving the technical problem of low real-time performance in VR controller positioning. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the VR controller positioning method provided in Embodiment 1 above, and other technical features of this electronic device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0199] It should be understood that various parts of this disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0200] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0201] Example 4

[0202] This embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, which are used to execute the VR controller positioning method in the first embodiment described above.

[0203] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical link having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0204] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0205] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by an electronic device, the electronic device causes the following: to acquire an infrared image of a VR controller; to perform image processing on the infrared image to obtain the center of each light spot; to filter a first number of light spots from the light spot centers based on the contour size of each light spot center, and to acquire the image coordinate values ​​of the first number of light spots; to acquire the actual environmental coordinate values ​​corresponding to a second number of light spots on the VR controller; to match a one-to-one correspondence between the second number of light spots and the first number of light spots according to the image coordinate values ​​and the actual environmental coordinate values, wherein the second number of light spots includes the first number of light spots; and to locate the VR controller according to the correspondence to obtain the positioning information of the VR controller.

[0206] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be linked to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be linked to an external computer (e.g., via the Internet using an Internet service provider).

[0207] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0208] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0209] The computer-readable storage medium provided in this application stores computer-readable program instructions for executing the above-described VR controller positioning method, thus solving the technical problem of low real-time performance in VR controller positioning. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the VR controller positioning method provided in the above-described embodiments, and will not be repeated here.

[0210] Example 5

[0211] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the VR controller positioning method described above.

[0212] The computer program product provided in this application solves the technical problem of low real-time performance in VR controller positioning. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the VR controller positioning method provided in the above embodiments, and will not be repeated here.

[0213] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A VR controller positioning method, characterized in that, The VR controller positioning method, applied to VR devices, includes: Acquire the infrared image of the VR controller, perform image processing on the infrared image, and obtain the center of each light spot; Based on the contour size of the center of each light spot, a first number of light spots are selected from the centers of each light spot, and the image coordinate values ​​of the first number of light spots are obtained; Obtain the actual environmental coordinate values ​​corresponding to the second number of light spots on the VR controller, and match the one-to-one correspondence between the second number of light spots and the first number of light spots based on the image coordinate values ​​and the actual environmental coordinate values, wherein the second number of light spots includes the first number of light spots; Based on the correspondence, the VR controller is positioned to obtain the positioning information of the VR controller; The step of locating the VR controller according to the correspondence to obtain the positioning information of the VR controller includes: The light points are sorted according to their image coordinate values ​​to obtain the sorting relationship between them. Obtain the one-to-one correspondence between the image coordinates of each light point and the actual environment coordinates, and determine the VR controller pose for each frame based on the correspondence. Based on the VR controller poses corresponding to the two frames preceding the current frame, the pose of the VR controller is predicted to obtain the current pose. Based on the image coordinates of each light point in the sorting relationship, the actual environment coordinates, camera intrinsics, and the current pose, determine whether the current pose is successfully matched. If the match is successful, the current pose is set to the VR controller positioning information corresponding to the current frame, and pose prediction is continued in subsequent frames to obtain the positioning information. The step of determining whether the current pose is successfully matched based on the image coordinates of each light point in the sorting relationship, the actual environment coordinates, camera intrinsic parameters, and the current pose includes: The reprojection error is determined by the image coordinates of each light point in the sorting relationship, the actual environment coordinates, the camera intrinsic parameters, and the current pose. If the reprojection error is not greater than a preset threshold, then the current pose matching is determined to be successful; If the reprojection error is greater than the preset threshold, the first number of light spots and the second number of light spots are enumerated, matched and located to obtain the location information.

2. The VR controller positioning method as described in claim 1, characterized in that, The step of image processing the infrared image to obtain the center of each light spot includes: Contour extraction is performed on each light spot in the infrared image to obtain the contour corresponding to each light spot; Ellipse fitting is performed on the contours of each light point to obtain the shape of each target contour. Extract the geometric center of each of the contour shapes to obtain the center of each of the light spots.

3. The VR controller positioning method as described in claim 1, characterized in that, The step of sorting the light points according to their image coordinate values ​​to obtain the sorting relationship between them includes: Based on the image coordinate values ​​of each light point, determine the bounding rectangle corresponding to each light point; Rotate the circumscribed rectangle and each of the light spots to a preset direction, wherein the preset direction is either horizontal or vertical. When the circumscribed rectangle and each of the light points have been rotated to the preset direction, the coordinates of each of the light points are extracted. The sorting direction is determined based on the difference between the coordinates of each light point in the horizontal and vertical directions, wherein the sorting direction is either horizontal or vertical. The sorting relationship between the light points is determined based on the sorting direction and the coordinates of each light point.

4. The VR controller positioning method as described in claim 1, characterized in that, The step of enumerating and matching the first number of light spots with the second number of light spots to obtain the positioning information includes: The first number of light spots and the second number of light spots are enumerated and matched to obtain each matching combination; The VR controller is positioned according to each of the matching combinations to obtain the predicted pose of the VR controller corresponding to each matching combination; The positioning accuracy of each VR controller's predicted pose is evaluated based on the image coordinates and actual environment coordinates of each light point in the sorting relationship, the camera intrinsic parameters, and the current pose. Based on the positioning accuracy of the predicted pose of each VR controller, the positioning information is extracted from the predicted pose of each VR controller.

5. The VR controller positioning method as described in claim 1, characterized in that, Based on the aforementioned correspondence, the VR controller is positioned to obtain its positioning information: The light points are sorted according to their image coordinate values ​​to obtain the sorting relationship between them. The first number of light spots and the second number of light spots are enumerated and matched to obtain each matching combination; The VR controller is positioned according to each of the matching combinations to obtain the predicted pose of the VR controller corresponding to each matching combination; The positioning accuracy of each VR controller's predicted pose is evaluated based on the image coordinates of each light point in the sorting relationship, the actual environment coordinates, camera intrinsic parameters, and the VR controller's predicted pose. Based on the positioning accuracy of the predicted pose of each VR controller, the positioning information is extracted from the predicted pose of each VR controller.

6. A VR controller positioning device, characterized in that, The VR controller positioning device includes: The light spot extraction module is used to acquire infrared images of the VR controller, perform image processing on the infrared images, and obtain the centers of each light spot; A spot filtering module is used to filter a first number of spots from the centers of the spots based on the outline size of the centers of the spots, and to obtain the image coordinate values ​​of the first number of spots. The light spot correspondence module is used to obtain the actual environmental coordinate values ​​corresponding to the second number of light spots on the VR controller, and match the one-to-one correspondence between the second number of light spots and the first number of light spots according to the image coordinate values ​​and the actual environmental coordinate values, wherein the second number of light spots includes the first number of light spots; A controller positioning module is used to position the VR controller according to the correspondence to obtain the positioning information of the VR controller; The controller positioning module is further configured to: sort the light points according to their image coordinate values ​​to obtain a sorting relationship between them; obtain a one-to-one correspondence between the image coordinate values ​​of each light point and the actual environment coordinate values, and determine the VR controller pose for each frame based on the correspondence; predict the pose of the VR controller based on the VR controller poses of the two frames preceding the current frame to obtain the current pose; determine whether the current pose is successfully matched based on the image coordinate values ​​of each light point in the sorting relationship, the actual environment coordinate values, camera intrinsics, and the current pose; if the match is successful, set the current pose as the VR controller positioning information corresponding to the current frame, and continue to perform pose prediction in subsequent frames to obtain the positioning information. The handle positioning module is further configured to: determine the reprojection error by using the image coordinates of each light point in the sorting relationship, the actual environment coordinates, the camera intrinsic parameters, and the current pose; if the reprojection error is not greater than a preset threshold, then determine that the current pose is successfully matched; if the reprojection error is greater than the preset threshold, then enumerate and match the first number of light points with the second number of light points and locate them to obtain the positioning information.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory that is communicatively linked to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the steps of the VR controller positioning method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program for implementing a VR controller positioning method, which is executed by a processor to implement the steps of the VR controller positioning method as described in any one of claims 1 to 5.

Images