Positioning Method, Device, Head-Mounted Display Device and Storage Medium of a Controller

Abstract

The present application discloses a positioning method, device, head-mounted display device and storage medium of a controller, relating to the field of virtual reality technology. The method includes: obtaining an image containing the controller in the real environment as a first image; obtaining a first number of light points that meet the preset distribution condition from the first image as a first light point set; determining, from the three-dimensional model corresponding to the controller, a three-dimensional model that matches the first light point set as a target three-dimensional model; mapping the target three-dimensional model to a two-dimensional plane to obtain a second image; obtaining a second number of light points that match the preset light point distribution in the second image from a second light point set in the first image as a third light point set; determining the pose information of the controller in the real environment based on the first light point set and the third light point set. In this way, a sufficient number of light point sets formed by the light-emitting units of the controller can be accurately recognized, so that the pose information of the controller determined based on the light point sets is more accurate.

Description

Technical Field

[0001] The present application relates to the field of virtual reality technology, and more specifically, to a positioning method, device, head-mounted display device, and storage medium for a controller. Background Art

[0002] A head-mounted display device (HMD) is a display device that can be worn on a user's head and can achieve different effects such as virtual reality (VR), augmented reality (AR), and mixed reality (MR). The HMD can be used in conjunction with a handle control tracker. During this process, a virtual reality, augmented reality, or mixed reality scene is presented on the HMD, and the user interacts with the elements in the above scene by controlling the handle controller held in the hand.

[0003] In the related art, a light-emitting unit is carried on the handle controller. An image of the controller during movement is acquired by an image acquisition device, and the pose and position information of the handle controller are obtained by inversely solving the position information of the light-emitting points corresponding to the light-emitting units in the image. However, since the light-emitting points formed by the light-emitting units of the controller on the acquired image cannot be accurately identified, the pose and position information of the handle controller inversely solved based on the light-emitting points are inaccurate. Summary of the Invention

[0004] In view of this, the present application provides a positioning method, device, head-mounted display device, and storage medium for a controller.

[0005] In a first aspect, an embodiment of the present application provides a positioning method for a controller. The method includes: acquiring an image including the controller in the real environment as a first image, where a plurality of light-emitting units are carried on the controller, and the first image includes light points corresponding to the light-emitting units; acquiring a first number of light points that meet a preset distribution condition from the first image to obtain a first light point set; determining, from the three-dimensional model corresponding to the controller, a three-dimensional model that matches the first light point set as a target three-dimensional model; mapping the target three-dimensional model to a two-dimensional plane to obtain a second image, where the second image includes preset light points corresponding to all the light-emitting units on the controller; acquiring a second number of light points that match the distribution of the preset light points in the second image from a second light point set in the first image to obtain a third light point set, where the second light point set includes other light points in the first image except the first light point set; and determining the pose information of the controller in the real environment based on the first light point set and the third light point set.

[0006] In a second aspect, an embodiment of the present application provides a positioning device for a controller. The device includes: an image acquisition module, a first light point set acquisition module, a three-dimensional model acquisition module, a mapping module, a third light point set acquisition module, and a positioning module. The image acquisition module is configured to acquire an image containing the controller in the real environment as a first image. A plurality of light-emitting units are carried on the controller, and the first image includes light points corresponding to the light-emitting units. The first light point set acquisition module is configured to acquire a first number of light points that meet a preset distribution condition from the first image to obtain a first light point set. The three-dimensional model acquisition module is configured to determine, from the three-dimensional model corresponding to the controller, a three-dimensional model that matches the first light point set as a target three-dimensional model. The mapping module is configured to map the target three-dimensional model onto a two-dimensional plane to obtain a second image, and the second image includes preset light points corresponding to all the light-emitting units on the controller. The third light point set acquisition module is configured to acquire a second number of light points that match the distribution of the preset light points in the second image from a second light point set in the first image to obtain a third light point set. The second light point set includes other light points in the first image except the first light point set. The positioning module is configured to determine the pose information of the controller in the real environment based on the first light point set and the third light point set.

[0007] In a third aspect, an embodiment of the present application provides a head-mounted display device, including: one or more processors; a memory; one or more programs, where the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the positioning method for the controller provided in the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which program code is stored, and the program code can be called by a processor to execute the positioning method for the controller provided in the first aspect.

[0009] In the solution provided by this application, an image including a controller in the real environment is obtained as a first image. A plurality of light-emitting units are carried on the controller, and light points corresponding to the light-emitting units are included in the first image. A first number of light points meeting a preset distribution condition are obtained from the first image to obtain a first light point set. From the three-dimensional model corresponding to the controller, a three-dimensional model matching the first light point set is determined as a target three-dimensional model. The target three-dimensional model is mapped to a two-dimensional plane to obtain a second image, and preset light points corresponding to all the light-emitting units on the controller are included in the second image. A second number of light points matching the distribution of the preset light points in the second image are obtained from a second light point set in the first image to obtain a third light point set. The second light point set includes other light points in the first image except the first light point set. Based on the first light point set and the third light point set, the pose information of the controller in the real environment is determined. In this way, based on the distribution of the preset light points in the second image, a sufficient number of light point sets formed by the light-emitting units of the controller are accurately identified from the first image, so that the pose information of the controller determined based on the light point sets is more accurate, that is, the accuracy of positioning and tracking the controller is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative efforts.

[0011] Figure 1 FIG. shows a schematic diagram of an application scenario provided by an embodiment of the present application.

[0012] Figure 2 FIG. shows a schematic flowchart of a method for positioning a controller provided by an embodiment of the present application.

[0013] Figure 3 FIG. shows a schematic diagram of the light-emitting units of a controller provided by an embodiment of the present application.

[0014] Figure 4 FIG. shows a schematic diagram of the preset light points in a second image provided by an embodiment of the present application.

[0015] Figure 5 FIG. shows a schematic flowchart of a method for positioning a controller provided by another embodiment of the present application.

[0016] Figure 6 FIG. shows a schematic diagram of the light point distribution of a controller provided by an embodiment of the present application.

[0017] Figure 7 FIG. shows Figure 5Flow schematic diagram of the sub-steps in step S370.

[0018] Figure 8 Shows Figure 7 Flow schematic diagram of the sub-steps in step S372.

[0019] Figure 9 Shows the flow schematic diagram of the positioning method of the controller provided in another embodiment of the present application.

[0020] Figure 10 Is a block diagram of a positioning device for a controller provided in an embodiment of the present application.

[0021] Figure 11 Is a block diagram of a head-mounted display device for executing the positioning method of the controller according to the embodiment of the present application.

[0022] Figure 12 Is a storage unit for storing or carrying the program code for implementing the positioning method of the controller according to the embodiment of the present application. Detailed implementation manners

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application.

[0024] A head-mounted display device is a display device that can be worn on the user's head and can achieve different effects such as VR, AR, and MR. The head-mounted display device can be used in cooperation with a handle control tracker. During this process, virtual reality, augmented reality, or mixed reality scenes are presented on the head-mounted display device, and the user interacts with the elements in the above scenes by controlling the handle controller held in the hand.

[0025] In the related art, a light-emitting unit is carried on the handle controller. The image of the controller during movement is acquired by an image acquisition device, and the attitude and position information of the handle controller are obtained by inverse-solving according to the position information of the light-emitting points corresponding to the light-emitting unit in the image, so as to realize real-time tracking of the handle controller. However, there are generally interfering light sources in the real environment where the handle controller is located, resulting in inaccurate position information of the acquired light-emitting points, and further inaccurate attitude and position information of the inverse-solved handle controller.

[0026] In view of the above problems, the inventor proposes a method and device for positioning a controller, a head-mounted display device, and a storage medium. First, a first set of light points can be obtained from a first image including the controller, then a second image mapped by a target three-dimensional model of the controller can be obtained based on the first set of light points, and a third set of light points can be obtained based on the second image. Finally, the pose information of the controller can be determined according to the first set of light points and the third set of light points. The following will describe this in detail.

[0027] The following introduces the application environment of the controller positioning method provided in the embodiments of the present application.

[0028] Please refer to Figure 1 , Figure 1 , which shows a schematic diagram of an application scenario provided in an embodiment of the present application. The application scenario includes a controller positioning system 10. Among them, the controller positioning system 10 may include a controller 110 and a head-mounted display device 120. The controller 110 and the head-mounted display device 120 are connected through a wireless or wired network to realize data transmission between the controller 110 and the head-mounted display device based on this network connection. The transmitted data includes but is not limited to audio, video, text, images, etc. Among them, the number of controllers 110 may be 1 or multiple, and this embodiment does not limit this.

[0029] In some embodiments, the head-mounted display device 120 may capture an image of the controller in the display environment based on its own image acquisition device, obtain a first set of light points and a third set of light points from the image, and then determine the pose information of the controller in the real environment based on the first set of light points and the third set of light points to realize the positioning of the controller.

[0030] In other embodiments, the head-mounted display device 120 may capture an image of the controller in the display environment based on its own image acquisition device and send the image to the server. The server obtains a first set of light points and a third set of light points from the image, determines the pose information of the controller in the real environment based on the first set of light points and the third set of light points, and finally feeds back the determined pose information to the head-mounted display device 120 to realize the positioning of the controller. Among them, the server includes but is not limited to a single server, a server cluster, a local server, a cloud server, etc.

[0031] Please refer to Figure 2 , Figure 2 A method and device for positioning a controller, a head-mounted display device, and a storage medium provided in an embodiment of the present application. The following will elaborate on the controller positioning method provided in the embodiments of the present application in combination with Figure 2 The controller positioning method may include the following steps:

[0032] Step S210: Obtain an image including a controller in the real environment as a first image. A plurality of light-emitting units are carried on the controller, and light points corresponding to the light-emitting units are included in the first image.

[0033] In this embodiment, taking the head-mounted display device analyzing and calculating the pose information of the controller as an example, the head-mounted display device can capture and obtain an image including the controller in the real environment as the first image based on a built-in or external image acquisition device connected to itself. Among them, the image acquisition device includes but is not limited to a monocular camera, a binocular camera, a multiocular camera, a black-and-white camera, a color camera, etc.; a plurality of light-emitting units are carried on the controller, such as Figure 3 the light-emitting unit 11, the light-emitting unit 12, the light-emitting unit 13, the light-emitting unit 14, the light-emitting unit 15, the light-emitting unit 16, the light-emitting unit 21, the light-emitting unit 22, the light-emitting unit 23, the light-emitting unit 24, the light-emitting unit 25, and the light-emitting unit 26 in. Of course, the number of light-emitting units may not be limited to the number in the figure, and this number can be preset, such as 10, 16, or 20, etc. This embodiment does not limit this. The light-emitting unit can be a light-emitting diode. The intervals between different adjacent light-emitting units are inconsistent, and the arrangement of two adjacent light-emitting units on the same ring is one high and one low. In this way, it can be ensured that when the controller is photographed from different angles, the arrangement of the light-emitting units on the controller is not repeated and has distinctiveness. Furthermore, the arrangement of the light points in the first image obtained at different angles is different, improving the efficiency and accuracy of obtaining the light points. The shape of the light-emitting unit includes but is not limited to a circle, a triangle, a pentagram, etc., and the color of the light-emitting unit can also be any color, such as red, yellow, blue, red and blue, etc. This embodiment does not limit this.

[0034] In some embodiments, an image including the controller in the real environment can be collected through the image acquisition device every preset time period as the first image. Among them, the preset time period can be a preset time period, such as 100 ms, 500 ms, or 1 s, etc., and the time period can also be adjusted according to different application scenarios. This embodiment does not limit this.

[0035] Step S220: Obtain a first number of light points that meet the preset distribution condition from the first image to obtain a first light point set.

[0036] In this embodiment, the real environment where the controller is located may include other interfering light sources. Therefore, in the first image obtained, there may be both the light points formed by the light-emitting units on the controller and the interfering light points formed by the interfering light sources. Based on this, in order to avoid misidentifying the interfering light points, that is, to accurately identify all the light points in the first image, a first number of light points can be obtained from the first image according to a preset distribution condition to obtain a first light point set, so as to find other light points from the first image based on the first light point set later.

[0037] It can be understood that in the PNP (Perspective-n-Point) algorithm, when a given image is provided, at least 3 points are required to reverse-solve the pose information of the controller. Based on this, the value of the first number can be preset to 3. Correspondingly, the preset distribution condition can be that the included angle between two line segments formed by 3 light points is greater than a preset degree (such as 120 degrees) and the lengths of the two line segments match the preset length.

[0038] Step S230: Determine, from the three-dimensional model corresponding to the controller, the three-dimensional model that matches the first light point set as the target three-dimensional model.

[0039] Based on this, after obtaining the first light point set, the P3P (Perspective-3-Point) method can be used to preliminarily estimate the pose information of the controller in the first image as the estimated pose information; then, based on the pre-stored three-dimensional model of the controller, the three-dimensional model under this estimated pose information is obtained as the target three-dimensional model. Specifically, obtain the three-dimensional coordinates of the light-emitting units corresponding to the first light point set in the world coordinate system, and obtain the pixel coordinates of the first light point set in the first image in the pixel coordinate system; according to the geometric relationship between the three-dimensional coordinates, pixel coordinates, and each light point in the first light point set, obtain the rotation matrix and translation matrix from the world coordinate system to the pixel coordinate system; based on this rotation matrix and translation matrix, obtain the estimated pose information of the controller.

[0040] Step S240: Map the target three-dimensional model to a two-dimensional plane to obtain a second image, and the second image includes the preset light points corresponding to all the light-emitting units on the controller.

[0041] In this embodiment, after obtaining the target three-dimensional model, the target three-dimensional model can be mapped to a two-dimensional plane to obtain a second image. It can be understood that the mapped second image includes the preset light points corresponding to all the light-emitting units on the controller.

[0042] Please refer to Figure 4 , Figure 4 shows Figure 3Schematic diagram of the arrangement of each preset light point in the second image mapped by the middle controller under different pose information. Of course, the aforementioned estimated pose information includes, but is not limited to Figure 4 several different pose information in

[0043] Step S250: Obtain a second number of light points that match the preset light point distribution in the second image from the second light point set in the first image to obtain a third light point set, where the second light point set includes other light points in the first image except the first light point set.

[0044] Based on this, other light points in the first image except the first light point set are obtained as the second light point set. It can be understood that the second light point set includes the remaining light points and interference light points. Therefore, a second number of light points that match the preset distribution can be obtained from the second light point set based on the preset distribution of the preset light points in the second image to obtain a third light point set. Since the second image is mapped based on the target three-dimensional model and the distribution of each light-emitting unit in the target three-dimensional model in three-dimensional space is known, correspondingly, the distribution of the preset light points corresponding to each light-emitting unit in the second image is also known.

[0045] In some embodiments, the second number can be a preset number. For example, 4, 5, 8, 10, or 15, etc. This embodiment does not limit this. According to the different positioning accuracies of the controller, the second number can be adjusted correspondingly. When the positioning accuracy requirement for the controller is high, the second number can be set to a larger value. In this way, more accurate positioning of the controller can be achieved; when the timeliness requirement for the controller positioning is high, the second number can be set to a smaller value. In this way, the calculation speed can be increased, thereby ensuring the timeliness of the controller positioning and avoiding positioning delay.

[0046] In some other embodiments, the second quantity may be determined based on the target three-dimensional model and the pose information of the controller. Specifically, the number of visible light points in the second image mapped by the target three-dimensional model under the pose information is obtained, and the difference between the number of visible light points and the first quantity is obtained as the aforementioned second quantity. In this way, all visible light points are obtained as much as possible, and then the pose information of the controller is determined based on all the visible light points, ensuring the accuracy of the controller positioning. In this method, if the number of visible light points obtained within the preset time duration is still less than the second quantity, the already obtained visible light points are used as the third light point set, and no other visible light points are obtained continuously. In this way, the problem that the light-emitting unit on the controller is blocked by other objects, resulting in the non-display of the visible light points that should originally be displayed in the first image, and thus increasing the time duration for finding the visible light points can be avoided. That is to say, the real-time performance of the controller positioning is effectively ensured, ensuring that the head-mounted display device can timely obtain the pose information of the controller, and realizing the real-time tracking and positioning of the controller.

[0047] Step S260: Determine the pose information of the controller in the real environment based on the first light point set and the third light point set.

[0048] In this embodiment, after obtaining the first light point set and the third light point set, methods such as PNP (Perspective-n-Point), such as EPnP (Efficient Perspective-n-Point), DLS (Direct Least-Squares), or BA (Bundle Adjustment) can be used. This embodiment does not limit this, and the first light point set and the third light point set are inversely solved to obtain the pose information of the controller in the display environment. Among them, the pose information may include one or both of the position information and the attitude information. The position information can be represented by coordinates, and the attitude information can be represented by angles. In this way, based on the first image collected at each preset time interval, the pose information of the controller at different times is obtained, realizing the real-time tracking and positioning of the controller.

[0049] In some embodiments, an IMU (Inertial Measurement Unit) sensor can be built into the controller to obtain the pose information of the controller collected by the IMU as the first pose information; obtain the pose information of the controller determined based on the first light point set and the third light point set as the second pose information; and fuse the first pose information and the second pose information to obtain the actual pose information of the controller. In this way, fusing the pose information determined based on the IMU sensor and the pose information determined according to the light point set can improve the accuracy of the finally obtained pose information of the controller, which helps the head-mounted display device to perform real-time tracking and positioning of the controller.

[0050] In some embodiments, in the real environment, there may be a situation where some interfering light sources are very close to the light-emitting units on the controller. To avoid misidentifying the interfering light points formed by the interfering light sources, after obtaining the third light point set by obtaining the second quantity of light points that match the preset light point distribution in the second image from the second light point set in the first image, further determine whether the size of each light point in the third light point set meets the preset size condition; if there is a target light point whose size does not meet the preset size condition, remove the target light point from the third light point set, where the target light point is any light point in the third light point set; finally, based on the first light point set and the third light point set after removing the target light point, determine the pose information of the controller in the real environment. In this way, the size judgment condition for the light points is introduced, which avoids misidentifying other interfering light points with similar positions, improves the accuracy of the obtained light point set, and further makes the pose information of the controller determined based on the light point set more accurate, that is, improves the accuracy of positioning and tracking the controller.

[0051] In some embodiments, the obtained first image includes multiple controllers. For example, if the user holds a controller in each hand, the first image includes 2 controllers, such as the first controller and the second controller. The light-emitting units on the 2 controllers are arranged in a mirror image, that is, the arrangements of the light-emitting units on the 2 controllers are different. Therefore, according to the distribution of the light points in the first image, the first light point set corresponding to the first controller and the first light point set corresponding to the second controller can be determined. Based on the first light point set corresponding to the first controller, the pose information of the first controller in the real environment is determined, and according to the first light point set corresponding to the second controller, the pose information of the second controller in the real environment is determined. The specific implementation manners can refer to the content in the foregoing embodiments and will not be elaborated here. In this way, due to the different arrangements of the light-emitting units on the multiple controllers, it helps the head-mounted display device to determine the first light point set corresponding to each controller according to the distribution of the light points in the first image, and realizes the tracking and positioning of each controller in the multiple controllers.

[0052] In this embodiment, a target three-dimensional model is determined through a first set of light points, and then a second image of the target three-dimensional model mapped to a two-dimensional plane is obtained, and the distribution of preset light points in the second image is obtained. Finally, based on the distribution of the preset light points in the second image, a sufficient number of sets of light points formed by the light-emitting units of the controller are accurately identified from the first image, so that the pose information of the controller determined based on the set of light points is more accurate, that is, the accuracy of positioning and tracking the controller is improved.

[0053] Please refer to Figure 5 , Figure 5 A positioning method, device, head-mounted display device and storage medium for a controller provided in another embodiment of the present application. The following will be combined with Figure 5 The positioning method of the controller provided in the embodiment of the present application will be elaborated in detail. The positioning method of the controller may include the following steps:

[0054] Step S310: Obtain an image including a controller in a real environment as a first image. A plurality of light-emitting units are carried on the controller, and the first image includes light points corresponding to the light-emitting units.

[0055] In this embodiment, the specific content in step S310 may refer to the content in the foregoing embodiment and will not be elaborated here.

[0056] Step S320: Group all the light points in the first image to obtain a variety of light point combinations, and the number of light points included in each light point combination in the variety of light point combinations is the first number.

[0057] In this embodiment, all the light points in the first image are grouped to obtain a variety of light point combinations. Among them, the number of light points included in each light point combination is the first number. Among them, the first number of light points included in each light point combination may all be adjacent (for example, 3 adjacent light points are one kind of light point combination); of course, the first number of light points included in each light point combination may also not be adjacent, and different grouping methods may be preset according to different needs, and this embodiment does not limit this.

[0058] In some embodiments, if multiple light-emitting units carried on the controller are arranged in multiple rings, all the light points corresponding to any one of the multiple rings on the first image are grouped to obtain the multiple light-point combinations. For example, when the first quantity is 3 and the number of rings is 2, each light-point combination in the multiple light-point combinations includes 3 light points in the upper ring, or each includes 3 light points in the lower ring. Thus, since the number of multiple light-point combinations grouped from the same ring is limited, it helps to match the first light-point set among the multiple light-point combinations later, improves the matching speed of the first light-point set, and further improves the efficiency of positioning and tracking the controller.

[0059] In other embodiments, if multiple light-emitting units carried on the controller are arranged in multiple rings, all the light points on the target ring among the multiple rings are grouped to obtain multiple first light-point combinations, and the quantity included in each first light-point combination is the third quantity; all the light points on any other ring except the target ring among the multiple rings are grouped to obtain multiple second light-point combinations, and the quantity included in each second light-point combination is the fourth quantity; the multiple first light-point combinations and the multiple second light-point combinations are combined in different ways to obtain the aforementioned multiple light-point combinations, where the sum of the third quantity and the fourth quantity is equal to the first quantity. Exemplarily, when the first quantity is 3 and the number of rings is 2, each light-point combination in the multiple light-point combinations includes 1 light point in the upper ring and 2 light points in the lower ring, or each includes 2 light points in the upper ring and 1 light point in the lower ring, and this embodiment does not limit this. Thus, since the multiple light-point combinations are grouped from different rings, there are more included light-point combinations, providing more matching options.

[0060] Step S330: Obtain the distance between adjacent light points in each light-point combination among the multiple light-point combinations, and the angles between multiple line segments formed by the adjacent light points, as the light-point distribution information corresponding to each light-point combination.

[0061] Based on this, after obtaining the multiple light-point combinations, obtain the distance between adjacent light points in each light-point combination, and the angles between multiple line segments formed by the adjacent light points, as the light-point distribution information corresponding to each light-point combination. Exemplarily, please refer to Figure 6 , light-point combination 1 includes light point A, light point B, and light point C. Obtain the distances AB and BC between adjacent light points, and the angle ∠1 between line segment AB and line segment BC, as the light-point distribution information corresponding to light-point combination 1.

[0062] Step S340: From the multiple light-point combinations, obtain the light-point combinations whose light-point distribution information simultaneously meets the preset distance distribution condition and the preset angle distribution condition, as the first light-point set.

[0063] In this embodiment, the preset distance distribution condition may be that the difference between the distance between adjacent light points and the preset distance is within the error threshold, and the preset angle distribution condition may be that the angle between two line segments formed by adjacent light points is not less than the preset angle. After obtaining the light point distribution information corresponding to each light point combination in multiple light point combinations, it is respectively determined whether the light point distribution information corresponding to each light point combination simultaneously meets the preset distance distribution condition and the preset angle distribution condition. If there is target light point distribution information that simultaneously meets the preset distance distribution condition and the preset angle distribution condition, the light point combination corresponding to the target light point distribution information is obtained as the first light point set, where the target light point distribution information is the light point distribution information corresponding to any light point combination.

[0064] In some embodiments, if there are multiple pieces of target light point distribution information that simultaneously meet the preset distance distribution condition and the preset angle distribution condition, the light point combination corresponding to the target light point distribution information that best meets the preset distance distribution condition and the preset angle distribution condition among the multiple pieces of target light point distribution information can be obtained as the first light point set. Specifically, the light point combination corresponding to the target light point distribution information with the smallest difference between the distance between adjacent light points and the preset distance, and / or the smallest difference between the angle between two line segments formed by adjacent light points and the preset angle among the multiple pieces of target light point distribution information is obtained as the first light point set.

[0065] Step S350: Determine, from the three-dimensional model corresponding to the controller, the three-dimensional model that matches the first light point set as the target three-dimensional model.

[0066] Step S360: Map the target three-dimensional model to a two-dimensional plane to obtain a second image, and the second image includes the preset light points corresponding to all the light-emitting units on the controller.

[0067] In this embodiment, the specific content in steps S350 to S360 can refer to the content in the foregoing embodiments and will not be elaborated here.

[0068] Step S370: Obtain a third light point set by acquiring, from the second light point set in the first image, the second number of light points that match the preset light point distribution in the second image, where the second light point set includes other light points in the first image except the first light point set.

[0069] In some embodiments, please refer to Figure 7 , step S370 may include the following steps:

[0070] Step S371: According to the orientation of the target three-dimensional model, obtain the visible light points among the preset light points in the second image, where the visible light points are the light points included in the image obtained by photographing the controller from the direction facing the orientation.

[0071] In this embodiment, due to the singularity of the distribution of the light-emitting units on the controller, in the first images of the controller taken from different directions, the included light points and the distribution of the light points are different. Still referring to Figure 4 shown, Figure 4 shows the different distribution states of the light-emitting units on the controller observed in different directions (front, back, right, and top view). Therefore, the visible light points in the first images obtained by photographing the controller from different directions are also different.

[0072] It can be understood that the target three-dimensional model is obtained based on the first light point set in the first image. Therefore, the visible light points in the second image obtained based on the target three-dimensional model will all be photographed and displayed at the corresponding positions in the first image. Based on this, according to the obtained orientation of the target three-dimensional model, the direction facing the orientation of the target three-dimensional model can be determined as the target photographing direction, and the light points included in the image obtained by photographing the controller from the target photographing direction in the second image are obtained as the visible light points. Exemplarily, Figure 4 the black points in show the visible light points, and the gray points represent the other light points among the preset light points except the visible light points.

[0073] Step S372: From the second light point set, obtain the second number of light points that match the distribution of the visible light points in the second image, and obtain the third light point set.

[0074] In some embodiments, the light points in the first light point set are used as the first light points, and the light points in the second light point set are used as the second light points. Referring to 8, step S372 may include the following steps:

[0075] Step S3721: Obtain the relative position information between each second light point in the second light point set and each first light point in the first light point set, and obtain the position distribution information corresponding to each second light point, where the position distribution information includes the relative position information between each second light point and all the first light points in the first light point set.

[0076] In this embodiment, since it is necessary to obtain the light points in the second light point set that match the visible light point distribution in the second image, therefore, the relative position information between each second light point in the second light point set and each first light point in the first light point set can be obtained first, and the position distribution information corresponding to each second light point can be obtained. Among them, the position distribution information includes the relative position information between each second light point and all the first light points in the first light point set. The relative position information includes the distance between the second light point and each first light point, and / or the angle between the multiple line segments formed by the second light point and each first light point. For the specific implementation manner of obtaining the relative position information, reference can be made to the content in the foregoing embodiments, and details are not described herein again.

[0077] Step S3722: Obtain the visible light points corresponding to the first light point set from the visible light points as the first visible light points.

[0078] Step S3723: Obtain the other visible light points except the first visible light points from the visible light points as the second visible light points.

[0079] Among them, there are visible light points in the second image that correspond one by one to the first light point set. Therefore, the visible light points corresponding to the first light point set can be obtained as the first visible light points; then, the other visible light points in the second image except the first visible light points can be obtained as the second visible light points. It can be understood that the second light points in the second light point set that match the distribution of the second visible light points can be added to the third light point set.

[0080] Step S3724: Obtain the preset position information between each of the second visible light points and each of the first visible light points to obtain the preset distribution information corresponding to each of the second visible light points. The preset distribution information includes the preset position information between each of the second visible light points and all of the first visible light points.

[0081] Based on this, further obtain the preset position information between each second visible light point and each first visible light point to obtain the preset distribution information corresponding to each second visible light point. Among them, the preset distribution information includes the preset position information between each second visible light point and all first visible light points. The preset position information includes the distance between the second visible light point and each first visible light point, and / or the angle between the multiple line segments formed by the second visible light point and each first visible light point. The preset position information can be the position information obtained from the pre-set position database according to the orientation of the target three-dimensional model, or the position information calculated in real time according to the orientation of the target three-dimensional model. This embodiment does not limit this.

[0082] Step S3725: Obtain second light points in the second light point set whose position distribution information matches the preset position information, to obtain the third light point set.

[0083] In this embodiment, after obtaining the preset distribution information of the second visible light points and the position distribution information of the second light points, the position distribution information of each second light point in the second light point set is matched with the preset distribution information of each second visible light point. If there is a target second light point that matches successfully, then based on the target second light point, a third light point set is generated. Wherein, the target second light point is any second light point in the second light point set.

[0084] Specifically, if the position distribution information and the preset position information meet the specified distribution conditions, it is determined that the position distribution information matches the preset information. Wherein, the specified distribution conditions may include a specified distance distribution condition and a specified angle distribution condition. The specified distance distribution condition may be that the difference between the distance between the second light point and the first light point and the distance between the corresponding second visible light point and the first visible light point is within a specified distance threshold, and the specified angle distribution condition may be that the difference between the angles between multiple line segments formed by the second light point and the first light point and the angles between multiple line segments formed by the corresponding second visible light point and the first visible light point is within a specified angle threshold.

[0085] Step S380: Determine the pose information of the controller in the real environment based on the first light point set and the third light point set.

[0086] In this embodiment, the specific content in step 380 can refer to the content in the foregoing embodiments, and will not be elaborated herein.

[0087] In this embodiment, first obtain the distribution of the second visible light points in the second image, then obtain the second light points in the second light point set in the first image that match the distribution of the second visible light points to obtain the third light point set. Finally, based on the first light point set and the third light point set, determine the pose information of the controller in the real environment. In this way, only the second light points in the second light point set are matched with the distribution of the second visible light points in the second image, reducing the number of matches, improving the efficiency of obtaining the third light point set, and further improving the efficiency of real-time positioning and tracking of the controller based on the light point set.

[0088] Please refer to Figure 9 , Figure 9 For a positioning method, device, head-mounted display device and storage medium of a controller provided in another embodiment of the present application. The following will combine Figure 9 to elaborate in detail on the positioning method of the controller provided in the embodiments of the present application. The positioning method of the controller may include the following steps:

[0089] Step S410: Obtain an image containing a controller in the real environment as a first image. A plurality of light-emitting units are carried on the controller, and light points corresponding to the light-emitting units are included in the first image.

[0090] Step S420: Obtain a first number of light points that meet the preset distribution condition from the first image to obtain a first light point set.

[0091] Step S430: Determine, from the three-dimensional model corresponding to the controller, a three-dimensional model that matches the first light point set as a target three-dimensional model.

[0092] Step S440: Map the target three-dimensional model to a two-dimensional plane to obtain a second image, and preset light points corresponding to all the light-emitting units on the controller are included in the second image.

[0093] In this embodiment, the specific content in steps S410 to S440 may refer to the content in the foregoing embodiments and will not be elaborated herein.

[0094] Step S450: Obtain a light point that matches the preset light point distribution from a second light point set in the first image and add it to the third light point set.

[0095] In some embodiments, when there are multiple light points in the second light point set that match the preset light point distribution, any one of the matching light points may be selected and added to the third light point set. In this way, the speed of obtaining the light points belonging to the third light point set is ensured, and the real-time performance of tracking and positioning the controller is ensured.

[0096] In other embodiments, when there are multiple light points in the second light point set that match the preset light point distribution, the light point that most matches the preset light point distribution may be obtained and added to the third light point set. Among them, the principle of obtaining the light point that most matches the preset light point distribution is similar to the principle of obtaining the light point combination corresponding to the target light point distribution information that most meets the preset distance distribution condition and the preset angle distribution condition among the multiple target light point distribution information in the foregoing embodiments. The most matching light point can be determined according to the matching condition between the foregoing position distribution information and the foregoing preset position information and will not be elaborated herein. In this way, the obtained target three-dimensional model can be made more accurate each time, thereby improving the accuracy of obtaining the light points that match the preset light point distribution based on the target three-dimensional model subsequently, making the finally obtained third light point set more accurate, so as to ensure the accuracy of real-time positioning and tracking of the controller.

[0097] Step S460: If the number of light points included in the third light point set is less than the second number, obtain a three-dimensional model that matches the first light point set and the third light point set as the target three-dimensional model, and repeatedly execute the step of mapping the target three-dimensional model onto a two-dimensional plane to obtain a second image, obtaining a light point that matches the preset light point distribution in the second image from the second light point set in the first image, and adding it to the third light point set until the number of light points included in the third light point set is equal to the second number.

[0098] Based on this, it can be determined whether the number of light points included in the third light point set is less than the second number. If it is less, it indicates that not all the light points in the first image have been acquired. Then, based on the first light point set and the third light point set, obtain the three-dimensional model corresponding to the controller, and repeatedly execute the step of mapping the three-dimensional model onto a two-dimensional plane to obtain a second image, obtaining a light point that matches the preset light point distribution in the second image from the second light point set in the first image, and adding it to the third light point set until the number of light points included in the third light point set is equal to the second number. In this way, only one light point is acquired each time, added to the third light point set, and then the corresponding target three-dimensional model is determined based on the third light point set and the first light point set, improving the accuracy of the determined target three-dimensional model. Furthermore, it also improves the accuracy of subsequently obtaining the third light point set based on the target three-dimensional model. That is to say, a light point set with a sufficient number and a more accurate distribution is obtained, making the pose information of the controller determined based on the light point set more accurate, and improving the accuracy of positioning and tracking the controller.

[0099] In some embodiments, if the above steps are repeatedly executed multiple times and the number of light points included in the third light point set is still less than the second number, determine whether the number of times of repeatedly executing the steps has reached a preset number, or determine whether the duration of obtaining the third light point set has reached a preset duration. If the number of times of repeatedly executing the steps has reached the preset number, or the duration of obtaining the third light point set has reached the preset duration, it means that some light points in the first image may be blocked by other objects and the second number of light points cannot be acquired. At this time, the step of determining the pose information of the controller in the real environment based on the first light point set and the third light point set can be directly executed. In this way, the problem of positioning delay caused by the long duration of obtaining the third light point set can be avoided, ensuring that the head-mounted display device can timely obtain the pose information of the controller and realizing the real-time tracking and positioning of the controller.

[0100] Step S470: Determine the pose information of the controller in the real environment based on the first light point set and the third light point set.

[0101] In this embodiment, the specific content in step S470 can refer to the content in the foregoing embodiment and will not be elaborated herein.

[0102] In this embodiment, each time only 1 light point matching the preset light point distribution in the second image is obtained from the second light point set in the first image, added to the third light point set, and then the corresponding target three-dimensional model is determined based on the third light point set and the first light point set. In this way, the accuracy of the determined target three-dimensional model is improved, so that the distribution of the preset light points determined based on the target three-dimensional model is more accurate. Furthermore, it is possible to more accurately identify the light point set matching the preset light point distribution from the first image, making the pose information of the controller determined based on the light point set more accurate and improving the accuracy of positioning and tracking the controller.

[0103] Please refer to Figure 10 , which shows a structural block diagram of a positioning device 500 for a controller provided in an embodiment of the present application. The device 500 may include: an image acquisition module 510, a first light point set acquisition module 520, a three-dimensional model acquisition module 530, a mapping module 540, a third light point set acquisition module 550, and a positioning module 560.

[0104] The image acquisition module 510 is configured to acquire an image including the controller in the real environment as the first image. The controller is provided with a plurality of light emitting units, and the first image includes light points corresponding to the light emitting units.

[0105] The first light point set acquisition module 520 is configured to acquire a first number of light points meeting the preset distribution condition from the first image to obtain a first light point set.

[0106] The three-dimensional model acquisition module 530 is configured to determine, from the three-dimensional model corresponding to the controller, the three-dimensional model matching the first light point set as the target three-dimensional model.

[0107] The mapping module 540 is configured to map the target three-dimensional model to a two-dimensional plane to obtain a second image, and the second image includes preset light points corresponding to all the light emitting units on the controller.

[0108] The third light point set acquisition module 550 is configured to acquire a second number of light points matching the preset light point distribution in the second image from the second light point set in the first image to obtain a third light point set. The second light point set includes other light points in the first image except the first light point set.

[0109] The positioning module 560 is configured to determine the pose information of the controller in the real environment based on the first light point set and the third light point set.

[0110] In some embodiments, the preset distribution condition includes a preset distance distribution condition and a preset angle distribution condition. The first light point set acquisition module 520 may include: a light point combination acquisition unit, a light point distribution acquisition unit, and a first light point set acquisition unit. Among them, the light point combination acquisition unit may be configured to group all the light points in the first image to obtain a plurality of light point combinations, and the number of light points included in each light point combination in the plurality of light point combinations is the first number. The light point distribution acquisition unit may be configured to obtain the distance between adjacent light points in each light point combination in the plurality of light point combinations, and the angle between multiple line segments formed by the adjacent light points, as the light point distribution information corresponding to each light point combination. The first light point set acquisition unit may be configured to obtain, from the plurality of light point combinations, the light point combinations whose light point distribution information simultaneously meets the preset distance distribution condition and the preset angle distribution condition, as the first light point set.

[0111] In this manner, the light point combination acquisition unit may be specifically configured to, if the multiple light emitting units carried on the controller are arranged in multiple rings, group all the light points corresponding to any one of the multiple rings in the first image to obtain the plurality of light point combinations.

[0112] In some embodiments, the third light point set acquisition module 550 may include: a visible light point acquisition unit and a third light point set acquisition unit. Among them, the visible light point acquisition unit may be configured to obtain the visible light points among the preset light points in the second image according to the orientation of the three-dimensional model, and the visible light points are the light points included in the image obtained by photographing the controller from the direction facing the orientation. The third light point set acquisition unit may be configured to obtain, from the second light point set, the second number of light points that match the distribution of the visible light points in the second image, to obtain the third light point set.

[0113] In this manner, the light points in the first light point set are used as the first light points, and the light points in the second light point set are used as the second light points. The third light point set acquisition unit may specifically be configured to: acquire the relative position information between each second light point in the second light point set and each first light point in the first light point set, to obtain the position distribution information corresponding to each second light point, where the position distribution information includes the relative position information between each second light point and all the first light points in the first light point set; acquire the visible light points corresponding to the first light point set from the visible light points as the first visible light points; acquire the other visible light points except the first visible light points from the visible light points as the second visible light points; acquire the preset position information between each second visible light point and each first visible light point, to obtain the preset distribution information corresponding to each second visible light point, where the preset distribution information includes the preset position information between each second visible light point and all the first visible light points; and acquire, from the second light point set, the second light points whose position distribution information matches the preset position information, to obtain the third light point set.

[0114] In some embodiments, the positioning device 500 of the controller may further include a size judgment module. The size judgment module may be configured to, after acquiring, from the second light point set in the first image, the second number of light points that match the preset light point distribution in the second image to obtain the third light point set, judge whether the size of each light point in the third light point set meets the preset size condition. If there is a target light point whose size does not meet the preset size condition, remove the target light point from the third light point set, where the target light point is any light point in the third light point set. The positioning module 560 may specifically be configured to determine the pose information of the controller in the real environment based on the first light point set and the third light point set after removing the target light point.

[0115] In some other embodiments, the third light point set acquisition module 550 may specifically be configured to: acquire one light point that matches the preset light point distribution in the second image from the second light point set in the first image and add it to the third light point set; if the number of light points included in the third light point set is less than the second number, acquire the three-dimensional model that matches the first light point set and the third light point as the target three-dimensional model, and repeatedly execute the step of mapping the target three-dimensional model to the two-dimensional plane to obtain the second image to acquiring one light point that matches the preset light point distribution in the second image from the second light point set in the first image and adding it to the third light point set until the number of light points included in the third light point set is equal to the second number.

[0116] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0117] In several embodiments provided in the present application, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0118] In addition, in each embodiment of the present application, each functional module can be integrated in a processing module, or each module can exist physically alone, or two or more modules can be integrated in one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0119] In summary, in the solution provided by the embodiments of the present application, an image including a controller in a real environment is obtained as a first image. A plurality of light-emitting units are carried on the controller, and the first image includes light points corresponding to the light-emitting units; a first number of light points that meet a preset distribution condition are obtained from the first image to obtain a first light point set; from the three-dimensional model corresponding to the controller, a three-dimensional model that matches the first light point set is determined as a target three-dimensional model; the target three-dimensional model is mapped to a two-dimensional plane to obtain a second image, and the second image includes preset light points corresponding to all the light-emitting units on the controller; a second number of light points that match the distribution of the preset light points in the second image are obtained from a second light point set in the first image to obtain a third light point set, and the second light point set includes other light points in the first image except the first light point set; based on the first light point set and the third light point set, the pose information of the controller in the real environment is determined. In this way, a sufficient number of light point sets formed by the light-emitting units of the controller can be accurately identified, so that the pose information of the controller determined based on the light point sets is more accurate, that is, the accuracy of positioning and tracking the controller is improved.

[0120] Next, a head-mounted display device provided by the present application will be described with reference to the drawings.

[0121] Refer to Figure 11 , Figure 11 shows a structural block diagram of a head-mounted display device 600 provided by an embodiment of the present application. The positioning method of the controller provided by the embodiment of the present application can be executed by the head-mounted display device 600. Among them, the head-mounted display device 600 can be a device capable of running application programs, etc.

[0122] The head-mounted display device 600 in the embodiments of the present application may include one or more of the following components: a processor 601, a memory 602, and one or more applications. One or more applications may be stored in the memory 602 and configured to be executed by one or more processors 601. One or more programs are configured to execute the methods described in the foregoing method embodiments.

[0123] The processor 601 may include one or more processing cores. The processor 601 connects various parts within the entire head-mounted display device 600 using various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 602, and by calling data stored in the memory 602, the processor 601 executes various functions of the head-mounted display device 600 and processes data. Optionally, the processor 601 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 601 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, and applications, etc.; the GPU is responsible for rendering and drawing display content; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may also be integrated into the processor 901 and implemented separately through a communication chip.

[0124] The memory 602 may include random access memory (RAM) and may also include read-only memory. The memory 602 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 602 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for implementing at least one function (such as touch function, sound playback function, image playback function, etc.), and instructions for implementing the following method embodiments, etc. The data storage area may also store data created during the use of the head-mounted display device 600 (such as the above various corresponding relationships), etc.

[0125] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0126] In several embodiments provided by the present application, the coupling, direct coupling or communication connection between the displayed or discussed modules may be through some interfaces, and the indirect coupling or communication connection of the devices or modules may be in electrical, mechanical or other forms.

[0127] In addition, each functional module in each embodiment of the present application may be integrated in a processing module, or each module may exist physically alone, or two or more modules may be integrated in one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0128] Please refer to Figure 12 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Program code is stored in the computer-readable medium 700, and the program code can be called by a processor to execute the method described in the above method embodiment.

[0129] The computer-readable storage medium 700 may be an electronic memory such as a flash memory, EEPROM (electrically erasable programmable read-only memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 700 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 700 has a storage space for the program code 710 that executes any method step in the above method. These program codes can be read out from or written into one or more computer program products. The program code 710 may be compressed in a suitable form, for example.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A positioning method for a controller, characterized in that, the method includes: Obtain an image containing the controller in the real environment as the first image. Multiple light-emitting units are carried on the controller, and the first image includes light points corresponding to the light-emitting units; Obtain a first number of light points that meet the preset distribution conditions from the first image to obtain a first light point set; Determine a 3D model that matches the first light point set from the 3D model corresponding to the controller as the target 3D model; Map the target 3D model to a two-dimensional plane to obtain a second image, and the second image includes preset light points corresponding to all the light-emitting units on the controller; Obtain a second number of light points that match the distribution of the preset light points in the second image from the second light point set in the first image to obtain a third light point set. The second light point set includes other light points in the first image except the first light point set; Determine the pose information of the controller in the real environment based on the first light point set and the third light point set.

2. The method according to claim 1, characterized in that, the preset distribution conditions include a preset distance distribution condition and a preset angle distribution condition. The step of obtaining a first number of light points that meet the preset distribution conditions from the first image to obtain a first light point set includes: Group all the light points in the first image to obtain multiple light point combinations, and the number of light points included in each light point combination in the multiple light point combinations is the first number; Obtain the distance between adjacent light points in each light point combination in the multiple light point combinations, and the angles between multiple line segments formed by the adjacent light points as the light point distribution information corresponding to each light point combination; Obtain the light point combination whose light point distribution information meets both the preset distance distribution condition and the preset angle distribution condition from the multiple light point combinations as the first light point set.

3. The method according to claim 2, characterized in that, the step of grouping all the light points included in the first image to obtain multiple light point combinations includes: If multiple light-emitting units carried on the controller are arranged in multiple rings, group all the light points corresponding to any one of the multiple rings in the first image to obtain the multiple light point combinations.

4. The method according to any one of claims 1-3, characterized in that, the step of obtaining a second number of light points that match the distribution of the preset light points in the second image from the second light point set in the first image to obtain a third light point set includes: According to the orientation of the target 3D model, obtain the visible light points among the preset light points in the second image. The visible light points are the light points included in the image obtained by photographing the controller from the direction facing the orientation; Obtain a second number of light points that match the distribution of the visible light points in the second image from the second light point set to obtain the third light point set.

5. The method according to claim 4, characterized in that, The light points in the first light point set are used as the first light points, and the light points in the second light point set are used as the second light points. Obtaining a second number of light points that match the visible light point distribution in the second image from the second light point set to obtain the third light point set includes: Obtaining the relative position information between each second light point in the second light point set and each first light point in the first light point set to obtain the position distribution information corresponding to each second light point, where the position distribution information includes the relative position information between each second light point and all the first light points in the first light point set; Obtaining the visible light points corresponding to the first light point set from the visible light points as the first visible light points; Obtaining the other visible light points except the first visible light points from the visible light points as the second visible light points; Obtaining the preset position information between each second visible light point and each first visible light point to obtain the preset distribution information corresponding to each second visible light point, where the preset distribution information includes the preset position information between each second visible light point and all the first visible light points; Obtaining the second light points in the second light point set whose position distribution information matches the preset position information to obtain the third light point set.

6. According to the method described in claim 1, after obtaining a second number of light points that match the preset light point distribution in the second image from the second light point set in the first image to obtain the third light point set, the method further includes: Judging whether the size of each light point in the third light point set meets the preset size condition; If there is a target light point whose size does not meet the preset size condition, removing the target light point from the third light point set, where the target light point is any light point in the third light point set; Determining the pose information of the controller in the real environment based on the first light point set and the third light point set includes: Determining the pose information of the controller in the real environment based on the first light point set and the third light point set after removing the target light point.

7. According to the method described in any one of claims 1-3, characterized in that obtaining a second number of light points that match the preset light point distribution in the second image from the second light point set in the first image to obtain the third light point set includes: Obtaining one light point that matches the preset light point distribution in the second image from the second light point set in the first image and adding it to the third light point set; If the number of light points included in the third light point set is less than the second number, obtain a three-dimensional model that matches the first light point set and the third light point as the target three-dimensional model, and repeatedly execute the steps of mapping the target three-dimensional model to a two-dimensional plane to obtain a second image, obtaining a light point that matches the preset light point distribution in the second image from the second light point set in the first image, and adding it to the third light point set until the number of light points included in the third light point set is equal to the second number.

8. A positioning device for a controller Characterized in that The device includes: An image acquisition module, configured to acquire an image including a controller in a real environment as a first image. A plurality of light-emitting units are carried on the controller, and the first image includes light points corresponding to the light-emitting units; A first light point set acquisition module, configured to acquire a first number of light points that meet a preset distribution condition from the first image to obtain a first light point set; A three-dimensional model acquisition module, configured to determine a three-dimensional model that matches the first light point set from the three-dimensional model corresponding to the controller as the target three-dimensional model; A mapping module, configured to map the target three-dimensional model to a two-dimensional plane to obtain a second image, and the second image includes preset light points corresponding to all the light-emitting units on the controller; A third light point set acquisition module, configured to acquire a second number of light points that match the preset light point distribution in the second image from the second light point set in the first image to obtain a third light point set, where the second light point set includes other light points in the first image except the first light point set; A positioning module, configured to determine the pose information of the controller in the real environment based on the first light point set and the third light point set.

9. A head-mounted display device Characterized in that It includes: One or more processors; A memory; One or more programs, where the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs are configured to execute the method according to any one of claims 1-7.

10. A computer-readable storage medium Characterized in that Program code is stored in the computer-readable storage medium, and the program code can be called by a processor to execute the method according to any one of claims 1-7.

Images