Controller positioning method, device, electronic device and computer storage medium

Abstract

An embodiment of the present application provides a method, apparatus, electronic device, and computer storage medium for controller positioning. The method includes: obtaining a first positioning image collected by a camera and first attitude information of a controller collected by an attitude sensor; determining a position area where the controller is located in the first positioning image according to the position of the image of the positioning identifier in the first positioning image; obtaining the first attitude information collected by the attitude sensor in the controller; determining a target positioning identifier set according to the position area and the first attitude information, and determining first position information and second attitude information of the controller according to the target positioning identifier set and the number of positioning identifiers in the first positioning image; obtaining a positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information. Applying the solution of the present application can solve the problem of large computational complexity during controller positioning.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of spatial positioning, and in particular, to a method and device for positioning a controller, an electronic device, and a computer storage medium. Background Art

[0002] Virtual Reality (VR) enables people in a virtual world to have an immersive feeling through technologies such as three-dimensional technology, simulation technology, and display technology. With the development of social productivity and science and technology, the demand for VR technology is increasing day by day.

[0003] Currently, VR controls actions in the virtual world through a controller. When calculating the pose of the controller, the projection relationship of multiple positioning points is used, and the group with the smallest error is selected by calculating multiple groups of projection relationships, thereby obtaining the pose of the controller.

[0004] However, the amount of calculation when calculating multiple groups of projection relationships is large. For example, if there are 15 positioning points on the controller and the camera captures 5 positioning points, the number of calculations is 15*14*13*12*11. A large amount of calculation results in a large delay when actually using VR, thus affecting the VR experience. Summary of the Invention

[0005] In view of this, the embodiments of the present application provide a method and device for positioning a controller, an electronic device, and a computer storage medium to at least partially solve the above problems.

[0006] According to a first aspect of the embodiments of the present application, a method for positioning a controller is provided. A plurality of positioning identifiers and at least one attitude sensor are provided on the controller. The method includes: obtaining a first positioning image collected by a camera and first attitude information of the controller collected by the attitude sensor, where the first positioning image includes images of at least one of the positioning identifiers; determining a position area where the controller is located in the first positioning image according to the position of the image of the positioning identifier in the first positioning image; determining a target positioning identifier set according to the position area and the first attitude information, where the target positioning identifier set includes at least one positioning identifier, and when the controller is located in the position area and the corresponding attitude information is within a first attitude information interval where the first attitude information is located, the positioning image collected by the camera includes images of at least one of the positioning identifiers in the target positioning identifier set; determining first position information and second attitude information of the controller according to the target positioning identifier set and the number of positioning identifiers in the first positioning image; and obtaining a positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information.

[0007] According to a second aspect of the embodiments of the present application, a controller positioning device is provided. The device includes: an acquisition module, configured to acquire a first positioning image collected by a camera and first attitude information of the controller collected by the attitude sensor, wherein the first positioning image includes images of at least one of the positioning identifiers; a positioning module, configured to determine a position area where the controller is located in the first positioning image according to the position of the image of the positioning identifier in the first positioning image; a determination module, configured to determine a target positioning identifier set according to the position area and the first attitude information, wherein the target positioning identifier set includes at least one positioning identifier, and when the controller is located in the position area and the corresponding attitude information is within a first attitude information interval where the first attitude information is located, the positioning image collected by the camera includes images of at least one of the positioning identifiers in the target positioning identifier set; a calculation module, configured to determine first position information and second attitude information of the controller according to the target positioning identifier set and the number of positioning identifiers in the first positioning image; and an integration module, configured to obtain a positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information.

[0008] According to a third aspect of the embodiments of the present application, an electronic device is provided, including: a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the method described in the first aspect.

[0009] According to a fourth aspect of the embodiments of the present application, a computer storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, it implements the method described in the first aspect.

[0010] According to the controller positioning method provided by the embodiments of the present invention, after a first positioning image is collected by a camera, the area where the controller is located can be obtained according to the position of the image of the positioning identifier included in the first positioning image. Then, according to the area where the controller is located and the first attitude information collected by the sensor in the controller, a target positioning identifier set can be determined. Then, according to the target positioning identifier set and the number of positioning identifiers in the first positioning image, first position information and second attitude information are obtained. Finally, through the first position information and at least one of the first attitude information and the second attitude information, a positioning result of the controller is obtained. It can be seen that in this solution, not all positioning identifiers participate in the calculation, but only some positioning identifiers in the target positioning set participate in the calculation, thereby reducing the calculation amount and further reducing the delay during the use of VR. Description of the Drawings

[0011] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments described in the embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0012] Figure 1 is a flowchart of a method for positioning a controller provided by an embodiment of the present application;

[0013] Figure 2 is a schematic structural diagram of a VR controller provided by an embodiment of the present application;

[0014] Figure 3 is a schematic diagram of the position of the center point of a positioning mark provided by an embodiment of the present application;

[0015] Figure 4 is a schematic diagram of a controller positioning device provided by an embodiment of the present application;

[0016] Figure 5 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0017] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art shall fall within the scope of protection of the embodiments of the present application.

[0018] The terms used in the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "the" and "said" used in the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0019] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to a determination".

[0020] An embodiment of the present application provides a controller positioning method. A plurality of positioning identifiers and at least one attitude sensor are provided on the controller. The method may include:

[0021] Obtain a first positioning image collected by a camera and first attitude information of the controller collected by the attitude sensor. Among them, the first positioning image includes images of at least one of the positioning identifiers;

[0022] Determine the position area where the controller is located in the first positioning image according to the position of the image of the positioning identifier in the first positioning image;

[0023] Determine a target positioning identifier set according to the position area and the first attitude information. The target positioning identifier set includes at least one positioning identifier. When the controller is located in the position area and the corresponding attitude information is in the first attitude information interval where the first attitude information is located, the positioning image collected by the camera includes images of at least one of the positioning identifiers in the target positioning identifier set;

[0024] Determine first position information and second attitude information of the controller according to the target positioning identifier set and the number of positioning identifiers in the first positioning image;

[0025] Obtain the positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information.

[0026] In the embodiment of the present application, after collecting the first positioning image by the camera, the area where the controller is located can be obtained according to the position of the image of the positioning identifier included in the first positioning image. Then, according to the area where the controller is located and the first attitude information collected by the sensor in the controller, the target positioning identifier set can be determined. Then, according to the target positioning identifier set and the number of positioning identifiers in the first positioning image, the first position information and the second attitude information are obtained. Finally, through the first position information and at least one of the first attitude information and the second attitude information, the positioning result of the controller is obtained. It can be seen that in this solution, not all positioning identifiers participate in the calculation, but only some positioning identifiers in the target positioning set participate in the calculation, thereby reducing the calculation amount and further reducing the delay during the use of VR.

[0027] Figure 1 is a flowchart of a controller positioning method provided by an embodiment of the present application. As Figure 1 shown, the method includes the following steps 101 to step 105:

[0028] Step 101: Obtain a first positioning image collected by a camera and first attitude information of a controller collected by an attitude sensor, where the first positioning image includes images of at least one positioning identifier.

[0029] The camera can be set on the VR device or in front of the user. When the user uses the controller, the camera takes a picture of the controller to obtain the first positioning image, which includes the image of the controller and the images of some positioning identifiers on the controller.

[0030] It should be understood that the attitude sensor can be a gyroscope, an accelerometer, an IMU sensor, etc., which is not limited in this application.

[0031] Figure 2 It is a schematic structural diagram of a VR controller provided by an embodiment of this application, as Figure 2 shown, multiple positioning identifiers are set on the VR controller. The positioning identifiers can be evenly distributed on the surface of the VR controller or randomly distributed on the surface of the VR controller. The specific distribution form is not limited in this application.

[0032] It should be understood that the positioning identifier can have various forms, which can be optical marking points, such as visible LED light, infrared LED light, etc., or solid color patches, etc. The positioning identifier only needs to be distinguished from the color of the controller, and the specific form is not limited in this application.

[0033] Step 102: Determine the position area where the controller is located in the first positioning image according to the position of the image of the positioning identifier in the first positioning image.

[0034] The image collected by the camera is divided into n*n regions. For example, dividing 3*3 into 9 regions, or dividing 4*4 into 16 regions, etc. When the controller is located in different position regions relative to the camera, the image of the controller is located in different image regions in the image collected by the camera. After obtaining the first positioning image, the controller is positioned by the image of the positioning identifier on the controller to determine the position area where the controller is located in the first positioning image.

[0035] Step 103: Determine a target positioning identifier set according to the position area and the first attitude information, where the target positioning identifier set includes at least one positioning identifier. When the controller is located in the position area and the corresponding attitude information is within the first attitude information interval where the first attitude information is located, the positioning image collected by the camera includes the images of at least one positioning identifier in the target positioning identifier set.

[0036] Each positioning identifier has a corresponding number. For example, the number of the first positioning identifier is 1, the number of the second positioning identifier is 2, and so on. When the controller is in the position area and the first attitude information interval where the first attitude information is located, all possible positioning identifiers that the camera can capture form the target positioning identifier set.

[0037] Step 104: Determine the first position information and the second attitude information of the controller according to the target positioning identifier set and the number of positioning identifiers in the first positioning image.

[0038] Determine the first position information and the second attitude information according to all the positioning identifiers included in the target positioning identifier set and the number of positioning identifiers included in the first positioning image. The first position information is used to indicate the position area where the controller is located in the first positioning image, and the second attitude information is the attitude information of the controller calculated by an algorithm.

[0039] Step 105: Obtain the positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information.

[0040] It should be understood that the positioning result includes the position information and the attitude information of the controller. The attitude information is determined by at least one of the first attitude information and the second attitude information, and the position information is the first position information.

[0041] In the embodiment of the present application, after the first positioning image is collected by the camera, the area where the controller is located can be obtained according to the position of the image of the positioning identifier included in the first positioning image. Then, according to the area where the controller is located and the first attitude information collected by the sensor in the controller, the target positioning identifier set can be determined. Then, according to the target positioning identifier set and the number of positioning identifiers in the first positioning image, the first position information and the second attitude information are obtained. Finally, through the first position information and at least one of the first attitude information and the second attitude information, the positioning result of the controller is obtained. It can be seen that in this solution, not all positioning identifiers participate in the calculation, but only some positioning identifiers in the target positioning set participate in the calculation, thereby reducing the amount of calculation and further reducing the delay during the use of VR.

[0042] In a possible implementation manner, when determining the position area where the controller is located in the first positioning image according to the position of the image of the positioning identifier in the first positioning image, if the first positioning image includes an image of one positioning identifier, then determine the position area where the controller is located in the first positioning image according to the position of the image of the positioning identifier in the first positioning image. If the first positioning image includes images of at least two positioning identifiers, then determine the position area where the controller is located in the first positioning image according to the position where the center points of the images of the positioning identifiers in the first positioning image are located.

[0043] The first positioning image may contain an image with one positioning identifier or an image with multiple positioning identifiers. When it contains an image with one positioning identifier, the area where the controller is located is determined according to the position of the center point of the image of the positioning identifier. For example, if the positioning identifier is a dot and the center point of the dot is at area 11 of the first positioning image, it is determined that the controller is at area 11. When it contains an image with multiple positioning identifiers, the area where the controller is located is determined according to the central coordinates of the figure formed by connecting the center points of all the images of the positioning identifiers. For example, if there are four positioning identifiers in the first positioning image, the area where the controller is located is determined according to the position of the central coordinates of the quadrilateral formed by the four positioning identifiers in the first positioning image.

[0044] It should be understood that when calculating the center points of the images of the positioning identifiers and the central coordinates of the figure formed by connecting the center points, the geometric center, the gray center, etc. can be calculated, and this application does not make any limitations here.

[0045] In the embodiments of this application, by calculating the center points of the positioning identifiers and the central coordinates of the figure formed by connecting the center points of multiple positioning identifiers, the area where the controller is located is determined, so that the area where the controller is located can be accurately determined, and the positioning accuracy is improved.

[0046] In a possible implementation manner, when determining the target positioning identifier set according to the position area and the first attitude information, after determining the first attitude information interval where the first attitude information is located, the target positioning identifier set corresponding to the position area and the first attitude information interval can be determined from the pre-created index table. The index table includes the corresponding relationship between the image area where the controller is located, the attitude information interval, and the positioning identifier set. When the position and attitude of the controller are in an image area and an attitude information interval, the image collected by the camera includes at least one image of the positioning identifier corresponding to the image area and the attitude information interval.

[0047] The controller is calibrated in advance to form an index table. The position of the camera is fixed as the origin of the coordinate system, the controller is placed in the coordinate system, the z-axis is set to be fixed, the x and y coordinates are traversed at a certain interval to form a position area, and then the Euler angles are traversed at a certain interval to form an attitude interval, and then the possible ranges of the calculated positioning identifiers are corresponded to form an index table. The index table is shown in Table 1 below.

[0048] Table 1

[0049]

[0050]

[0051] It should be understood that the attitude intervals formed after traversing the Euler angles are the angle intervals around the x-axis, the angle interval around the y-axis, and the angle interval around the z-axis. For example: from 5 degrees to 15 degrees around the x-axis, from 5 degrees to 15 degrees around the y-axis, etc. The position regions in the index table correspond to the angles of the three axes and the numbers of the positioning identifiers. For example: in region 11, when it is from 0 degrees to 10 degrees around the x-axis, from 0 degrees to 10 degrees around the y-axis, and from 0 degrees to 10 degrees around the z-axis, the set of positioning identifiers includes positioning identifier 1 to positioning identifier 5. In the same region 11, when it is from 0 degrees to 10 degrees around the x-axis, from 0 degrees to 10 degrees around the y-axis, and from 5 degrees to 15 degrees around the z-axis, the set of positioning identifiers includes positioning identifier 2 to positioning identifier 6, and so on.

[0052] It should also be understood that during the traversal process, the traversal of the Euler angles can be replaced by other rotation traversal methods, and the presentation forms can be quaternions, rotation matrices, axis angles, etc. This application does not make any limitations here.

[0053] Figure 3 It is a schematic diagram of the position of the center point of a positioning identifier provided by an embodiment of this application, as Figure 3 shown. The possible ranges of the positioning identifiers calculated in advance can be obtained through programming calculations or three-dimensional simulations. For example: during three-dimensional animation simulations, different colors represent different numbers, and the number range can be counted according to the colors of the light points in the image, and so on.

[0054] In the embodiment of this application, all possible positioning identifiers in the positioning images captured by the camera in a certain region and a certain attitude interval can be determined from the pre-created index table, so that the range of the positioning identifiers can be reduced, and further the calculation amount during positioning can be reduced.

[0055] In a possible implementation manner, when obtaining the positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information, the first position information and the first attitude information can be determined as the positioning result of the positioner.

[0056] In the embodiment of this application, the first position information and the first attitude information collected by the attitude sensor are determined as the positioning result of the positioner, so as to quickly position the controller, and further improve the use experience of the VR device.

[0057] In a possible implementation manner, when obtaining the positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information, the first position information and the second attitude information can be determined as the positioning result of the positioner.

[0058] In the embodiment of this application, the first position information and the second attitude information obtained through calculation are determined as the positioning result of the positioner, so as to quickly position the controller, and further improve the use experience of the VR device.

[0059] In a possible implementation, when obtaining the positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information, the first attitude information and the second attitude information may be fused and calculated to determine the third attitude information of the controller. The third attitude information and the first position information are determined as the positioning result of the locator.

[0060] When determining the positioning result, the attitude information in the positioning result may be determined according to the first attitude information or the second attitude information. However, since there is a certain offset in the first attitude information collected by the attitude sensor and there is also a certain error in the second attitude information calculated, the first attitude information and the second attitude information may be fused and calculated through a fusion algorithm to obtain the third attitude information, and then the third attitude information is used as the attitude information in the positioning result.

[0061] In the embodiment of the present application, the first attitude information and the second attitude information are fused and calculated to obtain the third attitude information, and the third attitude information is determined as the attitude information in the positioning result, so that more accurate attitude information can be obtained, and further the use experience of the VR device can be improved.

[0062] In a possible implementation, when determining the first position information and the second attitude information of the controller according to the target positioning identifier set and the number of positioning identifiers in the first positioning image, the target positioning identifier set may be used to determine positioning identifier subsets, each positioning identifier subset includes n different positioning identifiers, different positioning identifier subsets include non-identical positioning identifiers, n is used to represent the number of positioning identifiers in the first positioning image, m is used to represent the number of positioning identifiers in the target positioning identifier set, and then according to each positioning identifier subset, the position and attitude of the controller are solved to obtain the solution results of each positioning identifier subset. The target positioning identifier subset with the smallest error of the corresponding solution result is determined from each positioning identifier subset, and the position information and attitude information included in the solution result of the target positioning identifier subset are respectively determined as the first position information and the second attitude information.

[0063] It should be understood that by arranging and combining all the positioning identifiers in the target positioning identifier set according to the number of positioning identifiers in the first positioning image, positioning identifier subsets can be obtained. For example, if the target positioning identifier set contains positioning identifiers 1 to 4 and the number of positioning identifiers in the first positioning image is 3, then there are 4 permutation ways after permutation and combination, and the 4 possible permutation ways are determined as 4 positioning identifier subsets.

[0064] In the embodiment of the present application, all the positioning identifiers in the target positioning identifier set are arranged and combined according to the number of positioning identifiers in the first positioning image to obtain a positioning identifier subset. Then, after solving the positioning identifier subset, the errors of each result are calculated, and the solution result with the smallest error is used as the positioning result. Since the errors are calculated after solving the positioning identifier subset rather than all the positioning identifiers, the computational amount is small.

[0065] In a possible implementation manner, when determining the target positioning identifier subset with the smallest error corresponding to the solution result from each positioning identifier subset, for each positioning identifier subset, the reprojection error between the position and pose indicated by the solution result of the positioning identifier subset and the first positioning image can be calculated, and then the positioning identifier subset with the smallest corresponding reprojection error is determined as the target positioning identifier subset.

[0066] Project the solution result of each positioning identifier subset onto a two-dimensional plane, compare it with the first positioning image, calculate the error between the pixel points in the images of the controller in the two images, and select the solution result corresponding to the positioning identifier subset with the smallest error as the positioning result.

[0067] In the embodiment of the present application, by calculating the errors of the solution results of each positioning identifier subset and then selecting the result with the smallest error as the positioning result, the positioning accuracy is improved, thereby improving the usage experience of the VR device.

[0068] In a possible implementation manner, perform positioning identifier recognition on the first positioning image, count the positioning identifiers in the first positioning image whose corresponding areas are larger than a preset area threshold, and determine the counting result as the number of positioning identifiers in the first positioning image.

[0069] In the first positioning image captured by the camera, there may be images of half or a little positioning identifier at the edge of the image of the locator. The positioning identifiers with image areas smaller than the area threshold do not participate in the calculation.

[0070] It should be understood that in the positioning identifier set of the index table, it includes all the positioning identifiers that the camera may capture within a certain area and within a certain pose range. Therefore, although the image area of the positioning identifier is smaller than the preset area threshold, the positioning identifier is still included in the positioning identifier set.

[0071] In the embodiment of the present application, the positioning identifiers with image areas smaller than the preset area threshold in the first positioning image do not participate in the calculation, further reducing the computational amount.

[0072] As Figure 4 shown, Figure 4 is a schematic diagram of a controller positioning device provided in an embodiment of the present application. The device 400 includes:

[0073] An acquisition module 401 is configured to acquire a first positioning image collected by a camera and first attitude information of a controller collected by an attitude sensor, where the first positioning image includes images of at least one positioning identifier;

[0074] A positioning module 402 is configured to determine a position area where the controller is located in the first positioning image according to the position of the image of the positioning identifier in the first positioning image;

[0075] A determination module 403 is configured to determine a target positioning identifier set according to the position area and the first attitude information, where the target positioning identifier set includes at least one positioning identifier, and when the controller is located in the position area and the corresponding attitude information is within a first attitude information interval where the first attitude information is located, the positioning image collected by the camera includes images of at least one positioning identifier in the target positioning identifier set;

[0076] A calculation module 404 is configured to determine first position information and second attitude information of the controller according to the target positioning identifier set and the number of positioning identifiers in the first positioning image;

[0077] An integration module 405 obtains a positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information.

[0078] In an embodiment of the present application, the acquisition module 401 may be configured to execute step 101 in the above method embodiment, the positioning module 402 may be configured to execute step 102 in the above method embodiment, the determination module 403 may be configured to execute step 103 in the above method embodiment, the calculation module 404 may be configured to execute step 104 in the above method embodiment, and the integration module 405 may be configured to execute step 105 in the above method embodiment.

[0079] In a possible implementation manner, if the first positioning image includes an image of one positioning identifier, the position area where the controller is located in the first positioning image is determined according to the position of the image of the positioning identifier in the first positioning image; if the first positioning image includes images of at least two positioning identifiers, the position area where the controller is located in the first positioning image is determined according to the positions of the centers of the images of the positioning identifiers in the first positioning image.

[0080] In a possible implementation, a first attitude information interval where the first attitude information is located is determined; a target positioning identification set corresponding to the position area and the first attitude information interval is determined from a pre-created index table, where the index table includes the correspondence between the image area where the controller is located, the attitude information interval, and the positioning identification set. When the position and attitude of the controller are in an image area and an attitude information interval, the image collected by the camera includes an image of at least one positioning identification in the positioning identification set corresponding to the image area and the attitude information interval.

[0081] In a possible implementation, the first attitude information and the second attitude information are fused and calculated to determine the third attitude information of the controller; the third attitude information and the first position information are determined as the positioning result of the locator.

[0082] In a possible implementation, according to the target positioning identification set, m positioning identification subsets are determined, where each positioning identification subset includes n different positioning identifications, and different positioning identification subsets include not completely identical positioning identifications. n is used to represent the number of positioning identifications in the first positioning image, and m is used to represent the number of positioning identifications in the target positioning identification set; according to each positioning identification subset, the position and attitude of the controller are solved respectively to obtain the solution result of each positioning identification subset, where the solution result includes the position information and attitude information of the controller; the target positioning identification subset with the smallest error corresponding to the solution result is determined from each positioning identification subset; the position information and attitude information included in the solution result of the target positioning identification subset are respectively determined as the first position information and the second attitude information.

[0083] In a possible implementation, for each positioning identification subset, the reprojection error between the position and attitude indicated by the solution result of the positioning identification subset and the first positioning image is calculated; the positioning identification subset with the smallest corresponding reprojection error is determined as the target positioning identification subset.

[0084] In a possible implementation, positioning identification recognition is performed on the first positioning image, the positioning identifications in the first positioning image with an area greater than a preset area threshold are counted, and the counting result is determined as the number of positioning identifications in the first positioning image.

[0085] It should be noted that for the information interaction, execution process, etc. between the modules in the above controller positioning device, since they are based on the same concept as the foregoing embodiments of the controller positioning method, the specific content can be referred to the description in the foregoing embodiments of the controller positioning method and will not be elaborated here.

[0086] Refer to Figure 5, which shows a schematic structural diagram of an electronic device according to an embodiment of the present application. The specific implementation of the electronic device in the specific embodiments of the present application is not limited.

[0087] As Figure 5 shown, the electronic device may include: a processor 502, a communications interface 504, a memory 506, and a communication bus 508.

[0088] Among them:

[0089] The processor 502, the communications interface 504, and the memory 506 communicate with each other through the communication bus 508.

[0090] The communications interface 504 is used to communicate with other electronic devices or servers.

[0091] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in the above-mentioned embodiment of the controller positioning method.

[0092] Specifically, the program 510 may include program code, and the program code includes computer operation instructions.

[0093] The processor 502 may be a central processing unit CPU, or a graphics processing unit GPU (Graphics Processing Unit), or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the intelligent device may be of the same type of processor, such as one or more CPUs; one or more GPUs; or may be different types of processors, such as one or more CPUs and one or more GPUs and one or more ASICs.

[0094] The memory 506 is used to store the program 510. The memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0095] The program 510 is specifically used to cause the processor 502 to execute the controller positioning method in any of the foregoing embodiments.

[0096] For the specific implementation of each step in Program 510, reference may be made to the corresponding steps and descriptions in the corresponding units in any of the foregoing embodiments of the controller positioning method, which will not be elaborated herein. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the devices and modules described above can refer to the corresponding process descriptions in the foregoing method embodiments, which will not be repeated herein.

[0097] In the embodiment of the present application, after the first positioning image is collected by the camera, the area where the controller is located can be obtained according to the position of the image of the positioning identifier included in the first positioning image. Then, according to the area where the controller is located and the first attitude information collected by the sensor in the controller, the target positioning identifier set can be determined. Then, according to the target positioning identifier set and the number of positioning identifiers in the first positioning image, the first position information and the second attitude information are obtained. Finally, through the first position information and at least one of the first attitude information and the second attitude information, the positioning result of the controller is obtained. It can be seen that in this solution, not all positioning identifiers participate in the calculation, but only some positioning identifiers in the target positioning set participate in the calculation, thereby reducing the calculation amount and further reducing the delay during the use of VR.

[0098] The embodiment of the present application also provides a computer program product, including computer instructions, which instruct a computing device to execute the operations corresponding to any one of the foregoing multiple method embodiments.

[0099] It should be noted that according to the needs of implementation, each component / step described in the embodiment of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiment of the present application.

[0100] The method according to the embodiments of the present application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-transitory machine-readable medium and downloaded via a network and will be stored in a local recording medium, so that the method described herein can be stored as such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (such as RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the controller positioning method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the controller positioning method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the controller positioning method shown herein.

[0101] Those of ordinary skill in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0102] The above embodiments are only used to illustrate the embodiments of the present application, rather than to limit the embodiments of the present application. Those of ordinary skill in the relevant technical field can also make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application, and the patent protection scope of the embodiments of the present application should be defined by the claims.

Claims

1. A method for positioning a controller, wherein a plurality of positioning marks and at least one attitude sensor are provided on the controller, characterized in that, The method includes: Obtaining a first positioning image collected by a camera and first attitude information of the controller collected by the attitude sensor, where the first positioning image includes images of at least one of the positioning identifiers; Determining a position area of the controller in the first positioning image according to a position where an image of the positioning identifier is located in the first positioning image; Determining a target positioning identifier set according to the position area and the first attitude information, where the target positioning identifier set includes at least one positioning identifier, and when the controller is located in the position area and the corresponding attitude information is within a first attitude information interval where the first attitude information is located, the positioning image collected by the camera includes images of at least one of the positioning identifiers in the target positioning identifier set; Determining first position information and second attitude information of the controller according to the target positioning identifier set and the number of positioning identifiers in the first positioning image; Obtaining a positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information.

2. The method according to claim 1, wherein The determining a position area of the controller in the first positioning image according to a position where an image of the positioning identifier is located in the first positioning image includes: If the first positioning image includes an image of one of the positioning identifiers, determining a position area of the controller in the first positioning image according to a position where the image of the positioning identifier is located in the first positioning image; If the first positioning image includes images of at least two of the positioning identifiers, determining a position area of the controller in the first positioning image according to positions where center points of the images of the positioning identifiers in the first positioning image are located.

3. The method according to claim 1, characterized in that, The determining a target positioning identifier set according to the position area and the first attitude information includes: Determining a first attitude information interval where the first attitude information is located; Determining the target positioning identifier set corresponding to the position area and the first attitude information interval from a pre-created index table, where the index table includes a correspondence between an image area where the controller is located, an attitude information interval, and a positioning identifier set, and when the position and attitude of the controller are in an image area and an attitude information interval, the image collected by the camera includes images of at least one of the positioning identifiers in the positioning identifier set corresponding to the image area and the attitude information interval.

4. The method according to claim 1, characterized in that The obtaining a positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information includes: Performing a fusion calculation on the first attitude information and the second attitude information to determine third attitude information of the controller; Determining the third attitude information and the first position information as a positioning result of the positioner.

5. The method according to claim 1, wherein The determining first position information and second attitude information of the controller according to the target positioning identifier set and the number of positioning identifiers in the first positioning image includes: Determine according to the set of target positioning identifiers subsets of positioning identifiers, where each subset of positioning identifiers includes k different positioning identifiers, and different subsets of positioning identifiers include not completely identical positioning identifiers. k is used to represent the number of positioning identifiers in the first positioning image, and r is used to represent the number of positioning identifiers in the set of target positioning identifiers; Solve the position and attitude of the controller respectively according to each of the positioning identification subsets, and obtain the solution results of each of the positioning identification subsets, where the solution results include the position information and attitude information of the controller; Determine a target positioning identification subset with the smallest error corresponding to the solution result from each of the positioning identification subsets; Determine the position information and attitude information included in the solution result of the target positioning identification subset as the first position information and the second attitude information respectively.

6. The method according to claim 5, wherein The determining a target positioning identification subset with the smallest error corresponding to the solution result from each of the positioning identification subsets includes: For each of the positioning identification subsets, calculate the reprojection error between the position and attitude indicated by the solution result of the positioning identification subset and the first positioning image; Determine the positioning identification subset with the smallest corresponding reprojection error as the target positioning identification subset.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Perform positioning identification on the first positioning image, count the positioning identifications in the first positioning image whose corresponding area is greater than a preset area threshold, and determine the counting result as the number of positioning identifications in the first positioning image.

8. A controller positioning device, characterized in that, The device includes: An acquisition module, configured to acquire a first positioning image collected by a camera and first attitude information of the controller collected by an attitude sensor, where the first positioning image includes images of at least one positioning identification; A positioning module, configured to determine a position area where the controller is located in the first positioning image according to the position of the image of the positioning identification in the first positioning image; A determination module, configured to determine a target positioning identification set according to the position area and the first attitude information, where the target positioning identification set includes at least one positioning identification, and when the controller is located in the position area and the corresponding attitude information is within a first attitude information interval where the first attitude information is located, the positioning image collected by the camera includes images of at least one positioning identification in the target positioning identification set; A calculation module, configured to determine first position information and second attitude information of the controller according to the target positioning identification set and the number of positioning identifications in the first positioning image; An integration module, configured to obtain a positioning result of the controller according to the first position information and at least one of the first attitude information and the second attitude information.

9. An electronic device, comprising: A processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the controller positioning method according to any one of claims 1-7.

10. A computer storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the controller positioning method according to any one of claims 1-7.

Images