Equipment positioning method and device and computer readable storage medium

Abstract

The invention provides an equipment positioning method and apparatus, and a computer readable storage medium. The method comprises the steps of obtaining space coordinate information and first plane coordinate information of each positioning mark; and determining attitude information of the target equipment according to the space coordinate information of each positioning mark and the first plane coordinate information. According to the method, space coordinates of positioning marks on equipment and first plane coordinates of the positioning marks are obtained, and each positioning mark corresponds to one space coordinate and one first plane coordinate, so that one space coordinate corresponds to one first plane coordinate. Therefore, the position information of each positioning mark can be calculated according to the two coordinates under the corresponding relation. Therefore, by determining the position information of the positioning mark, the attitude information of the equipment can be effectively determined, and after the attitude information of the equipment is determined, projection work can be better carried out subsequently according to the attitude information of the equipment.

Description

Technical Field

[0001] This application relates to the field of virtual reality technology, and particularly to a device positioning method, apparatus, and computer-readable storage medium. Background Art

[0002] VR (Virtual Reality) virtual reality technology allows users to experience the real world in virtual applications. In desktop VR technology, it is necessary to track the user's perspective in real time. By capturing the user's perspective, the display of the desktop device is controlled to present a real 3D experience to the user. Therefore, in the process of tracking the user's perspective, it is necessary to effectively position the pose of the user's head-mounted VR device. Thus, the problem of how to effectively determine the pose of the user's VR device needs to be overcome. Summary of the Invention

[0003] This application provides a device positioning method that can effectively position the attitude information of the VR device and reduce the projection error.

[0004] In a first aspect, this application provides a device positioning method. Multiple positioning markers are set on the target device. The method includes:

[0005] Obtain the spatial coordinate information and first plane coordinate information of each of the positioning markers;

[0006] Determine the attitude information of the target device according to the spatial coordinate information and the first plane coordinate information of each of the positioning markers.

[0007] In a second aspect, this application further provides a device positioning apparatus. Multiple positioning markers are set on the target device. The apparatus includes:

[0008] An obtaining module, configured to obtain the spatial coordinate information and first plane coordinate information of each of the positioning markers;

[0009] A determining module, configured to determine the attitude information of the target device according to the spatial coordinate information and the first plane coordinate information of each of the positioning markers;

[0010] Preferably, the determining module is specifically configured to:

[0011] Convert the spatial coordinate information into second plane coordinate information;

[0012] Determine the attitude information of the target device according to the second plane coordinate information and the first plane coordinate information;

[0013] Preferably, the determining module is further specifically configured to:

[0014] Determine the coordinate information conversion factor between the spatial coordinate information and the second plane coordinate information;

[0015] Determine the product result of the spatial coordinate information and the coordinate information conversion factor;

[0016] Determine the converted second plane coordinate information according to the product result;

[0017] Preferably, the determining module is further specifically configured to:

[0018] Divide the coordinate information in the second plane coordinate information into coordinate information to be verified and verification plane coordinate information;

[0019] Determine multiple candidate attitude information of the target device according to the coordinate information to be verified;

[0020] Determine the first plane coordinate information corresponding to each of the second plane coordinate information;

[0021] Determine the first mapped plane coordinate information corresponding to the verification plane coordinate information in the first plane coordinate information according to the first plane coordinate information corresponding to each of the second plane coordinate information;

[0022] Determine the coordinate information position error between the verification plane coordinate information and the first mapped plane coordinate information;

[0023] Determine one of the candidate attitude information in each of the candidate attitude information as the attitude information of the target device according to the coordinate information position error;

[0024] Preferably, the determining module is further specifically configured to:

[0025] Determine multiple preliminary candidate attitude information of the target device according to the coordinate information to be verified;

[0026] Analyze each of the preliminary candidate attitude information, determine the target preliminary candidate attitude information to be filtered, and obtain multiple candidate attitude information of the target device;

[0027] Preferably, the determining module is further specifically configured to:

[0028] Each of the spatial coordinate information corresponds to a second plane coordinate information and a first plane coordinate information. According to the second plane coordinate information and the first plane coordinate information respectively corresponding to each spatial coordinate information, determine the first plane coordinate information corresponding to each second plane coordinate information;

[0029] Determine the first plane coordinate information corresponding to each of the verification plane coordinate information according to the first plane coordinate information corresponding to each second plane coordinate information, and obtain the first mapped plane coordinate information;

[0030] Preferably, the determining module is further specifically configured to:

[0031] Determine the target space coordinate information corresponding to the verified plane coordinate information;

[0032] According to the target space coordinate information, the rotation information, and the position information, determine the coordinate position error between the verified plane coordinate information and the first mapped plane coordinate information;

[0033] Preferably, the determining module is further specifically configured to:

[0034] According to the attitude information, control the target device to perform image display.

[0035] In a third aspect, the present application further provides a device positioning device, where the device positioning device includes a plurality of positioning markers, a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps in any one of the device positioning methods described above.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to implement the steps in any one of the device positioning methods described above.

[0037] The device positioning method provided by the present application obtains the space coordinates of the positioning markers on the target device and the first plane coordinates of the positioning markers. At the same time, since each positioning marker corresponds to a space coordinate and a first plane coordinate, a space coordinate also corresponds to a first plane coordinate. In other words, there is a corresponding relationship between the space coordinates and the first plane coordinates. Therefore, the position information of each positioning marker can be calculated based on these two coordinates under the corresponding relationship. Moreover, since the positioning markers are set on the target device, after determining the position information of the positioning markers, the attitude information of the target device can be effectively determined. Furthermore, after determining the attitude information of the target device, subsequent projection work can be better carried out according to the attitude information of the target device. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those skilled in the art can obtain other drawings without creative efforts based on these drawings.

[0039] Figure 1 It is a schematic diagram of the scenario of the device positioning system provided in the embodiments of the present application;

[0040] Figure 2 It is a schematic flowchart of an embodiment of the device positioning method in an embodiment of the present application;

[0041] Figure 3 It is a schematic diagram of a functional module of the device positioning device in an embodiment of the present application;

[0042] Figure 4 It is a schematic structural diagram of glasses in an embodiment of the present application. Detailed implementation manners

[0043] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0044] In the description of the present application, it should be understood that the terms "first" and "second" are only used for descriptive purposes, and cannot be construed as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present application, "a plurality" means two or more, unless otherwise specifically defined.

[0045] In the present application, the term "exemplary" is used to mean "serving as an example, illustration, or description". Any embodiment described as "exemplary" in the present application is not necessarily construed as being more preferred or having more advantages than other embodiments. At the same time, it can be understood that in the specific implementation manners of the present application, when it comes to relevant data such as user information and user data, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards of relevant countries and regions.

[0046] In order to enable any person skilled in the art to implement and use the present application, the following description is given. In the following description, details are set forth for purposes of explanation. It should be understood that those skilled in the art can realize the present application without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope that conforms to the principles and features disclosed in the present application.

[0047] The present application provides a device positioning method, device, glasses, and storage medium, which will be described in detail below.

[0048] Please refer to Figure 1 , Figure 1 , which is a schematic diagram of the scenario of the device positioning system provided by the embodiment of the present application. The device positioning system may include a VR device 100 and an image sensor 200, and the image sensor 200 may transmit image data to the VR device 100. As shown in Figure 1 the VR device 100 in, the image data of the positioning mark on the VR device 100 captured by the image sensor 200 can be obtained to execute the device positioning method in the present application.

[0049] In the embodiment of the present application, the VR device 100 may be any kind of glasses including functions such as data processing and image projection, and the specific embodiment of the present application is not limited.

[0050] In the embodiment of the present application, communication between the VR device 100 and the image sensor 200 can be achieved through any communication method, including but not limited to mobile communication based on the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), Worldwide Interoperability for Microwave Access (WiMAX), or computer network communication based on the TCP / IP Protocol Suite (TCP / IP), User Datagram Protocol (UDP), etc.

[0051] It should be noted that Figure 1 the schematic diagram of the scenario of the device positioning system shown is only an example. The device positioning system and scenario described in the embodiment of the present application are for more clearly explaining the technical solution of the embodiment of the present application, and do not constitute a limitation to the technical solution provided by the embodiment of the present application. Those skilled in the art know that with the evolution of the device positioning system and the emergence of new service scenarios, the technical solution provided by the embodiment of the present application is equally applicable to similar technical problems.

[0052] As shown in Figure 2 , Figure 2 which is a schematic diagram of the process of an embodiment of the device positioning method in the embodiment of the present application. A plurality of positioning marks are set on the target device, and the device positioning method may include the following steps 201 to 202:

[0053] 201. Obtain the spatial coordinate information and the first plane coordinate information of each positioning mark.

[0054] In some actual application scenarios, when a user wears a head-mounted VR device, such as VR glasses, the projection technology requires a special projector and a transparent screen. This screen is usually placed outside the user's glasses, allowing the user to see the real world while also seeing the projected virtual reality scene. The projector projects the virtual reality scene through a 3D image generated by a computer, and the user wears VR glasses to view and experience this scene. The projected scene can be in the fields of games, movies, education, etc.

[0055] In the real world, what the human eye can see depends on the direction of the human eye's observation. When the observation direction of the human eye changes, the physical objects randomly viewed also change. Specifically, as the human eye moves with the head and body, the line of sight changes, resulting in a change in the physical objects observed. Similarly, in virtual reality, at this time, the VR glasses replace the human eye. After the user wears the VR glasses, as the user's head swings, body rotates, and other actions, the orientation of the glasses worn by the user can be changed, that is, the posture of the glasses changes. If the picture presented in the glasses needs to match the user's actions, it is necessary to determine the posture information of the glasses, so as to be able to present the corresponding picture in the glasses. For this reason, it is necessary to effectively calculate the posture information of the glasses, so as to be able to select the picture that matches this posture information and present it on the glasses. For example: Before the game starts, the orientation of the user's glasses can be calibrated, such as horizontally forward. At this time, the initial picture is called and presented in the user's glasses. If the user's head then rotates 5 degrees to the right horizontally and at the same time looks up 5 degrees upward, the initial picture is adjusted 5 degrees to the left and 5 degrees downward. At this time, the user can simulate the action of turning the head in reality. Similarly, if the user moves forward a certain distance, the picture is enlarged at this time, then the real feeling of the user moving forward can be matched. Thus, it can be seen that determining the posture, position and other information of the glasses is crucial for the projection of the picture.

[0056] In the embodiments of the present application, positioning marks can be set on the glasses, and the light reflected by the positioning marks is stronger. When the stronger reflected light of the positioning marks is obtained, the position information of the positioning marks can be determined, that is, the position information of the glasses can be determined. For example: If 2 positioning marks are installed on the glasses, and two points form a straight line, at this time, by measuring the posture of the straight line, the posture of the glasses can be determined. Of course, in order to be able to more accurately determine the posture of the glasses, multiple positioning marks can be set, such as 5, 10, etc. Specifically, the embodiments of the present application do not make any limitations. Among them, the material of the positioning marks can be any material with strong reflective ability, such as fluorescent material, etc. At the same time, the shape of the positioning marks can be any shape, such as: triangle, rhombus, square, circle, etc. Specifically, the embodiments of the present application do not make any limitations.

[0057] In addition, in the embodiments of the present application, the spatial coordinates of the glasses can be obtained by means of an image sensor. For example, whenever the user uses the glasses, it is necessary to place the supporting image sensor in the room. For example, an image sensor is placed on both the front and side of the room so that the two image sensors can capture the interior of the room. When the user wears the glasses and appears in the room, the image sensor can capture the positioning marks on the glasses, thereby determining the spatial coordinates of the positioning marks in this space. It should be noted that there are as many spatial coordinates as there are positioning marks, and one positioning mark corresponds to one spatial coordinate.

[0058] In order for the user to use the glasses better, when the user starts using the glasses, the initial position of the user can be calibrated. For example, an initial program is set in the glasses, including setting fixed calibration parameters and fixing the initial attitude information of the user's glasses. Preferably, a corresponding position calibration device can be set, such as a signal receiver and a signal transmitter. When the user uses the glasses, it is necessary to place the signal receiver in the room, such as in the corner of the room and other positions. At this time, after the user wears the glasses, the signal transmitter on the glasses can emit a signal. And the signal receiver can determine the relative distance between the two according to the signal emitted by the signal transmitter. At the same time, information such as a moving arrow can appear in the picture in the glasses to guide the user's head to drive the glasses to directly face the signal transmitter. In this way, when the distance between the signal transmitter and the signal receiver meets the requirements and the user has completed the direct viewing of the signal receiver, the initial position calibration of the glasses can be completed at this time. It should be noted that the signal receiver and the signal transmitter in the embodiments of the present application can be any type of sensor, and the specific embodiments of the present application do not make limitations. At the same time, the user's initial position can also be calibrated by an image sensor. For example, the spatial coordinates of the positioning marks are located through the image sensor, and through the spatial coordinates, a moving arrow is presented in the glasses to guide the user to determine the initial position of the glasses.

[0059] In the embodiments of the present application, when the user starts using the glasses, any target plane can be used as the first plane. It should be noted that the first plane needs to be perpendicular to the ground. After determining the first plane, a first plane coordinate system can be constructed according to the first plane. After obtaining the spatial coordinates, the spatial coordinates are projected and calculated onto the first plane, and the first plane coordinates of the positioning marks on the first plane coordinate system can be obtained.

[0060] 202. Determine the attitude information of the target device according to the spatial coordinate information and the first plane coordinate information of each positioning mark.

[0061] According to the above embodiment, since the first plane coordinates can be obtained by projecting the spatial coordinates. Therefore, there is a mapping relationship between a spatial coordinate and a first plane coordinate after projection, and the mapping relationship is the corresponding relationship in the embodiment of the present application. After the projection calculation is completed, the spatial coordinates corresponding to each positioning mark and the corresponding first plane coordinates are solved, and a displacement posture information of the glasses can be obtained, for example, the current horizontal orientation angle and the vertical orientation angle of the glasses are determined, and then the specific direction of the glasses is determined according to the horizontal orientation angle and the vertical orientation angle, so as to select the picture matching the specific orientation direction and present it in the glasses. It should be noted that the method for solving the posture information of the glasses in the embodiment of the present application can refer to any coordinate positioning method, and the specific embodiments of the present application will not be repeated.

[0062] After the posture information of the target device is acquired according to step 202, for example, the posture information of the glasses is acquired, the following scheme may also be included: according to the posture information, the target device is controlled to display an image.

[0063] Since the target device can be a VR glasses, when the posture information of the target device represents that the glasses have changed in posture such as movement or rotation, for example, a displacement of 0.5 meters and a rotation of 10 degrees to the right, the image in the glasses can be controlled to move proportionally to keep the image movement in the glasses matching the actual posture movement of the glasses.

[0064] The device positioning method provided by the present application obtains the spatial coordinates of the positioning mark on the glasses and the first plane coordinates of the positioning mark. At the same time, since each positioning mark corresponds to a spatial coordinate and a first plane coordinate, a spatial coordinate also corresponds to a first plane coordinate. In other words, there is a corresponding relationship between the spatial coordinates and the first plane coordinates. Therefore, the position information of each positioning mark can be calculated based on the two coordinates under the corresponding relationship. Moreover, the positioning mark is set on the glasses, so by determining the position information of the positioning mark, the posture information of the glasses can be effectively determined, and after determining the posture information of the glasses, the subsequent projection work can be better performed according to the posture information of the glasses.

[0065] In order to better implement the embodiment of the present application, in one embodiment of the present application, determining the posture information of the glasses according to the spatial coordinates and the first plane coordinates includes:

[0066] The spatial coordinates are converted into second plane coordinates on a second plane coordinate system; and the posture information of the glasses is determined according to the second plane coordinates and the first plane coordinates.

[0067] The above embodiments provide an implementation manner for solving the attitude information of glasses directly according to the correspondence between spatial coordinates and first plane coordinates. In order to solve the attitude information of glasses more simply, in the embodiments of the present application, the spatial coordinates of the positioning marks can be converted into plane coordinates on another plane, and the attitude information of the glasses can be determined by comparing the differences between the two plane coordinates.

[0068] Among them, in the embodiments of the present application, the second plane coordinate system is different from the first plane coordinate system, that is, the second plane coordinate system corresponds to the second plane. Specifically, when the spatial coordinates are obtained, any two planes can be selected, that is, the spatial coordinates are projected onto the two planes at the same time to determine the projected coordinates of the spatial coordinates on the two planes. Moreover, the second plane is also perpendicular to the ground, and the method of selecting the second plane can also be randomly selected, and the specific embodiments of the present application do not make any limitations.

[0069] At the same time, both the second plane coordinates and the first plane coordinates are obtained by projecting the spatial coordinates. Therefore, one second plane coordinate corresponds to one spatial coordinate, and one first plane coordinate corresponds to one spatial coordinate. Furthermore, the first plane coordinate corresponding to each second plane coordinate can be determined, that is, the correspondence between the second plane coordinates and the first plane coordinates can be obtained.

[0070] After obtaining the first plane coordinates and the second plane coordinates of each positioning mark, the attitude information of the glasses can be obtained by calculating the coordinate differences between the first plane coordinates and the second plane coordinates that are in a corresponding relationship in the two plane coordinates. Compared with the calculation of the attitude information between the spatial coordinates and the plane coordinates, the calculation between the plane coordinates can omit the calculation parameters of the z-axis, which is convenient for solving the attitude information of the glasses.

[0071] In order to better implement the embodiments of the present application, in one embodiment of the present application, converting the spatial coordinates into the second plane coordinates on the second plane coordinate system includes:

[0072] Determine the coordinate conversion factor between the spatial coordinates and the second plane coordinate system; according to the coordinate conversion factor, convert the spatial coordinates into the second plane coordinates.

[0073] The above embodiments provide an implementation manner for converting spatial coordinates into second plane coordinates by projection. In the embodiments of the present application, the spatial coordinates can also be converted into second plane coordinates by other methods. Specifically, as shown in the following formula (1):

[0074] p gi = T cg -1 · p ci ……(1)

[0075] Among them, p giis the second planar coordinate after transformation; p ci is the spatial coordinate; T cg is the transformation factor, specifically, it can be the transformation matrix between the spatial coordinate and the planar coordinate; i represents the ordinal number of the positioning mark. Among them, in the embodiments of the present application, the transformation factor T cg can be set according to the actual situation, and specifically, it is not limited in the embodiments of the present application.

[0076] To better implement the embodiments of the present application, in one embodiment of the present application, according to the second planar coordinate and the first planar coordinate, the attitude information of the glasses is determined, including:

[0077] Divide the coordinates in the second planar coordinate into coordinates to be verified and verification coordinates; determine multiple candidate attitude information of the glasses according to the coordinates to be verified; determine the first planar coordinates corresponding to each of the second planar coordinates; determine the first mapped planar coordinates corresponding to the verification planar coordinates in the first planar coordinate system according to the first planar coordinates corresponding to each of the second planar coordinates; determine the coordinate position error between the verification coordinates and the first mapped planar coordinates; determine one of the candidate attitude information as the attitude information of the glasses according to the coordinate position error.

[0078] To minimize the error of the obtained attitude information of the glasses, the embodiments of the present application also provide a calculation method for calculating the attitude information. Specifically, assume that there are 5 positioning marks, then there are also 5 second planar coordinates. After obtaining the second planar coordinates of each positioning mark, any three second planar coordinates can be regarded as coordinates to be verified, and the remaining two second planar coordinates can be regarded as verification coordinates. Of course, the number of coordinates to be verified and verification coordinates can be set according to the actual situation. At this time, since the number of coordinates to be verified is less than the number of second planar coordinates, the error of solving the attitude information of the glasses only according to the coordinates to be verified is relatively large. In other words, if only the coordinates to be verified are used to solve the attitude information of the glasses, there may be multiple solutions of attitude information, that is, there are multiple candidate attitude information. Therefore, it is necessary to determine a target attitude information from each candidate attitude information as the actual attitude information of the glasses. At the same time, the verification coordinates are also second planar coordinates. At this time, according to the corresponding relationship, the first planar coordinates corresponding to the verification coordinates can be determined to obtain the first mapped planar coordinates.

[0079] After that, according to the verification coordinates and the first mapped planar coordinates, the first coordinate error between the two is determined, and the second coordinate error between the coordinates to be verified corresponding to each candidate attitude information and the corresponding first planar coordinates is obtained, and the second coordinate error closest to the first coordinate error is obtained, and the candidate attitude information corresponding to the closest second coordinate error is the actual attitude information of the glasses.

[0080] In order to better implement the embodiments of the present application, in one embodiment of the present application, according to the coordinate information to be verified, multiple candidate pose information of the target device is determined, including:

[0081] According to the coordinate information to be verified, multiple preliminary candidate pose information of the target device is determined; each preliminary candidate pose information is analyzed to determine the target preliminary candidate pose information to be filtered, and multiple candidate pose information of the target device is obtained.

[0082] In the actual calculation process, when solving the pose information of the target device, that is, the glasses, according to each coordinate information to be verified, some of the multiple candidate pose information obtained will be abnormal candidate pose information. Similar to solving a quadratic equation of one variable, negative solutions may be obtained. Based on the actual situation, the pose information of the glasses in space does not include negative angles or negative displacements. Therefore, it is necessary to filter out the candidate pose information with negative values. Therefore, in the embodiments of the present application, when solving the pose information of the glasses for each coordinate information to be verified, multiple preliminary candidate pose information will be obtained. After obtaining the preliminary candidate pose information, the sign of the value of each preliminary candidate pose information can be determined, and the preliminary candidate pose information with a negative sign is deleted, and the preliminary candidate pose information with a positive sign is retained, and the retained preliminary candidate pose information is the multiple candidate pose information of the target device. It should be noted that in the embodiments of the present application, the preliminary candidate pose information with a negative sign can be characterized as the pose information of the glasses facing away from the camera, and the pose information with a negative depth of the glasses, etc.

[0083] In order to better implement the embodiments of the present application, in one embodiment of the present application, the candidate pose information includes the rotation information and position information of the glasses, and determining the coordinate position error between the verification coordinate and the first mapped plane coordinate includes:

[0084] Determine the target space coordinate corresponding to the verification coordinate; according to the target space coordinate, rotation information, and position information, determine the coordinate position error between the verification coordinate and the first mapped plane coordinate.

[0085] The above embodiments provide an implementation manner of determining the actual pose information of the glasses according to the coordinate error. The embodiments of the present application also provide another implementation manner, specifically including: the coordinate position error between the verification coordinate and the first mapped plane coordinate can be determined according to formula (2) and formula (3). The specific formula (2) and formula (3) are as follows:

[0086] e i =sqrt((u i -p1 i [0]) 2 +(v i -p1i [1]) 2 ……(2)

[0087] p1 i = k·(rcg·p i + tcg) / tcg[2]……(3)

[0088] Wherein, [rcg, tcg] is candidate pose information, and the candidate pose information is calculated according to the coordinates to be verified. rcg can be a 3x3 matrix representing the rotation information of the glasses relative to the camera, and tcg is a 3x1 matrix that can represent the position information of the glasses translated relative to the camera; p i is the spatial coordinate of the i-th positioning mark; k is a fixed parameter matrix; u i and v i are the i-th verification coordinates (u i , v i ); e i is the coordinate position error.

[0089] Since the coordinates to be verified are different, the verification coordinates are also different. Therefore, according to the above formulas (2) and (3), the candidate pose information obtained according to different coordinates to be verified is also different, and thus the obtained coordinate position error e i is also different. In the embodiment of the present application, after obtaining multiple coordinate position errors, the coordinate position error with the smallest value is selected as the output, and the rcg and tcg of the coordinate position error with the smallest value are determined as the pose information output of the glasses.

[0090] To better implement the embodiment of the present application, in an embodiment of the present application, obtaining the spatial coordinate information of each positioning mark includes:

[0091] Controlling the first image sensor to capture each positioning mark on the glasses to obtain image data of different angles of each positioning mark; and obtaining the spatial coordinates of each positioning mark according to the image data of different angles of each positioning mark.

[0092] The above embodiment provides an implementation manner of obtaining the spatial coordinate information of the positioning mark according to two image sensors. The embodiment of the present application also provides a solution for obtaining the spatial coordinates of the positioning mark according to one image sensor.

[0093] Specifically, the first image sensor can be a binocular camera, which can simulate the human eyes. The human eyes determine the distance and direction of an object based on the different views of the object observed by the two eyes, thereby determining the parallax of the object (i.e., the difference between the object images at two angles, which is the image data at different angles), and then determining the specific position of the object. Similarly, based on the parallax of the positioning mark determined by the binocular camera, the spatial coordinates of the positioning mark can be better determined.

[0094] In order to better implement the embodiments of the present application, in an embodiment of the present application, obtaining the first plane coordinate information of each positioning mark includes:

[0095] Controlling the second image sensor to capture each positioning mark on the glasses to obtain the target captured images of each positioning mark; constructing the first plane coordinate information based on the target captured images to obtain the first plane coordinate information of each positioning mark.

[0096] The above embodiment provides an implementation manner of projecting the spatial coordinates to obtain the first plane coordinates. The present application also provides another implementation manner. Since the shapes of the positioning marks can be different, such as circular, square, diamond, etc., and of course can also be characters, including 1, 2, 3, etc. Therefore, in the embodiments of the present application, the outer shape of the positioning mark can be set as Roman numerals. At this time, the second image sensor can be placed at a fixed position to obtain the first plane coordinates of the positioning mark. The second image sensor can be any monocular camera. After the first image sensor obtains the spatial coordinates of the positioning mark, since the shape of each positioning mark is different, each shape corresponds to a spatial coordinate. At the same time, when the second image sensor obtains the first plane coordinates of each positioning mark, after the second image sensor captures each positioning mark, the target captured images of each positioning mark can be obtained. At this time, the shapes of each positioning mark can be displayed on the target captured images, and then each positioning mark can be distinguished according to the shape, so that the spatial coordinates and the first plane coordinates corresponding to each positioning mark can be determined. It should be noted that the coordinates and coordinate information in the embodiments of the present application are all coordinates.

[0097] In order to better implement the device positioning method in the embodiments of the present application, on the basis of the device positioning method, an embodiment of the present application also provides a device positioning device, and a plurality of positioning marks are arranged on the device positioning device, such as Figure 3 As shown, the device 300 includes:

[0098] An obtaining module 301, configured to obtain the spatial coordinate information and the first plane coordinate information of each of the positioning marks;

[0099] A determination module 302, configured to determine the attitude information of the target device according to the spatial coordinate information of each positioning mark and the first plane coordinate information.

[0100] The device positioning device provided by this application can obtain the spatial coordinates of the positioning marks on the glasses and the first plane coordinates of the positioning marks through the acquisition module 301. At the same time, since each positioning mark corresponds to a spatial coordinate and a first plane coordinate, a spatial coordinate also corresponds to a first plane coordinate. In other words, there is a corresponding relationship between the spatial coordinate and the first plane coordinate. Therefore, the two coordinates under this corresponding relationship can be determined by the determination module 302, and the position information of each positioning mark can be calculated. Moreover, since the positioning marks are set on the glasses, after determining the position information of the positioning marks, the attitude information of the glasses can be effectively determined. Furthermore, after determining the attitude information of the glasses, subsequent projection work can be better carried out according to the attitude information of the glasses.

[0101] In some embodiments of this application, the determination module 302 is specifically configured to:

[0102] Convert the spatial coordinates into second plane coordinates;

[0103] Determine the attitude information of the glasses according to the second plane coordinates and the first plane coordinates.

[0104] In some embodiments of this application, the determination module 302 is specifically further configured to:

[0105] Determine the coordinate conversion factor between the spatial coordinates and the second plane coordinate system;

[0106] Determine the product result of the spatial coordinate information and the coordinate information conversion factor;

[0107] Determine the converted second plane coordinate information according to the product result.

[0108] In some embodiments of this application, the determination module 302 is specifically further configured to:

[0109] Divide the coordinates in the second plane coordinates into coordinates to be verified and verification coordinates;

[0110] Determine multiple candidate attitudes of the glasses according to the coordinates to be verified;

[0111] Determine the first plane coordinates corresponding to each second plane coordinate;

[0112] Determine the first mapped plane coordinates corresponding to the verification plane coordinates in the first plane coordinate system according to the first plane coordinates corresponding to each second plane coordinate;

[0113] Determine the coordinate position error between the verification coordinates and the first mapped plane coordinates;

[0114] Determine one candidate pose information among the candidate pose information as the pose information of the glasses according to the coordinate position error.

[0115] In some embodiments of the present application, the determining module 302 is specifically further configured to:

[0116] Determine multiple preliminary candidate pose information of the target device according to the coordinate information to be verified;

[0117] Analyze each preliminary candidate pose information, determine the target preliminary candidate pose information to be filtered, and obtain multiple candidate pose information of the target device.

[0118] In some embodiments of the present application, the determining module 302 is specifically further configured to:

[0119] Each spatial coordinate information corresponds to a second plane coordinate information and a first plane coordinate information respectively. According to the second plane coordinate information and the first plane coordinate information corresponding to each spatial coordinate information respectively, determine the first plane coordinate information corresponding to each second plane coordinate information;

[0120] According to the first plane coordinate information corresponding to each second plane coordinate information, determine the first plane coordinate information corresponding to each verification plane coordinate information, and obtain the first mapped plane coordinate information.

[0121] In some embodiments of the present application, the determining module 302 is specifically further configured to:

[0122] Determine the target spatial coordinate corresponding to the verification coordinate;

[0123] According to the target spatial coordinate, rotation information, and position information, determine the coordinate position error between the verification coordinate and the first mapped plane coordinate.

[0124] In some embodiments of the present application, the obtaining module 301 is specifically configured to:

[0125] Control the first image sensor to capture each positioning mark on the glasses, and obtain image data of different angles of each positioning mark;

[0126] According to the image data of different angles of each positioning mark, obtain the spatial coordinates of each positioning mark.

[0127] In some embodiments of the present application, the obtaining module 301 is specifically further configured to:

[0128] Control the second image sensor to capture each positioning mark on the glasses, and obtain the target captured image of each positioning mark;

[0129] Construct the first plane coordinate information according to the target captured image, and obtain the first plane coordinate information of each positioning mark.

[0130] In some embodiments of the present application, the determining module 302 is further specifically configured to:

[0131] Control the target device to perform image display according to the attitude information.

[0132] An embodiment of the present application further provides a pair of glasses, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the steps in the device positioning method according to any one of the embodiments of the present application. Among them, this pair of glasses integrates any one of the device positioning methods provided by the embodiments of the present application. As Figure 4 shown, it shows a schematic structural diagram of the glasses involved in the embodiments of the present application. Specifically:

[0133] The pair of glasses may include a processor 401 with one or more processing cores, a memory 402 of one or more computer-readable storage media, a power supply 403, an input unit 404, and other components. Those skilled in the art can understand that Figure 4 the glasses structure shown in does not constitute a limitation on the glasses, and may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements. Among them:

[0134] The processor 401 is the control center of the pair of glasses, connecting various parts of the entire pair of glasses through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, it executes various functions of the pair of glasses and processes data, thereby monitoring the pair of glasses as a whole. Optionally, the processor 401 may include one or more processing cores; the processor 401 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Preferably, the processor 401 may integrate an application processor and a modulation and demodulation processor. Among them, the application processor mainly processes the operating system, user interface, and application programs, etc., and the modulation and demodulation processor mainly processes wireless communication. It can be understood that the above modulation and demodulation processor may not be integrated into the processor 401.

[0135] The memory 402 can be used to store the control programs and modules of the glasses. The processor 401 executes various functional applications and data processing by running the control programs and modules of the glasses stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area may store data created according to the use of the glasses. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0136] The glasses also include a power supply 403 for powering each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 403 may also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0137] The glasses may also include an input unit 404, which can be used to receive input digital or character information, and generate joystick, optical or trackball signal inputs related to user settings and function control. For example, the user can simulate a person walking through the joystick, thereby changing the picture presented in the glasses.

[0138] Although not shown, the glasses may also include a display unit, which can display pictures, and will not be elaborated here. Specifically, in this embodiment, the processor 401 in the glasses will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402 to implement various functions, such as:

[0139] Obtain the spatial coordinate information and the first plane coordinate information of each positioning mark;

[0140] Determine the attitude information of the target device according to the spatial coordinate information and the first plane coordinate information of each positioning mark.

[0141] Those of ordinary skill in the art can understand that all or part of the steps in the above-mentioned various methods can be completed by instructions, or by controlling related hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by the processor.

[0142] To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), a magnetic disk, an optical disc, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the device positioning methods provided by the embodiments of the present application. For example, when the computer program is loaded by the processor, the following steps may be executed:

[0143] Obtain the spatial coordinate information and the first planar coordinate information of each positioning flag;

[0144] Determine the attitude information of the target device according to the spatial coordinate information and the first planar coordinate information of each positioning flag.

[0145] In the above embodiments, the descriptions of the respective embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference may be made to the detailed descriptions of other embodiments above, and details will not be repeated here.

[0146] In specific implementation, the above-mentioned respective units or structures may be implemented as independent entities, or may be combined arbitrarily to be implemented as the same or several entities. For the specific implementation of the above-mentioned respective units or structures, reference may be made to the method embodiments above, and details will not be repeated here.

[0147] For the specific implementation of the above respective operations, reference may be made to the foregoing embodiments, and details will not be repeated here.

[0148] The above provides a detailed introduction to a device positioning method and device provided by an embodiment of the present application. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A device positioning method, characterized in that, Multiple positioning marks are set on the target device, and the method includes: Obtain the spatial coordinate information and the first plane coordinate information of each of the positioning marks; Determine the attitude information of the target device according to the spatial coordinate information and the first plane coordinate information of each of the positioning marks.

2. The device positioning method according to claim 1, wherein The determining the attitude information of the target device according to the spatial coordinate information and the first plane coordinate information of each of the positioning marks includes: Convert the spatial coordinate information into second plane coordinate information; Determine the attitude information of the target device according to the second plane coordinate information and the first plane coordinate information.

3. The device positioning method according to claim 2, characterized in that, The determining the attitude information of the target device according to the spatial coordinate information and the first plane coordinate information of each of the positioning marks includes: Determine the coordinate information conversion factor between the spatial coordinate information and the second plane coordinate information; Determine the product result of the spatial coordinate information and the coordinate information conversion factor; Determine the converted second plane coordinate information according to the product result.

4. The device positioning method according to claim 2, wherein, The determining the attitude information of the target device according to the second plane coordinate information and the first plane coordinate information includes: Divide the coordinate information in the second plane coordinate information into to-be-verified coordinate information and verification plane coordinate information; Determine multiple candidate attitude information of the target device according to the to-be-verified coordinate information; Determine the first plane coordinate information corresponding to each of the second plane coordinate information; Determine the first mapped plane coordinate information corresponding to the verification plane coordinate information in the first plane coordinate information according to the first plane coordinate information corresponding to each of the second plane coordinate information; Determine the coordinate information position error between the verification plane coordinate information and the first mapped plane coordinate information; Determine one of the candidate attitude information in each of the candidate attitude information as the attitude information of the target device according to the coordinate information position error.

5. The device positioning method according to claim 4, wherein, The determining multiple candidate attitude information of the target device according to the to-be-verified coordinate information includes: Determine multiple preliminary candidate attitude information of the target device according to the to-be-verified coordinate information; Analyze each of the preliminary candidate attitude information, determine the target preliminary candidate attitude information to be filtered, and obtain multiple candidate attitude information of the target device.

6. The device positioning method according to claim 4, characterized in that, The determining the first mapped plane coordinate information corresponding to the verification plane coordinate information in the first plane coordinate information according to the first plane coordinate information corresponding to each of the second plane coordinate information includes: Each of the spatial coordinate information corresponds to a second plane coordinate information and a first plane coordinate information. According to the second plane coordinate information and the first plane coordinate information respectively corresponding to each spatial coordinate information, determine the first plane coordinate information corresponding to each second plane coordinate information; Determine the first plane coordinate information corresponding to each of the verification plane coordinate information according to the first plane coordinate information corresponding to each second plane coordinate information, and obtain the first mapped plane coordinate information.

7. The device positioning method according to claim 4, wherein, The candidate pose information includes the rotation information and position information of the target device. Determining the coordinate position error between the verification plane coordinate information and the first mapped plane coordinate information includes: Determining the target space coordinate information corresponding to the verification plane coordinate information; Determining the coordinate position error between the verification plane coordinate information and the first mapped plane coordinate information according to the target space coordinate information, the rotation information, and the position information.

8. The device positioning method according to claim 1, characterized in that, After determining the pose information of the target device according to the spatial coordinate information and the first plane coordinate information of each positioning mark, the method further includes: Controlling the target device to perform image display according to the pose information.

9. A device positioning device, characterized in that, A plurality of positioning marks are provided on the target device. The device includes: An acquisition module, configured to acquire the spatial coordinate information and the first plane coordinate information of each positioning mark; A determination module, configured to determine the pose information of the target device according to the spatial coordinate information and the first plane coordinate information of each positioning mark; Preferably, the determination module is specifically configured to: Convert the spatial coordinate information into second plane coordinate information; Determine the pose information of the target device according to the second plane coordinate information and the first plane coordinate information; Preferably, the determination module is specifically further configured to: Determine the coordinate conversion factor between the spatial coordinate information and the second plane coordinate information; Determine the product result of the spatial coordinate information and the coordinate conversion factor; Determine the converted second plane coordinate information according to the product result; Preferably, the determination module is specifically further configured to: Divide the coordinate information in the second plane coordinate information into to-be-verified coordinate information and verification plane coordinate information; Determine multiple candidate pose information of the target device according to the to-be-verified coordinate information; Determine the first plane coordinate information corresponding to each second plane coordinate information; Determine the first mapped plane coordinate information corresponding to the verification plane coordinate information in the first plane coordinate information according to the first plane coordinate information corresponding to each second plane coordinate information; Determine the coordinate position error between the verification plane coordinate information and the first mapped plane coordinate information; Determine one candidate pose information among the candidate pose information as the pose information of the target device according to the coordinate position error; Preferably, the determination module is specifically further configured to: Determine multiple preliminary candidate pose information of the target device according to the to-be-verified coordinate information; Analyze each preliminary candidate pose information to determine the target preliminary candidate pose information to be filtered, and obtain multiple candidate pose information of the target device; Preferably, the determination module is specifically further configured to: Each spatial coordinate information corresponds to a second plane coordinate information and a first plane coordinate information. According to the second plane coordinate information and the first plane coordinate information respectively corresponding to each spatial coordinate information, determine the first plane coordinate information corresponding to each second plane coordinate information; Determine the first planar coordinate information corresponding to each of the verification planar coordinate information based on the first planar coordinate information corresponding to each second planar coordinate information, to obtain first mapped planar coordinate information; Preferably, the determining module is further specifically configured to: Determine the target space coordinate information corresponding to the verification planar coordinate information; Determine the coordinate information position error between the verification planar coordinate information and the first mapped planar coordinate information according to the target space coordinate information, the rotation information, and the position information; Preferably, the determining module is further specifically configured to: Control the target device to perform image display according to the attitude information.

10. A device positioning device, characterized in that, The device positioning device includes a plurality of positioning marks, a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps in the device positioning method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and the computer program is executed by a processor to implement the steps in the device positioning method according to any one of claims 1 to 8.

Images