A target positioning method, device and electronic device applied to AR equipment

Abstract

The present application provides a target positioning method, device and electronic device applied to an AR device, which relates to the technical field of virtual reality. The method is applied to a first AR device, and the first AR device is communicatively connected to a second AR device. The method includes: acquiring a first image and a second image; determining a first pixel coordinate of a preset target in the first image, and a second pixel coordinate of the preset target in the second image; determining a first sight direction vector for shooting the preset target based on the first pixel coordinate, and determining a second sight direction vector for shooting the preset target based on the second pixel coordinate; acquiring first position data, and acquiring second position data sent by the second AR device; determining target position data of the preset target based on the first sight direction vector, the second sight direction vector, the first position data and the second position data. The present application can improve the accuracy of the target object distance calculation of the AR device.

Description

Technical Field

[0001] The present application relates to the technical field of virtual reality, and in particular to a target positioning method, device and electronic device applied to an AR device. Background Art

[0002] Augmented reality (AR) devices are an innovative wearable technology that merges virtual scenes with the real world. Typically, an AR device includes a built-in display, camera, and multiple sensors that work together to enable users to see virtual elements in their real field of view. These virtual elements appear to interact with the surrounding environment as if they existed in the real world.

[0003] After wearing these glasses, users can see virtual objects, images, information and interactive elements in their field of vision. These virtual elements move with the user's line of sight, interweaving with the real world to create an engaging augmented reality experience. This technology has a wide range of applications. For example, more and more application functions of AR devices are being developed in the military field, which can improve the combat capability, battlefield perception and decision-making of individual soldiers. Soldiers can also obtain real-time intelligence through AR devices, including the location of the enemy, the movements of friendly forces and other important information, to help them better understand the battlefield situation.

[0004] When soldiers are performing tasks, they may be blocked by hidden targets, so they need to find the target in time and deal with it quickly. But sometimes other friendly forces are needed to assist in solving the target, so it is necessary to provide the friendly forces with information about the target first so that the friendly forces can find the target. There are related technologies that can share targets across devices through AR devices. After different AR devices capture the same target, they mark it on the display screen, which makes it easier for different soldiers to quickly find and determine the target. However, when determining the target, the location information of the target needs to be obtained. At present, there is a method to calculate the target position by comparing the parallax between two images from the AR device camera. Equipped with dual cameras on the AR device, stereo vision is used to calculate the distance between the object and the AR device by detecting the position difference of the same object in the two images. Depth information can be estimated. Finally, the position of the target is calculated based on the position of the AR device and the distance between the object and the AR device.

[0005] However, the distance estimation accuracy of the method of calculating the target position through the above stereo vision method may be uneven at different depths. Generally, the distance estimation of objects closer to the camera is more accurate, while objects far away from the camera may have a larger distance error, resulting in a lower accuracy of the target position calculated by this method. Therefore, a method is needed to improve the accuracy of the target object distance calculation of AR devices. Summary of the invention

[0006] The present application provides a target positioning method, device and electronic device applied to an AR device, which can improve the accuracy of target object distance calculation of the AR device.

[0007] In a first aspect of the present application, a target positioning method applied to an AR device is provided. The method is applied to a first AR device, where the first AR device is communicatively connected to a second AR device. The method includes:

[0008] Acquire a first image and a second image, where the first image is an image taken by a first AR device, and the second image is an image taken by a second AR device;

[0009] Determining first pixel coordinates of a preset target in the first image and second pixel coordinates of the preset target in the second image;

[0010] Determine a first sight direction vector for photographing the preset target based on the first pixel coordinates, and determine a second sight direction vector for photographing the preset target based on the second pixel coordinates;

[0011] Acquire first position data, and acquire second position data sent by the second AR device, wherein the first position data is the position data of the first AR device, and the second position data is the position data of the second AR device;

[0012] The target position data of the preset target is determined according to the first sight direction vector, the second sight direction vector, the first position data, and the second position data.

[0013] By adopting the above technical solution, compared with the stereoscopic vision method, images are collected through two cameras of a single AR device, and then the distance from the target to the AR device is calculated to determine the position of the target. The present application takes into account the position information of the preset target under two different perspectives, by acquiring the first image and the second image, and determining the pixel coordinates of the target in the two images, and then combining the position data of the two AR devices to calculate the target position data of the preset target. The present application utilizes information sharing and collaboration between multiple AR devices to improve calculation accuracy, and combines the sight direction vectors of the first AR device and the second AR device and their respective position data, so as to more accurately determine the position of the target object in space, and there will be no problem of uneven distance estimation accuracy at different depths.

[0014] Optionally, determining the target position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data, and the second position data specifically includes:

[0015] Performing a coordinate system transformation on the first sight direction vector to obtain a first direction vector, and performing a coordinate system transformation on the second sight direction vector to obtain a second direction vector;

[0016] Calculate the coordinates of the preset target, the specific calculation formula is as follows:

[0017]

[0018] Wherein, P is the coordinate of the preset target, d A ' is the first direction vector, d B ' is the second direction vector, the first position data is (x A ,y A , z A ), the second position data is (x B ,y B , z B );

[0019] The coordinates of the preset target are set as the target position data.

[0020] By adopting the above technical solution, the cross product of the first direction vector and the second direction vector is calculated. The cross product is a vector operation, and the result is a new vector that is perpendicular to the plane of the original two vectors. The direction of this cross product vector points to the plane determined by the two original vectors. The direction of the cross product of the first direction vector and the second direction vector will point to the three-dimensional space point where the target is located. The cross product of the result of the vector cross product is calculated with (xA-xB), (yA-yB) and (zA-zB), where (xA-xB), (yA-yB) and (zA-zB) are the differences between the position vectors of the first AR device and the second AR device. This difference vector represents the direction from the first AR device to the second AR device. Finally, the cross product result is divided by the square of the modulus of the cross product vector to obtain the coordinates of the preset target. This is achieved by vector addition and scalar multiplication to ensure that P is located on the plane determined by the two direction vectors. Finally, the coordinates of the preset target are set as the target position data.

[0021] Optionally, the determining, based on the first pixel coordinates, a first sight direction vector for the first AR device to photograph the preset target, and determining, based on the second pixel coordinates, a second sight direction vector for the second AR device to photograph the preset target specifically includes:

[0022] The first sight direction vector is calculated by the following formula:

[0023]

[0024] Among them, d Ais the first sight direction vector, and the first pixel coordinate is (u A , v A );

[0025] The second sight direction vector is calculated by the following formula:

[0026]

[0027] Among them, d B is the second sight direction vector, and the second pixel coordinate is (u B , v B ).

[0028] By adopting the above technical solution, based on the camera imaging principle and camera projection model, the pixel coordinates can be mapped to three-dimensional points in the camera coordinate system, and then the sight direction vector can be determined. The sight direction vector describes the direction from the camera of the AR device to the preset target, and these direction vectors will be used for the subsequent calculation of the target position data.

[0029] Optionally, determining a first pixel coordinate of a preset target in the first image and a second pixel coordinate of the preset target in the second image specifically includes:

[0030] Acquire a first target input by a first user, the first user wears the first AR device, and the first target is an image of the preset target in the first image;

[0031] Determine a first pixel coordinate of the first target in a preset image coordinate system, where the preset image coordinate system is a coordinate system set based on hardware parameters of an AR device when acquiring the first image and the second image;

[0032] Sending the feature of the first target to the second AR device;

[0033] After the second AR device identifies the second target based on the feature of the first target, the second pixel coordinates sent by the second AR device are acquired, where the second target is an image of the preset target in the second image.

[0034] By adopting the above technical solution, the first target input by the first user is obtained, and the first pixel coordinates of the first target in the preset image coordinate system are determined, and then the features of the first target are sent to the second AR device. After the second AR device recognizes the second target based on the features of the first target, the second pixel coordinates of the second target in the preset image coordinate system are obtained. The pixel coordinates of the first target and the second target in the preset image coordinate system can be accurately determined, so that the position of the target object in space can be determined more accurately.

[0035] Optionally, obtaining the first target input by the first user specifically includes:

[0036] Acquire pupil changes of the first user;

[0037] determining the first user's observation sightline according to the pupil change;

[0038] determining an intersection point between the observation line of sight and the first image;

[0039] Performing contour detection on the first image to identify a plurality of first pending targets;

[0040] Determine a second pending target among the plurality of first pending targets, wherein the second pending target includes the intersection point;

[0041] Determine the focusing time length of the intersection point at the second undetermined target;

[0042] It is determined whether the focusing time is greater than or equal to a preset time. If the focusing time is greater than or equal to the preset time, the second target to be determined is determined to be the first target.

[0043] By adopting the above technical solution, the preset target is determined by combining the first user's line of sight and the attention duration, and the preset target that the first user wants to share can be identified more accurately. At the same time, the first user does not need to manually select or mark the preset target, which reduces the operation steps of the first user. By automatically identifying and determining the target, the speed of target sharing can be accelerated, thereby improving the efficiency of target sharing.

[0044] Optionally, performing a coordinate system transformation on the first sight direction vector to obtain a first direction vector, and performing a coordinate system transformation on the second sight direction vector to obtain a second direction vector specifically includes:

[0045] The first sight direction vector in the preset image coordinate system is converted to the first direction vector in the world coordinate system, specifically by the following formula:

[0046] d A =R A ×d A +T A

[0047] Among them, d A ' is the first direction vector, R A is the rotation matrix of the first sight direction vector from the preset image coordinate system to the world coordinate system, T A is the translation matrix of the first sight direction vector for coordinate system transformation, d A is the first sight direction vector;

[0048] The second sight line direction vector in the preset image coordinate system is converted to the second direction vector in the world coordinate system, specifically by the following formula:

[0049] d B =R B ×d B +T B

[0050] Among them, d B ' is the second direction vector, R B is the rotation matrix of the second sight direction vector from the preset image coordinate system to the world coordinate system, T B is the translation matrix of the coordinate system transformation of the second sight direction vector, d B is the second sight direction vector.

[0051] By adopting the above technical solution, the sight direction vector in the preset image coordinate system is converted into the direction vector in the world coordinate system, which can ensure that the coordinate reference systems of different cameras are consistent, so as to perform three-dimensional reconstruction of the target position in a unified coordinate system. This is because the camera usually captures images in its own coordinate system, and the three-dimensional coordinates of the target are usually defined relative to a reference coordinate system.

[0052] Optionally, after determining the target position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data, and the second position data, the method further includes:

[0053] Sending the first image to the second AR device;

[0054] Sending the first position data to the second AR device;

[0055] After the second AR device determines the verification position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data, and the second position data, receiving the verification position data sent by the second AR device;

[0056] The target location data is compared with the verification location data to see whether they are consistent. If the target location data is consistent with the verification location data, the target location data is displayed to a first user and a second user, where the first user wears the first AR device and the second user wears the second AR device.

[0057] By adopting the above technical solution, by sending the first image and the first position data to the second AR device, the second AR device can determine the position of the preset target according to the first sight direction vector, the second sight direction vector, the first position data and the second position data. Then, the second AR device sends the verification position data back to the first AR device for comparison. If the target position data is consistent with the verification position data, the target position data is displayed to the first user and the second user. In this way, possible errors or abnormalities can be detected and corrected or processed in a timely manner.

[0058] In a second aspect of the present application, a target positioning device applied to an AR device is provided. The device is a first AR device, including an acquisition module, a recognition module, and a calculation module, wherein:

[0059] The acquisition module is used to acquire a first image and a second image, where the first image is an image taken by a first AR device and the second image is an image taken by a second AR device;

[0060] The recognition module is used to determine the first pixel coordinates of the preset target in the first image and the second pixel coordinates of the preset target in the second image;

[0061] The calculation module is used to determine a first sight direction vector for photographing the preset target based on the first pixel coordinates, and to determine a second sight direction vector for photographing the preset target based on the second pixel coordinates;

[0062] The acquisition module is configured to acquire first position data and acquire second position data sent by the second AR device, wherein the first position data is the position data of the first AR device and the second position data is the position data of the second AR device;

[0063] The calculation module is used to determine the target position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data and the second position data.

[0064] Optionally, the calculation module is used to perform a coordinate system transformation on the first sight direction vector to obtain a first direction vector, and perform a coordinate system transformation on the second sight direction vector to obtain a second direction vector.

[0065] The calculation module is used to calculate the coordinates of the preset target. The specific calculation formula is as follows:

[0066]

[0067] Wherein, P is the coordinate of the preset target, d A ' is the first direction vector, dB ' is the second direction vector, the first position data is (x A ,y A , z A ), the second position data is (x B ,y B , z B ).

[0068] The calculation module is used to set the coordinates of the preset target as the target position data.

[0069] Optionally, the calculation module is used to calculate the first sight direction vector by the following formula:

[0070]

[0071] Among them, d A is the first sight direction vector, and the first pixel coordinate is (u A , v A ).

[0072] The calculation module is used to calculate the second sight line direction vector by the following formula:

[0073]

[0074] Among them, d B is the second sight direction vector, and the second pixel coordinate is (u B , v B ).

[0075] Optionally, the acquisition module is used to acquire a first target input by a first user, the first user wears the first AR device, and the first target is an image of the preset target in the first image.

[0076] The recognition module is used to determine the first pixel coordinates of the first target in a preset image coordinate system, where the preset image coordinate system is a coordinate system set based on hardware parameters of the AR device when acquiring the first image and the second image.

[0077] The acquisition module is used to send the characteristics of the first target to the second AR device.

[0078] The acquisition module is used to acquire the second pixel coordinates sent by the second AR device after the second AR device recognizes the second target based on the features of the first target, where the second target is the image of the preset target in the second image.

[0079] Optionally, the acquisition module is used to acquire pupil changes of the first user.

[0080] The recognition module is used to determine the observation line of sight of the first user according to the pupil change.

[0081] The recognition module is used to determine the intersection point of the observation line of sight and the first image.

[0082] The recognition module is used to perform contour detection on the first image and identify a plurality of first pending targets.

[0083] The identification module is used to determine a second undetermined target among the plurality of first undetermined targets, wherein the second undetermined target includes the intersection point.

[0084] The calculation module is used to determine the focusing time length when the intersection point is located at the second target to be determined.

[0085] The identification module is used to determine whether the focusing duration is greater than or equal to a preset duration, and if the focusing duration is greater than or equal to the preset duration, determine that the second pending target is the first target.

[0086] Optionally, the calculation module is used to convert the first sight direction vector in a preset image coordinate system to the first direction vector in a world coordinate system, specifically by the following formula:

[0087] d A =R A ×d A +T A

[0088] Among them, d A ' is the first direction vector, R A is the rotation matrix of the first sight direction vector from the preset image coordinates to the world coordinate system, T A is the translation matrix of the first sight direction vector for coordinate system transformation, d A is the first sight direction vector.

[0089] The calculation module is used to convert the second sight line direction vector in the preset image coordinate system to the second direction vector in the world coordinate system, specifically by the following formula:

[0090] d B =R B ×d B +T B

[0091] Among them, d B ' is the second direction vector, R Bis the rotation matrix of the second sight direction vector from the preset image coordinates to the world coordinate system, T B is the translation matrix of the coordinate system transformation of the second sight direction vector, d B is the second sight direction vector.

[0092] Optionally, the acquisition module is used to send the first image to the second AR device.

[0093] The acquisition module is used to send the first position data to the second AR device.

[0094] The calculation module is used to receive the verification position data sent by the second AR device after the second AR device determines the verification position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data and the second position data.

[0095] The identification module is used to compare whether the target position data is consistent with the verification position data. If the target position data is consistent with the verification position data, the target position data is displayed to a first user and a second user. The first user wears the first AR device and the second user wears the second AR device.

[0096] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0097] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0098] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0099] Compared with the stereoscopic vision method, the two cameras of a single AR device are used to collect images, and then the distance from the target to the AR device is calculated to determine the position of the target. This application takes into account the position information of the preset target under two different perspectives, by acquiring the first image and the second image, and determining the pixel coordinates of the target in the two images, and then combining the position data of the two AR devices to calculate the target position data of the preset target. This application utilizes information sharing and collaboration between multiple AR devices to improve calculation accuracy, and combines the line of sight direction vectors of the first AR device and the second AR device and their respective position data, so as to more accurately determine the position of the target object in space, and there will be no problem of uneven distance estimation accuracy at different depths. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Figure 1 It is a flowchart of a target positioning method applied to an AR device disclosed in an embodiment of the present application;

[0101] Figure 2 It is a structural schematic diagram of a target positioning device applied to an AR device disclosed in an embodiment of the present application;

[0102] Figure 3 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0103] Explanation of the reference numerals: 201, acquisition module; 202, identification module; 203, calculation module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0104] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0105] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0106] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0107] Augmented reality (AR) devices are an innovative wearable technology that merges virtual scenes with the real world. Typically, an AR device includes a built-in display, camera, and multiple sensors that work together to enable users to see virtual elements in their real field of view. These virtual elements appear to interact with the surrounding environment as if they existed in the real world.

[0108] After wearing these glasses, users can see virtual objects, images, information and interactive elements in their field of vision. These virtual elements move with the user's line of sight, interweaving with the real world to create an engaging augmented reality experience. This technology has a wide range of applications. For example, more and more application functions of AR devices are being developed in the military field, which can improve the combat capability, battlefield perception and decision-making of individual soldiers. Soldiers can also obtain real-time intelligence through AR devices, including the location of the enemy, the movements of friendly forces and other important information, to help them better understand the battlefield situation.

[0109] When soldiers are performing tasks, they may be blocked by hidden targets, so they need to find the target in time and deal with it quickly. But sometimes other friendly forces are needed to assist in solving the target, so it is necessary to provide the friendly forces with information about the target first so that the friendly forces can find the target. There are related technologies that can share targets across devices through AR devices. After different AR devices capture the same target, they mark it on the display screen, which makes it easier for different soldiers to quickly find and determine the target. However, when determining the target, the location information of the target needs to be obtained. At present, there is a method to calculate the target position by comparing the parallax between two images from the AR device camera. Equipped with dual cameras on the AR device, stereo vision is used to calculate the distance between the object and the AR device by detecting the position difference of the same object in the two images. Depth information can be estimated. Finally, the position of the target is calculated based on the position of the AR device and the distance between the object and the AR device.

[0110] However, the distance estimation accuracy of the method of calculating the target position through the above stereo vision method may be uneven at different depths. Generally, the distance estimation of objects closer to the camera is more accurate, while objects far away from the camera may have a larger distance error, resulting in a lower accuracy of the target position calculated by this method. Therefore, a method is needed to improve the accuracy of the target object distance calculation of AR devices.

[0111] This embodiment discloses a target positioning method applied to an AR device. Figure 1 , including the following steps S110-S160:

[0112] S110, acquiring a first image and a second image.

[0113] A target positioning method applied to an AR device disclosed in an embodiment of the present application is applied to a first AR device, the first AR device is communicatively connected to a second AR device, the first AR device is any one of a group of multiple AR devices communicatively connected, and the second AR device is any one of a group of multiple AR devices communicatively connected.

[0114] The first AR device and the second AR device are both provided with a camera for capturing images directly in front of a user wearing the AR device in real time. The image captured by the camera of the first AR device is the first image, and the image captured by the camera of the second AR device is the second image.

[0115] When the first user who is the wearer of the first AR device needs to locate the preset target, he first needs to input the preset target into the first AR device so that the first AR device can obtain the preset target. The preset target can be any object in reality, such as a stone, a person, an animal, etc. The first AR device monitors the pupil changes of the first user by connecting to the built-in sensor. These changes may be the user's eyeballs constantly moving to adapt to different observation targets, or they may be the contraction or expansion of the pupil. Based on eye tracking technology, the observation line of the first user can be judged by the changes in the pupil of the first user. The above-mentioned determination of the user's observation line of sight by eye tracking technology is only a conventional technical means adopted by technicians in the relevant technical field, and will not be further elaborated here.

[0116] After the observation line of sight is determined, the first AR device needs to calculate the intersection of the observation line of sight and the first image. This is usually achieved by aligning the line of sight direction vector or coordinates with the coordinate system of the first image and then finding the intersection between the two coordinate systems. This step can be achieved using geometric algorithms or computer vision technology. The intersection is usually represented as a coordinate, such as an (x, y) coordinate on a two-dimensional plane.

[0117] The first AR device performs contour detection on the first AR device. First, the first AR device is preprocessed (such as filtering, denoising, etc.), and then the image segmentation or edge detection algorithm is used to find the contours of different pending targets in the first AR device. Through contour detection, the first AR device will identify multiple pending targets and obtain multiple first pending targets. Each pending target can be understood as a shape containing a contour or boundary in the image. Among the multiple first pending targets identified, the first AR device will determine a second pending target. The second pending target usually contains an intersection, that is, this target is watched by the first user. In the previous steps, the first AR device obtains the coordinates of the intersection, and further determines whether the intersection falls within the first pending target based on the coordinate data and shape data of each first pending target. If it is determined according to the coordinates that the intersection falls within a first pending target, the first pending target is determined to be the second pending target.

[0118] Furthermore, the timing starts from when the intersection falls into the second target to be determined, and the focusing time when the intersection is in the second target to be determined is recorded, that is, the time the first user focuses on the second target to be determined. If the first AR device determines that the focusing time of the user focusing on the second target to be determined is greater than or equal to the preset time, the second target to be determined is determined to be the preset target, and the preset target is the target that the first user needs to share. Among them, the specific value of the preset time in different embodiments can be set according to actual conditions, and this embodiment does not make specific limitations.

[0119] By combining the first user's line of sight and attention duration to determine the preset target, the preset target that the first user wants to share can be identified more accurately. At the same time, the first user does not need to manually select or mark the preset target, which reduces the first user's operation steps. By automatically identifying and determining the target, the speed of target sharing can be accelerated, thereby improving the efficiency of target sharing.

[0120] S120, determining first pixel coordinates of a preset target in the first image and second pixel coordinates of the preset target in the second image.

[0121] Pixel coordinates are usually provided directly by the image acquisition device (the camera of the AR device). They are the coordinates of each point in the image, usually expressed as (u, v), where u represents the horizontal coordinate and v represents the vertical coordinate. These coordinates are discrete and usually in pixels.

[0122] In the first image and the second image, the origin of the pixel coordinates is usually located at the upper left corner of the image, the coordinate values ​​of the horizontal direction u increase from left to right, and the coordinate values ​​of the vertical direction v increase from top to bottom. For example, the pixel coordinates of the upper left corner are usually (0, 0), and the pixel coordinates of the lower right corner depend on the resolution of the image.

[0123] When the camera of the first AR device captures the first image, or when the camera of the second AR device captures the second image, each object point will generate a pixel coordinate on the image sensor, which is obtained by the light from the object point to the corresponding position on the image sensor. Usually, the AR device automatically calculates the pixel coordinates for each point, which is completed by the imaging process of the camera, involving the lens, sensor and other camera parameters.

[0124] After the user inputs the information of the first target to the first AR device, the first AR device determines the outline of the first target, and then uses the coordinates of the pixel at the center point of the outline as the coordinates of the first target to obtain the first pixel coordinates (u A , v A ). Then the first AR device sends the contour features of the first target to the second AR device. The second AR device performs contour detection on the second image to obtain the pending contours of multiple targets, and then compares the multiple pending contours with the contour of the first target. The target corresponding to the pending contour closest to the first target among the multiple pending contours is set as the second target, and the second target is the image of the preset target in the second image. Finally, the pixel coordinates of the center point of the contour of the second target are also used as the coordinates of the second target to obtain the second pixel coordinates (u B , v B ).

[0125] The first target input by the first user is obtained, and the first pixel coordinates of the first target in the preset image coordinate system are determined, and then the features of the first target are sent to the second AR device. After the second AR device recognizes the second target based on the features of the first target, the second pixel coordinates of the second target in the preset image coordinate system are obtained. The pixel coordinates of the first target and the second target in the preset image coordinate system can be accurately determined, so that the position of the target object in space can be determined more accurately.

[0126] S130, determining a first sight line direction vector for photographing a preset target based on the first pixel coordinates, and determining a second sight line direction vector for photographing the preset target based on the second pixel coordinates.

[0127] Based on the imaging principle of the camera (the camera of the first AR device and the camera of the second AR device) and the camera projection model, the pixel coordinates can be mapped to the three-dimensional point in the camera coordinate system, and then the line of sight direction vector can be determined. The camera imaging principle is that the image captured by the camera is formed by light passing through the camera lens and focusing on the imaging plane (usually the image sensor). In this process, the light is projected onto the pixels on the imaging plane. This imaging process can be described using the camera projection model. The camera projection model is a mathematical model used to map points in the three-dimensional world to pixel coordinates on a two-dimensional image plane. This model usually includes internal parameters such as focal length, principal point coordinates, distortion correction parameters, and external parameters of the camera's position and orientation (rotation and translation matrix). According to the camera projection model, the three-dimensional point P (x, y, z) can be projected to the point (X, Y, Z) in the camera coordinate system. This projection can be expressed by the following formula:

[0128]

[0129] Among them, (X, Y, Z) is a 3D point in the camera coordinate system, (x, y, z) is a 3D point in the world coordinate system, and (f x , f y ) is the focal length of the camera, (c x , c y ) is the principal point coordinate. The principal point coordinate is a parameter in the camera projection model, which represents the coordinate of the intersection of the optical axis on the imaging plane and the center of the imaging plane.

[0130] Then, by projecting the point (X, Y, Z) in the camera coordinate system onto the imaging plane, we get the pixel coordinates (u, v). This projection can be expressed by the following formula:

[0131]

[0132] Finally, the sight direction vector can be calculated through the pixel coordinates (u, v). The sight direction vector is the direction vector pointing from the camera position to the target point. The sight direction vector describes the direction from the camera of the AR device to the preset target. These direction vectors will be used to calculate the subsequent target position data. The first sight direction vector can be expressed by the following formula:

[0133]

[0134] Among them, d A is the first sight direction vector, and the first pixel coordinate is (u A , v A );

[0135] The second sight direction vector is expressed by the following formula:

[0136]

[0137] Among them, d B is the second sight direction vector, and the second pixel coordinate is (u B , v B ).

[0138] S140: Acquire the first position data, and acquire the second position data sent by the second AR device.

[0139] First, ensure that both the first AR device and the second AR device can obtain their own location data. Usually, AR devices are required to have location tracking functions, such as GPS, inertial measurement unit (IMU), camera visual positioning, etc. AR devices can obtain their own location information through these sensors. The second AR device also needs to obtain its own location data. It can obtain location data in a similar way to the first AR device and transmit it to the first AR device.

[0140] S150, determining target position data of a preset target according to the first sight direction vector, the second sight direction vector, the first position data, and the second position data.

[0141] Since the first sight direction vector and the second sight direction vector are both data in the preset image coordinate system, and the first position data and the second position data are both data in the world coordinate system, it is necessary to perform coordinate system conversion to unify the coordinate system. The first sight direction vector in the preset image coordinate system is converted to the first direction vector in the world coordinate system, specifically through the following conversion formula:

[0142] d A =R A ×d A +T A

[0143] Among them, d A ' is the first direction vector, R A is the rotation matrix of the first sight direction vector from the preset image coordinate system to the world coordinate system, T A is the translation matrix of the first sight direction vector for coordinate system transformation, d A is the first sight direction vector.

[0144] The second sight direction vector in the preset image coordinate system is converted to the second direction vector in the world coordinate system, specifically through the following formula:

[0145] d B =R B ×d B +T B

[0146] Among them, d B ' is the second direction vector, R B is the rotation matrix of the second sight direction vector from the preset image coordinate system to the world coordinate system, T B is the translation matrix of the coordinate system transformation of the second sight direction vector, d B is the second sight direction vector.

[0147] The rotation matrix and translation matrix are used to describe the external parameters of the camera. They determine the position and orientation of the camera so as to transform the sight direction vector from the camera coordinate system to the world coordinate system. These parameters are usually determined by camera calibration and camera motion estimation.

[0148] The rotation matrix is ​​used to represent the orientation of the camera, that is, how the camera coordinate system rotates relative to the world coordinate system. Usually, this is a 3x3 matrix, and common representations include Euler angles, rotation vectors, quaternions, etc. The rotation matrix can be estimated by calibrating the camera or using the camera's attitude sensor (such as a gyroscope). The translation matrix is ​​used to represent the translation of the camera relative to the world coordinate system. It is usually a 3x1 vector that describes the offset of the camera position relative to the origin of the world coordinate system. The translation matrix can be estimated by calibrating the camera or using other sensors (such as GPS).

[0149] When calibrating a camera, it is usually necessary to capture images at different poses (rotation and translation) using a known calibration target or calibration plate. By analyzing these images and the position of the known calibration target, the rotation matrix and translation matrix of the camera can be estimated. This process is usually called camera calibration, and it can be done using various calibration methods and tools, such as the Zhang Zhengyou calibration method.

[0150] Converting the sight direction vector in the preset image coordinate system to the direction vector in the world coordinate system can ensure the consistency of the coordinate reference systems between different cameras so that the 3D reconstruction of the target position can be performed in a unified coordinate system. This is because the camera usually captures images in its own coordinate system, and the 3D coordinates of the target are usually defined relative to a reference coordinate system.

[0151] Then calculate the coordinates of the preset target using the following calculation formula:

[0152]

[0153] Where P is the coordinate of the preset target, d A ' is the first direction vector, d B ' is the second direction vector, the first position data is (x A ,y A , z A), the second position data is (x B ,y B , z B ).

[0154] Calculates the cross product of the first direction vector and the second direction vector. The cross product is a vector operation that results in a new vector that is perpendicular to the plane of the two original vectors. The direction of this cross product vector points into the plane determined by the two original vectors. In this context, the direction of the cross product of the first direction vector and the second direction vector will point to the three-dimensional space point where the target is located. The result of calculating the vector cross product is the same as (x A -x B )、(y A -y B ) and (z A -z B ), where (x A -x B )、(y A -y B ) and (z A -z B ) is the difference between the position vectors of the first AR device and the second AR device. This difference vector represents the direction from the first AR device to the second AR device. Finally, the cross product result is divided by the square of the modulus of the cross product vector to obtain the coordinates of the preset target. This is achieved through vector addition and scalar multiplication, ensuring that P is located in the plane determined by the two direction vectors. Finally, the coordinates of the preset target are set as the target position data.

[0155] By adopting the above technical solution, compared with the stereoscopic vision method, images are collected through two cameras of a single AR device, and then the distance from the target to the AR device is calculated to determine the position of the target. The present application takes into account the position information of the preset target under two different perspectives, by acquiring the first image and the second image, and determining the pixel coordinates of the target in the two images, and then combining the position data of the two AR devices to calculate the target position data of the preset target. The present application utilizes information sharing and collaboration between multiple AR devices to improve calculation accuracy, and combines the sight direction vectors of the first AR device and the second AR device and their respective position data, so as to more accurately determine the position of the target object in space, and there will be no problem of uneven distance estimation accuracy at different depths.

[0156] Furthermore, in order to verify whether the calculation result of the target position data is accurate, the first AR device needs to obtain the calculation result of the position data of the preset target by the second AR device, so as to compare it with the target position data to determine whether the calculation result is accurate. The first AR device only needs to send the first image and the first position data to the second AR device in sequence. Then the second AR device refers to the step of calculating the target position data by the first AR device, and calculates the first pixel coordinates of the preset target in the first image, and the second pixel coordinates of the preset target in the second image. The second image here is preferably an image other than the image sent to the first AR device. Then, based on the first pixel coordinates, the first sight direction vector for shooting the preset target is calculated, and based on the second pixel coordinates, the second sight direction vector for shooting the preset target is calculated. Finally, according to the first sight direction vector, the second sight direction vector, the first position data and the second position data, the verification position data of the preset target is calculated, and the verification position data is sent to the first AR device.

[0157] After receiving the verification position data, the first AR device compares the verification position data with the target position data to determine whether the two are consistent. If the two are inconsistent, it indicates that an error occurred in the process of determining the position of the preset target. This may be caused by the inconsistency of the targets observed by the two AR devices, or it may be caused by other reasons. In this case, it is necessary to re-determine the target and calculate the position. If the target position data is consistent with the verification position data, it indicates that the first AR device and the second AR device observe the same target. The first AR device displays the target position data to the first user, and the second AR device displays the target position data to the second user.

[0158] By sending the first image and the first position data to the second AR device, the second AR device can determine the position of the preset target according to the first sight direction vector, the second sight direction vector, the first position data and the second position data. Then, the second AR device sends the verification position data back to the first AR device for comparison. If the target position data is consistent with the verification position data, the target position data is displayed to the first user and the second user. In this way, possible errors or abnormalities can be detected and corrected or processed in a timely manner.

[0159] This embodiment also discloses a target positioning device applied to an AR device, the device is a first AR device, referring to Figure 2 , including an acquisition module 201, an identification module 202 and a calculation module 203, wherein:

[0160] An acquisition module 201 is used to acquire a first image and a second image, where the first image is an image taken by a first AR device and the second image is an image taken by a second AR device;

[0161] The recognition module 202 is used to determine the first pixel coordinates of the preset target in the first image and the second pixel coordinates of the preset target in the second image;

[0162] The calculation module 203 is used to determine a first sight direction vector for photographing a preset target based on the first pixel coordinates, and to determine a second sight direction vector for photographing the preset target based on the second pixel coordinates;

[0163] An acquisition module 201 is used to acquire first position data and acquire second position data sent by a second AR device, wherein the first position data is the position data of the first AR device and the second position data is the position data of the second AR device;

[0164] The calculation module 203 is used to determine the target position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data and the second position data.

[0165] In a possible implementation, the calculation module 203 is configured to perform a coordinate system transformation on the first sight line direction vector to obtain a first direction vector, and perform a coordinate system transformation on the second sight line direction vector to obtain a second direction vector.

[0166] The calculation module 203 is used to calculate the coordinates of the preset target. The specific calculation formula is as follows:

[0167]

[0168] Where P is the coordinate of the preset target, d A ' is the first direction vector, d B ' is the second direction vector, the first position data is (x A ,y A , z A ), the second position data is (x B ,y B , z B ).

[0169] The calculation module 203 is used to set the coordinates of the preset target as the target position data.

[0170] In a possible implementation, the calculation module 203 is configured to calculate the first sight direction vector using the following formula:

[0171]

[0172] Among them, d A is the first sight direction vector, and the first pixel coordinate is (u A , v A ).

[0173] The calculation module 203 is used to calculate the second sight line direction vector using the following formula:

[0174]

[0175] Among them, d B is the second sight direction vector, and the second pixel coordinate is (u B , v B ).

[0176] In a possible implementation, the acquisition module 201 is used to acquire a first target input by a first user, the first user wears a first AR device, and the first target is an image of a preset target in the first image.

[0177] The recognition module 202 is used to determine the first pixel coordinates of the first target in a preset image coordinate system, where the preset image coordinate system is a coordinate system set based on hardware parameters of the AR device when the first image and the second image are captured.

[0178] The acquisition module 201 is used to send the feature of the first target to the second AR device.

[0179] The acquisition module 201 is used to acquire the second pixel coordinates sent by the second AR device after the second AR device recognizes the second target based on the features of the first target, where the second target is the image of the preset target in the second image.

[0180] In a possible implementation, the acquisition module 201 is configured to acquire pupil changes of the first user.

[0181] The identification module 202 is used to determine the observation line of the first user according to the pupil change.

[0182] The identification module 202 is used to determine the intersection point between the observation line of sight and the first image.

[0183] The recognition module 202 is used to perform contour detection on the first image and recognize a plurality of first pending targets.

[0184] The identification module 202 is used to determine a second target to be determined among a plurality of first targets to be determined, where the second target to be determined includes an intersection point.

[0185] The calculation module 203 is used to determine the focusing time length when the intersection point is located at the second target to be determined.

[0186] The identification module 202 is used to determine whether the focusing time is greater than or equal to a preset time. If the focusing time is greater than or equal to the preset time, the second target to be determined is determined to be the first target.

[0187] In a possible implementation, the calculation module 203 is used to convert the first sight direction vector in the preset image coordinate system to the first direction vector in the world coordinate system, specifically by the following formula:

[0188] d A =R A ×d A +T A

[0189] Among them, d A ' is the first direction vector, R A is the rotation matrix of the first sight direction vector from the preset image coordinate system to the world coordinate system, T A is the translation matrix of the first sight direction vector for coordinate system transformation, d A is the first sight direction vector.

[0190] The calculation module 203 is used to convert the second sight line direction vector in the preset image coordinate system to the second direction vector in the world coordinate system, specifically through the following formula:

[0191] d B =R B ×d B +T B

[0192] Among them, d B ' is the second direction vector, R B is the rotation matrix of the second sight direction vector from the preset image coordinate system to the world coordinate system, T B is the translation matrix of the coordinate system transformation of the second sight direction vector, d B is the second sight direction vector.

[0193] In a possible implementation, the acquisition module 201 is configured to send the first image to a second AR device.

[0194] The acquisition module 201 is used to send the first position data to the second AR device.

[0195] The calculation module 203 is used to receive the verification position data sent by the second AR device after the second AR device determines the verification position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data and the second position data.

[0196] The identification module 202 is used to compare whether the target location data is consistent with the verification location data. If the target location data is consistent with the verification location data, the target location data is displayed to the first user and the second user. The first user wears the first AR device and the second user wears the second AR device.

[0197] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0198] This embodiment also discloses an electronic device, referring to Figure 3 The electronic device may include: at least one processor 301 , at least one communication bus 302 , a user interface 303 , a network interface 304 , and at least one memory 305 .

[0199] The communication bus 302 is used to realize the connection and communication between these components.

[0200] The user interface 303 may include a display screen (Display) and a camera (Camera). The optional user interface 303 may also include a standard wired interface and a wireless interface.

[0201] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0202] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or more combinations of a central processing unit 301 (Central Processing Unit, CPU), an image processor 301 (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0203] Among them, the memory 305 may include a random access memory 305 (Random Access Memory, RAM), and may also include a read-only memory 305 (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may also be at least one storage device located away from the aforementioned processor 301. As shown in the figure, the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface 303 module, and an application for a target positioning method applied to an AR device.

[0204] exist Figure 3 In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 301 can be used to call an application program stored in the memory 305 for a target positioning method applied to an AR device. When executed by one or more processors 301, the electronic device executes one or more methods in the above-mentioned embodiments.

[0205] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0206] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0207] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0208] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0209] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0210] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory 305. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory 305 and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory 305 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0211] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, it will be easy for those skilled in the art to think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A target positioning method applied to an AR device, characterized in that: The method is applied to a first AR device, where the first AR device is communicatively connected to a second AR device, and the method includes: Acquire a first image and a second image, where the first image is an image taken by a first AR device, and the second image is an image taken by a second AR device; Determining first pixel coordinates of a preset target in the first image, and second pixel coordinates of the preset target in the second image; Determine a first sight direction vector for photographing the preset target based on the first pixel coordinates, and determine a second sight direction vector for photographing the preset target based on the second pixel coordinates; Acquire first position data, and acquire second position data sent by the second AR device, wherein the first position data is the position data of the first AR device, and the second position data is the position data of the second AR device; Determining target position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data, and the second position data; The step of determining the target position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data, and the second position data specifically includes: Performing a coordinate system transformation on the first sight direction vector to obtain a first direction vector, and performing a coordinate system transformation on the second sight direction vector to obtain a second direction vector; Calculate the coordinates of the preset target, the specific calculation formula is as follows: Wherein, P is the coordinate of the preset target, d A ' is the first direction vector, d B ' is the second direction vector, the first position data is (x A ,y A , z A ), the second position data is (x B ,y B , z B ); The coordinates of the preset target are set as the target position data.

2. The target positioning method applied to an AR device according to claim 1, characterized in that: The determining, based on the first pixel coordinates, a first sight line direction vector for the first AR device to photograph the preset target, and determining, based on the second pixel coordinates, a second sight line direction vector for the second AR device to photograph the preset target specifically includes: The first sight direction vector is calculated by the following formula: Among them, d A is the first sight direction vector, and the first pixel coordinate is (u A , v A ); The second sight direction vector is calculated by the following formula: Among them, d B is the second sight direction vector, and the second pixel coordinate is (u B , v B ).

3. The target positioning method applied to an AR device according to claim 1, characterized in that: The determining of the first pixel coordinates of the preset target in the first image and the second pixel coordinates of the preset target in the second image specifically includes: Acquire a first target input by a first user, the first user wears the first AR device, and the first target is an image of the preset target in the first image; Determine a first pixel coordinate of the first target in a preset image coordinate system, where the preset image coordinate system is a coordinate system set based on hardware parameters of an AR device when acquiring the first image and the second image; Sending the feature of the first target to the second AR device; After the second AR device identifies the second target based on the feature of the first target, the second pixel coordinates sent by the second AR device are acquired, where the second target is an image of the preset target in the second image.

4. The target positioning method applied to an AR device according to claim 3, characterized in that: The obtaining of the first target input by the first user specifically includes: Acquire pupil changes of the first user; determining the first user's observation sightline according to the pupil change; determining an intersection point between the observation line of sight and the first image; Performing contour detection on the first image to identify a plurality of first pending targets; Determine a second pending target among the plurality of first pending targets, wherein the second pending target includes the intersection point; Determine the focusing time length of the intersection point at the second target to be determined; It is determined whether the focusing time is greater than or equal to a preset time. If the focusing time is greater than or equal to the preset time, the second target to be determined is determined to be the first target.

5. The target positioning method applied to an AR device according to claim 2, characterized in that: Performing a coordinate system transformation on the first sight direction vector to obtain a first direction vector, and performing a coordinate system transformation on the second sight direction vector to obtain a second direction vector, specifically includes: The first sight direction vector in the preset image coordinate system is converted to the first direction vector in the world coordinate system, specifically by the following formula: d’ A =R A ×d A +T A Among them, d A ' is the first direction vector, R A is the rotation matrix of the first sight direction vector from the preset image coordinate system to the world coordinate system, T A is the translation matrix of the first sight direction vector for coordinate system transformation, d A is the first sight direction vector; The second sight line direction vector in the preset image coordinate system is converted to the second direction vector in the world coordinate system, specifically by the following formula: d’ B =R B ×d B +T B Among them, d B ' is the second direction vector, R B is the rotation matrix of the second sight direction vector from the preset image coordinate system to the world coordinate system, T B is the translation matrix of the coordinate system transformation of the second sight direction vector, d B is the second sight direction vector.

6. The target positioning method applied to an AR device according to claim 1, characterized in that: After determining the target position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data, and the second position data, the method further includes: Sending the first image to the second AR device; Sending the first position data to the second AR device; After the second AR device determines the verification position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data, and the second position data, receiving the verification position data sent by the second AR device; The target location data is compared with the verification location data to see whether they are consistent. If the target location data is consistent with the verification location data, the target location data is displayed to a first user and a second user, where the first user wears the first AR device and the second user wears the second AR device.

7. A target positioning device applied to an AR device, characterized in that: The apparatus is a first AR device, comprising an acquisition module (201), an identification module (202) and a calculation module (203), wherein: The acquisition module (201) is used to acquire a first image and a second image, wherein the first image is an image taken by a first AR device and the second image is an image taken by a second AR device; The recognition module (202) is used to determine a first pixel coordinate of a preset target in the first image, and a second pixel coordinate of the preset target in the second image; The calculation module (203) is used to determine a first sight line direction vector for photographing the preset target based on the first pixel coordinates, and to determine a second sight line direction vector for photographing the preset target based on the second pixel coordinates; The acquisition module (201) is used to acquire first position data and acquire second position data sent by the second AR device, wherein the first position data is the position data of the first AR device and the second position data is the position data of the second AR device; The calculation module (203) is used to determine the target position data of the preset target according to the first sight direction vector, the second sight direction vector, the first position data and the second position data; The calculation module (203) is used to determine the target position data of the preset target in the following manner: Performing a coordinate system transformation on the first sight direction vector to obtain a first direction vector, and performing a coordinate system transformation on the second sight direction vector to obtain a second direction vector; Calculate the coordinates of the preset target, the specific calculation formula is as follows: Wherein, P is the coordinate of the preset target, d A ' is the first direction vector, d B ' is the second direction vector, the first position data is (x A ,y A , z A ), the second position data is (x B ,y B , z B ); The coordinates of the preset target are set as the target position data.

8. An electronic device, characterized in that: The electronic device comprises a processor (301), a memory (305), a user interface (303) and a network interface (304), wherein the memory (305) is used to store instructions, the user interface (303) and the network interface (304) are both used to communicate with other devices, and the processor (301) is used to execute the instructions stored in the memory (305) so that the electronic device executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 6 is performed.

Images