Positioning method and apparatus, terminal device, and storage medium

Abstract

The application is suitable for the technical field of data processing, and provides a positioning method and device, terminal equipment and storage medium. First, the image of a target area and the world coordinate system coordinates corresponding to each grid point in a priori area are acquired, then a monocular camera is calibrated to generate a calibration matrix, then the world coordinate system coordinates corresponding to each grid point are converted to the pixel coordinate system corresponding to the image to generate a reflection mapping matrix, finally the world coordinate system coordinates corresponding to each pixel point in the image are determined, and the world coordinate system coordinates corresponding to the target to be positioned are determined. Thus, by converting the world coordinate system coordinates of the priori area to the pixel coordinate system, the mapping relationship between the coordinates of each pixel point in the image and the world coordinate system coordinates is established, so that the world coordinate system coordinates of the target to be positioned in the monocular camera sensing area can be obtained only through the image collected by the monocular camera, thereby realizing accurate positioning and improving the accuracy and reliability of monocular visual positioning.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a positioning method, device, terminal equipment and storage medium. Background Technology

[0002] In related technologies, visual positioning methods can be divided into monocular visual positioning, binocular visual (stereo vision) positioning, and multi-view visual (omnidirectional vision) positioning, depending on the number of visual sensors used. However, as the number of visual sensors increases, the complexity and cost of positioning calculations also increase accordingly. Monocular visual positioning only uses a monocular camera for positioning. Due to the lack of depth information during the imaging process, it is impossible to obtain accurate position information of the target object, resulting in poor accuracy and reliability. Summary of the Invention

[0003] This application provides a positioning method, apparatus, terminal device, and storage medium, which can solve the problem that monocular vision positioning, which only uses a monocular camera for positioning, cannot obtain accurate position information of the target object due to the lack of depth information during the imaging process, resulting in poor accuracy and reliability.

[0004] The first aspect of this application provides a positioning method, including:

[0005] The image of the target area is obtained, as well as the world coordinate system coordinates of each grid point in the prior area. The image is captured by a monocular camera on the target area. The image includes multiple pixels and contains the target to be located. The target area includes the prior area, which includes multiple grid points.

[0006] Based on the image and the world coordinate system coordinates corresponding to each grid point, a calibration matrix is ​​generated for the monocular camera.

[0007] Using a calibration matrix, the world coordinates of each grid point are transformed to the pixel coordinates of the image to generate an inverse mapping matrix;

[0008] Based on the inverse mapping matrix, determine the world coordinate system coordinates corresponding to each pixel in the image, and based on the pixel where the target to be located is located in the image, determine the world coordinate system coordinates corresponding to the target to be located.

[0009] Optionally, in one possible implementation of the first aspect, the calibration matrix includes an intrinsic parameter matrix and an extrinsic parameter matrix. The world coordinate system coordinates corresponding to each grid point in the prior region are obtained through a depth sensing device. The calibration matrix is ​​generated by calibrating the monocular camera based on the image and the world coordinate system coordinates corresponding to each grid point, including:

[0010] Obtain the intrinsic parameter matrix of the monocular camera;

[0011] Perform object detection on the image to identify at least 5 calibrated targets within the prior region;

[0012] Determine the world coordinate system coordinates and pixel coordinates corresponding to at least 5 calibration targets;

[0013] Based on the intrinsic parameter matrix, the world coordinate system coordinates and pixel coordinates corresponding to at least 5 calibration targets, the extrinsic parameter matrix between the monocular camera and the depth sensing device is determined.

[0014] Optionally, in one possible implementation of the first aspect, obtaining the intrinsic parameter matrix of the monocular camera includes:

[0015] The monocular camera was calibrated using Zhang Zhengyou's calibration method to generate its intrinsic parameter matrix.

[0016] Optionally, in one possible implementation of the first aspect, the above-described object detection of the image to determine at least five labeled targets within a priori region includes:

[0017] The YOLOv5 algorithm is used to detect targets in the image to identify at least 5 calibrated targets in the prior region.

[0018] Optionally, in one possible implementation of the first aspect, before determining the world coordinate system coordinates and pixel coordinates corresponding to at least five calibration targets, the method further includes:

[0019] Time synchronization is performed between the monocular camera and the depth sensing device.

[0020] Optionally, in one possible implementation of the first aspect, the above-mentioned use of a calibration matrix to transform the world coordinates corresponding to each grid point to the pixel coordinates corresponding to the image to generate an inverse mapping matrix includes:

[0021] Using a calibration matrix, the world coordinates of each grid point are transformed to the pixel coordinates of the image to generate a sparse pixel matrix.

[0022] Upsample the sparse pixel matrix to generate an inverse mapping matrix.

[0023] Optionally, in one possible implementation of the first aspect, the above-mentioned use of a calibration matrix to transform the world coordinates corresponding to each grid point to the pixel coordinates corresponding to the image to generate a sparse pixel matrix includes:

[0024] Transform the world coordinates of each grid point to the pixel coordinates of the image, and determine the mapping relationship between each pixel and each grid point.

[0025] A sparse pixel matrix is ​​generated based on the mapping relationship between each pixel and each grid point.

[0026] Optionally, in one possible implementation of the first aspect, the transformation of the world coordinate system coordinates corresponding to each grid point to the pixel coordinate system corresponding to the image, and the determination of the mapping relationship between each pixel point and each grid point, includes:

[0027] Transform the world coordinate system of each grid point to the preset camera coordinate system using a rigid body transformation;

[0028] Transform the preset camera coordinate system to the preset image coordinate system using perspective transformation;

[0029] Based on the camera intrinsic parameters, the preset image coordinate system is transformed to the pixel coordinate system through an affine transformation;

[0030] Based on the world coordinate system coordinates and pixel coordinate system corresponding to each grid point, determine the mapping relationship between each pixel point and each grid point.

[0031] Optionally, in one possible implementation of the first aspect, the above-mentioned upsampling of the sparse pixel matrix to generate an inverse mapping matrix includes:

[0032] Among all pixels, those with corresponding world coordinate system coordinates are defined as non-nullable points, and those without corresponding world coordinate system coordinates are defined as nullable points.

[0033] Each pixel is traversed, and it is determined whether there is at least one non-nullable pixel among the surrounding pixels of each pixel. If there is at least one non-nullable pixel among the surrounding pixels of any pixel, the mean of the world coordinates of the surrounding pixels of any pixel is assigned to the pixel. If all the surrounding pixels of any pixel are nullable, the next pixel is traversed.

[0034] Repeat the above traversal process until all pixels are non-nullable.

[0035] Based on the world coordinates of each pixel after traversal processing, an inverse mapping matrix is ​​generated.

[0036] Optionally, in one possible implementation of the first aspect, the surrounding pixels of any pixel are the pixels in the eight-neighborhood of any pixel.

[0037] A second aspect of this application provides a positioning device, including:

[0038] The acquisition module is used to acquire images of the target area and acquire world coordinate system coordinates corresponding to each grid point in the prior area. The image is captured by a monocular camera on the target area. The image includes multiple pixels and contains the target to be located. The target area includes the prior area, which includes multiple grid points.

[0039] The calibration module is used to calibrate the monocular camera and generate a calibration matrix based on the image and the world coordinate system coordinates corresponding to each grid point.

[0040] The transformation module is used to transform the world coordinate system coordinates corresponding to each grid point to the pixel coordinate system corresponding to the image using the calibration matrix, so as to generate the inverse mapping matrix;

[0041] The determination module is used to determine the world coordinate system coordinates corresponding to each pixel in the image based on the inverse mapping matrix, and to determine the world coordinate system coordinates corresponding to the target to be located based on the pixel where the target to be located is located in the image.

[0042] A third aspect of this application provides a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the positioning method of the first aspect described above.

[0043] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the positioning method of the first aspect described above.

[0044] The fifth aspect of this application provides a computer program product that, when run on a terminal device, causes the terminal device to execute the positioning method described in the first aspect.

[0045] The beneficial effects of this application embodiment compared with the prior art are as follows: This application discloses a positioning method, apparatus, terminal device, and storage medium. The method first acquires an image of the target area and the world coordinate system coordinates corresponding to each grid point in the prior area of ​​the target area. Then, it calibrates the monocular camera to generate a calibration matrix. Next, it transforms the world coordinate system coordinates corresponding to each grid point to the pixel coordinate system corresponding to the image to generate an inverse mapping matrix. Finally, it determines the world coordinate system coordinates corresponding to each pixel point in the image and the world coordinate system coordinates corresponding to the target to be located. Thus, by transforming the world coordinate system coordinates of the prior area to the pixel coordinate system, a mapping relationship between the coordinates of each pixel point in the image and the world coordinate system coordinates is established. Therefore, the world coordinate system coordinates of the target to be located within the perception area of ​​the monocular camera can be obtained solely from the image acquired by the monocular camera, thereby achieving precise positioning and improving the accuracy and reliability of monocular visual positioning. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a positioning method provided in Embodiment 1 of this application;

[0048] Figure 2 This is a schematic diagram of the a priori region in the world coordinate system provided in Embodiment 1 of this application;

[0049] Figure 3 This is a schematic diagram of a local mesh point set in the mesh vertex set provided in Embodiment 1 of this application;

[0050] Figure 4 This is a flowchart illustrating a positioning method provided in Embodiment 2 of this application;

[0051] Figure 5 This is a schematic diagram of the structure of a positioning device provided in Embodiment 3 of this application;

[0052] Figure 6 This is a schematic diagram of the structure of a terminal device provided in Embodiment 4 of this application. Detailed Implementation

[0053] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0054] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0055] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0056] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0057] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0058] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0059] It should be understood that the sequence number of each step in this embodiment does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0060] In related technologies, visual positioning methods can be divided into monocular visual positioning, binocular visual (stereo vision) positioning, and multi-view visual (omnidirectional vision) positioning, depending on the number of visual sensors used. However, as the number of visual sensors increases, the complexity and cost of positioning calculations also increase accordingly. Monocular visual positioning only uses a monocular camera for positioning. Due to the lack of depth information during the imaging process, it is impossible to obtain accurate position information of the target object, resulting in poor accuracy and reliability.

[0061] In view of this, embodiments of this application provide a positioning method, apparatus, terminal device, and storage medium. By transforming the world coordinate system coordinates of the prior area to the pixel coordinate system, a mapping relationship between the coordinates of each pixel in the image and the world coordinate system coordinates is established. Thus, the world coordinate system coordinates of the target to be located within the perception area of ​​the monocular camera can be obtained using only the image acquired by the monocular camera, thereby achieving precise positioning and improving the accuracy and reliability of monocular vision positioning.

[0062] The following examples illustrate the application scenarios of the positioning method provided in this application. This application can be applied to a roadside millimeter-wave radar and monocular camera fusion system. Millimeter-wave radar, as a depth sensing device, outputs target coordinates in a world coordinate system. Monocular cameras, however, output target coordinates via a visual detection model, but only pixel coordinates, not world coordinates. Therefore, depth information from the monocular camera is required for depth fusion of the two sensors. Thus, the positioning method provided in this application can obtain the depth information of each pixel in the image acquired by the monocular camera, providing a new approach for subsequent fusion of millimeter-wave radar and monocular cameras.

[0063] To illustrate the technical solution of this application, specific embodiments are described below.

[0064] Reference Figure 1 The diagram illustrates a flowchart of a positioning method provided in Embodiment 1 of this application. Figure 1 As shown, the positioning method may include the following steps:

[0065] Step 101: Obtain the image of the target area and the world coordinate system coordinates corresponding to each grid point in the prior area.

[0066] In this embodiment of the application, the image is captured by a monocular camera on the target area. The image includes multiple pixels and contains the target to be located. The target area includes a prior region, which includes multiple grid points.

[0067] In one possible implementation, the prior region can be uniformly divided into small grids to obtain a uniformly distributed set of grid vertices, where the set of grid vertices includes multiple grid points, and the grid size and the spacing between adjacent grid points can be determined according to the actual application scenario.

[0068] For example, see Figure 2 This is a schematic diagram of the prior region in the world coordinate system, where the prior region is x:[-10,10], y:[-10,300], and the unit is meters (m). See also... Figure 3 This is a schematic diagram of a local grid point set in the grid vertex set, where the grid size is 0.1m*0.1m and the spacing between adjacent grid points is 0.1m.

[0069] The world coordinates of each grid point in the prior area can be obtained using a depth sensing device, such as a millimeter-wave radar. The world coordinate system can be established according to the actual application scenario, and this application does not impose any restrictions on this.

[0070] It should be noted that, since it is necessary to establish the mapping relationship between the coordinates of each pixel in the image and the coordinates in the world coordinate system, but using the entire target area as the prior region and then performing the mapping calculation would be too computationally intensive and difficult to implement, a portion of the target area can be selected as the prior region. The location and size of the prior region can be selected based on the actual application scenario, and this application does not impose any restrictions on this.

[0071] Step 102: Based on the image and the world coordinate system coordinates corresponding to each grid point, calibrate the monocular camera to generate a calibration matrix.

[0072] It should be noted that depth sensing devices can output the position information of the target to be located in the world coordinate system, while monocular cameras can output the position information of the target to be located in the pixel coordinate system. If it is necessary to obtain the position information of the target to be located in the world coordinate system using only a monocular camera, it is necessary to obtain the spatial mapping relationship between the world coordinate system and the pixel coordinate system. Therefore, in order to obtain the above spatial mapping relationship, the monocular camera needs to be calibrated to obtain the calibration matrix.

[0073] Furthermore, the calibration matrix includes the intrinsic parameter matrix K1 of the monocular camera and the extrinsic parameter matrix K2 between the depth sensing device and the monocular camera. The extrinsic parameter matrix K2 can be determined by selecting a calibration target within a priori region and then using the PnP (Perspective-n-Point) algorithm based on the calibration target. That is, in one possible implementation of this application embodiment, step 102 may include:

[0074] Obtain the intrinsic parameter matrix of the monocular camera;

[0075] Perform object detection on the image to identify at least 5 calibrated targets within the prior region;

[0076] Determine the world coordinate system coordinates and pixel coordinates corresponding to at least 5 calibration targets;

[0077] Based on the intrinsic parameter matrix, the world coordinate system coordinates and pixel coordinates corresponding to at least 5 calibration targets, the extrinsic parameter matrix between the monocular camera and the depth sensing device is determined.

[0078] Specifically, the world coordinates [x] of a certain calibrated target within the prior region can be obtained at a certain moment. r ,y r ,z rThen, select the image target detection box [u1, v1, u2, v2] corresponding to the same calibration target on the image, where [u1, v1] is the coordinate of the upper left corner of the image target detection box, and [u2, v2] is the coordinate of the lower right corner of the image target detection box. From this, the center point coordinates of the image target detection box [u, v] = [(u1+u2) / 2, (v1+v2) / 2] can be obtained, and the center point coordinates are used as the pixel coordinates corresponding to the calibration target. The coordinate pair of a calibration target in the world coordinate system and the pixel coordinate system can be represented as [x...]. r ,y r ,z r Repeat the above at least 5 times to obtain N (N>4) sets of coordinate pairs. Then, input the N sets of coordinate pairs and the camera intrinsic parameter K1 into the PnP algorithm to output the extrinsic parameter matrix K2 between the depth sensing device and the monocular camera.

[0079] In the embodiments of this application, any object within the prior area can be used as the calibration target, eliminating the need to manually set up special calibration objects (such as corner reflectors), thus reducing labor costs.

[0080] In one possible implementation, the intrinsic parameter matrix K1 can be obtained through Zhang Zhengyou calibration. That is, obtaining the intrinsic parameter matrix of a monocular camera can include: calibrating the monocular camera using the Zhang Zhengyou calibration method to generate its intrinsic parameter matrix. The Zhang Zhengyou calibration method is a calibration method based on a two-dimensional planar target. It uses a camera to capture multiple images of a planar target, such as a checkerboard pattern, at different angles, and then calculates and analyzes the corner points of the checkerboard pattern to obtain the camera's intrinsic parameter matrix.

[0081] As one possible implementation, the YOLO algorithm, such as YOLOv5, can be used to perform object detection on the image to identify at least five calibrated targets in the prior region. The YOLO algorithm is a real-time object detection algorithm that can quickly identify all objects in the image.

[0082] As a possible implementation, before determining the world coordinate system coordinates and pixel coordinates corresponding to at least 5 calibration targets, it may also include: time synchronization of the monocular camera and depth sensing device to ensure that the acquired image information and depth sensing information are at the same time, thereby improving the accuracy of target detection.

[0083] Step 103: Using the calibration matrix, the world coordinates of each grid point are transformed to the pixel coordinates of the image to generate the inverse mapping matrix.

[0084] It should be noted that, since it is necessary to transform the world coordinate system coordinates to the pixel coordinate system corresponding to the image, it is necessary to determine the transformation relationship between the world coordinate system and the pixel coordinate system. The transformation from the world coordinate system to the pixel coordinate system requires three steps: first, from the world coordinate system to the camera coordinate system; second, from the camera coordinate system to the image coordinate system; and finally, from the image coordinate system to the pixel coordinate system, thus obtaining the transformation relationship between the world coordinate system and the pixel coordinate system.

[0085] Specifically, the first step is to switch from the world coordinate system to the camera coordinate system. For ease of explanation, the center of the depth sensing device can be taken as the origin of the world coordinate system. w ,y w ,z w [x] represents a point in the world coordinate system. The center point of the camera can be taken as the origin of the camera coordinate system, with the camera's line of sight as the positive Z-axis. c ,y c ,z c [] represents a point in the camera coordinate system. The transformation from the world coordinate system to the camera coordinate system is a rigid body transformation, which can be achieved using the rotation matrix R and the translation matrix T to obtain the following relationship:

[0086]

[0087] Furthermore, the next step is to change from the camera coordinate system to the image coordinate system. The center of the image can be used as the origin of the image coordinate system, which is a two-dimensional planar coordinate system. [x] i ,y i The symbol ] represents a point in the image coordinate system. The transformation from the camera coordinate system to the image coordinate system involves a 3D to 2D transformation, which is a perspective transformation. Based on the pinhole camera model, the following relationship can be obtained using the camera intrinsic parameters:

[0088]

[0089] Furthermore, finally, from the image coordinate system to the pixel coordinate system, we can set the top-left corner of the image as the origin of the pixel coordinate system. The pixel coordinate system is a two-dimensional planar coordinate system, where [u,v] represents a point in the next pixel coordinate system. The unit of the image coordinate system is mm, which is a physical unit, while the unit of the pixel coordinate system is pixel. The transformation from the image coordinate system to the pixel coordinate system is an affine transformation, and the following relationship can be obtained using camera intrinsic parameters:

[0090]

[0091] In summary, the conversion from the world coordinate system to the pixel coordinate system can be expressed by the following formula:

[0092]

[0093]

[0094] Among them, let the intrinsic parameter matrix of the monocular camera The extrinsic parameter matrix between depth sensing devices and monocular cameras

[0095] The above formula provides the transformation relationship between the world coordinate system and the pixel coordinate system.

[0096] In one possible implementation, a pixel matrix mask is first created based on the image of the target region. Each pixel coordinate in the pixel matrix mask can correspond to a world coordinate coordinate. For example, if the image size is 1920*1080, then the pixel matrix size is 1920*1080*3. Then, according to the transformation relationship between the world coordinate system and the pixel coordinate system and the calibration matrix, the world coordinate system coordinates corresponding to each grid point are transformed to the pixel coordinate system corresponding to the image, and the world coordinate system coordinates of each pixel coordinate system in the pixel matrix mask are stored.

[0097] It should be noted that since the prior region is a part of the target region, and the pixels in the image do not correspond one-to-one with the world coordinate system points, the result after mapping is a sparse pixel matrix. Some pixel values ​​in this sparse pixel matrix correspond to world coordinate system coordinates, while some pixel values ​​do not have corresponding world coordinate system values. Therefore, the sparse pixel matrix can be upsampled to generate a complete pixel matrix, namely the inverse mapping matrix. Each pixel coordinate system coordinate in the inverse mapping matrix corresponds to a world coordinate system coordinate, thus obtaining the complete mapping relationship between all pixel coordinate system coordinates and world coordinate system coordinates in the image.

[0098] Step 104: Determine the world coordinate system coordinates corresponding to each pixel in the image based on the inverse mapping matrix, and determine the world coordinate system coordinates corresponding to the target to be located based on the pixel in the image where the target to be located is located.

[0099] It should be noted that the mapping relationship between world coordinate system coordinates and pixel coordinate system coordinates within the target area is obtained through the aforementioned steps. Therefore, the world coordinate system coordinates corresponding to any pixel in the image of the target area can be obtained by inputting the pixel coordinate system coordinates corresponding to that pixel into the inverse mapping matrix.

[0100] As one possible implementation, object detection algorithms, such as YOLOv5, can be used to detect objects in the image, obtaining the bounding box [u1, v1, u2, v2] of the target to be located. Then, the world coordinates [x, y, v] of the target can be obtained by indexing the inverse mapping matrix. r ,y r ,zr ], as shown below:

[0101] x r =mask[(u1+u2) / 2,(v1+v2) / 2,0];

[0102] y r =mask[(u1+u2) / 2,(v1+v2) / 2,1];

[0103] z r =mask[(u1+u2) / 2,(v1+v2) / 2,2];

[0104] The positioning method disclosed in the above embodiments of this application first acquires an image of the target area and the world coordinate system coordinates corresponding to each grid point in the prior area of ​​the target area. Then, it calibrates the monocular camera to generate a calibration matrix. Next, it transforms the world coordinate system coordinates corresponding to each grid point to the pixel coordinate system corresponding to the image to generate an inverse mapping matrix. Finally, it determines the world coordinate system coordinates corresponding to each pixel point in the image and the world coordinate system coordinates corresponding to the target to be located. Thus, by transforming the world coordinate system coordinates of the prior area to the pixel coordinate system, a mapping relationship between the coordinates of each pixel point in the image and the world coordinate system coordinates is established. Therefore, the world coordinate system coordinates of the target to be located within the perception area of ​​the monocular camera can be obtained using only the image acquired by the monocular camera, thereby achieving accurate positioning and improving the accuracy and reliability of monocular vision positioning.

[0105] See Figure 4 The diagram illustrates a flowchart of a positioning method provided in Embodiment 2 of this application. Figure 4 As shown, the positioning method may include the following steps:

[0106] Step 401: Obtain the image of the target area and the world coordinate system coordinates corresponding to each grid point in the prior area.

[0107] The image is captured by a monocular camera on the target area. The image contains multiple pixels and the target to be located. The target area contains a prior area, which contains multiple grid points.

[0108] Step 402: Based on the image and the world coordinate system coordinates corresponding to each grid point, calibrate the monocular camera to generate a calibration matrix.

[0109] The specific implementation process and principle of steps 401-402 above can be found in the detailed description of the above embodiments, and will not be repeated here.

[0110] Step 403: Using the calibration matrix, the world coordinates of each grid point are transformed to the pixel coordinates of the image to generate a sparse pixel matrix.

[0111] It should be noted that since the prior region is a part of the target region, and the pixels in the image do not correspond one-to-one with the world coordinate system points, the result after mapping is a sparse pixel matrix. Some pixel values ​​in this sparse pixel matrix correspond to world coordinate system coordinates, while some pixel values ​​do not have corresponding world coordinate system values.

[0112] Specifically, the coordinate systems of each pixel in the pixel matrix mask may fall into three categories: 1. The pixel coordinate system has no corresponding world coordinate system coordinate; in this case, the 3D information corresponding to the pixel coordinate system in the pixel matrix mask is assigned the value None; 2. The pixel coordinate system corresponds to a world coordinate system coordinate; in this case, the 3D information corresponding to the pixel coordinate system in the pixel matrix mask stores the world coordinate system coordinate [x]. w ,y w ,z w ]; 3. The pixel coordinate system coordinates correspond to N (N>1) world coordinate system coordinates. At this time, the three-dimensional information corresponding to the pixel coordinate system coordinates in the pixel matrix mask stores the average value of the coordinates of N world coordinate system points [(x w1 +...+x wN ) / N,(y w1 +...+y wN ) / N,(z w1 +...+z wN ) / N].

[0113] Furthermore, since it is necessary to transform the world coordinate system coordinates to the pixel coordinate system corresponding to the image, it is necessary to determine the transformation relationship between the world coordinate system and the pixel coordinate system. Combined with the calibration matrix, the mapping relationship between each pixel and each grid point can be obtained, thereby generating a sparse pixel matrix. That is, in one possible implementation of this application embodiment, step 403 above may include:

[0114] Transform the world coordinates of each grid point to the pixel coordinates of the image, and determine the mapping relationship between each pixel and each grid point.

[0115] A sparse pixel matrix is ​​generated based on the mapping relationship between each pixel and each grid point.

[0116] Furthermore, the transformation from the world coordinate system to the pixel coordinate system requires three steps: first, from the world coordinate system to the camera coordinate system; second, from the camera coordinate system to the image coordinate system; and finally, from the image coordinate system to the pixel coordinate system. This yields the transformation relationship between the world coordinate system and the pixel coordinate system. In one possible implementation of this application, the transformation of the world coordinate system coordinates corresponding to each grid point to the pixel coordinate system corresponding to the image, and the determination of the mapping relationship between each pixel point and each grid point, may include:

[0117] Transform the world coordinate system of each grid point to the preset camera coordinate system using a rigid body transformation;

[0118] Transform the preset camera coordinate system to the preset image coordinate system using perspective transformation;

[0119] Based on the camera intrinsic parameters, the preset image coordinate system is transformed to the pixel coordinate system through an affine transformation;

[0120] Based on the world coordinate system coordinates and pixel coordinate system corresponding to each grid point, determine the mapping relationship between each pixel point and each grid point.

[0121] The transformation relationship between the world coordinate system and the pixel coordinate system can be found in the relevant content of step 103 above, and will not be repeated here.

[0122] Step 404: Upsample the sparse pixel matrix to generate an inverse mapping matrix.

[0123] In this embodiment of the application, upsampling simply means expanding the sparse pixel matrix, which can be achieved through interpolation algorithms.

[0124] Furthermore, since the pixel matrix mask generated in step 403 is sparse, and the 3D information corresponding to many pixel coordinate system coordinates is None, interpolation can be performed by traversing each pixel coordinate system coordinate in the pixel matrix mask and combining it with the 3D information of the pixels surrounding the pixel coordinate system coordinates where the 3D information is None, until all pixel coordinate system coordinates in the pixel matrix mask correspond to world coordinate system coordinates. That is, in one possible implementation of this application embodiment, step 404 above may include:

[0125] Among all pixels, those with corresponding world coordinate system coordinates are defined as non-nullable points, and those without corresponding world coordinate system coordinates are defined as nullable points.

[0126] Each pixel is traversed, and it is determined whether there is at least one non-nullable pixel among the surrounding pixels of each pixel. If there is at least one non-nullable pixel among the surrounding pixels of any pixel, the mean of the world coordinates of the surrounding pixels of any pixel is assigned to the pixel. If all the surrounding pixels of any pixel are nullable, the next pixel is traversed.

[0127] Repeat the above traversal process until all pixels are non-nullable.

[0128] Based on the world coordinates of each pixel after traversal processing, an inverse mapping matrix is ​​generated.

[0129] Specifically, the process first iterates through each pixel, checking if its corresponding 3D information is None. If not, it moves to the next pixel. If it is None, it checks if all surrounding pixels are None. If all are None, it moves to the next pixel. If not all are None, it calculates the mean of the world coordinates of the surrounding pixels and assigns it to the current pixel's 3D information. After this calculation, it continues iterating to the next pixel. After all pixels have been traversed, it checks if any pixel has a corresponding 3D information value of None. If so, it repeats the above steps until all pixel coordinates in the pixel matrix mask correspond to world coordinates.

[0130] As one possible implementation, in order to ensure the accuracy of upsampling, the surrounding pixels of any pixel are the eight neighboring pixels of any pixel. The eight neighboring pixels refer to the top, bottom, left, right, top left, top right, bottom left, and bottom right of the corresponding position, which are the adjacent and diagonally adjacent positions.

[0131] Step 405: Determine the world coordinate system coordinates corresponding to each pixel in the image based on the inverse mapping matrix, and determine the world coordinate system coordinates corresponding to the target to be located based on the pixel in the image where the target to be located is located.

[0132] The specific implementation process and principle of step 405 above can be found in the detailed description of the above embodiments, and will not be repeated here.

[0133] The localization method disclosed in the above embodiments of this application first acquires an image of the target area and the world coordinate system coordinates corresponding to each grid point in the prior area. Then, based on the image and the world coordinate system coordinates corresponding to each grid point, the monocular camera is calibrated to generate a calibration matrix. Next, using the calibration matrix, the world coordinate system coordinates corresponding to each grid point are transformed to the pixel coordinate system corresponding to the image to generate a sparse pixel matrix. The sparse pixel matrix is ​​further upsampled to generate an inverse mapping matrix. Finally, based on the inverse mapping matrix, the world coordinate system coordinates corresponding to each pixel point in the image are determined, and the world coordinate system coordinates corresponding to the target to be located are determined based on the pixel point where the target to be located is located in the image. Thus, by upsampling the sparse pixel matrix, a complete pixel matrix, i.e., the inverse mapping matrix, is generated. Each pixel coordinate system coordinate in the inverse mapping matrix corresponds to a world coordinate system coordinate, thereby obtaining a complete mapping relationship between all pixel coordinate system coordinates in the image and the world coordinate system coordinates, further improving the accuracy and reliability of monocular vision localization.

[0134] See Figure 5 The diagram shows a structural schematic of a positioning device provided in Embodiment 3 of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0135] The positioning device may specifically include the following modules:

[0136] The acquisition module 501 is used to acquire the image of the target area and the world coordinate system coordinates corresponding to each grid point in the prior area.

[0137] The image is captured by a monocular camera on the target area. The image contains multiple pixels and the target to be located. The target area contains a prior area, which contains multiple grid points.

[0138] The calibration module 502 is used to calibrate the monocular camera and generate a calibration matrix based on the image and the world coordinate system coordinates corresponding to each grid point.

[0139] The transformation module 503 is used to transform the world coordinate system coordinates corresponding to each grid point to the pixel coordinate system corresponding to the image using the calibration matrix, so as to generate the inverse mapping matrix.

[0140] The determination module 504 is used to determine the world coordinate system coordinates corresponding to each pixel in the image based on the inverse mapping matrix, and to determine the world coordinate system coordinates corresponding to the target to be located based on the pixel in the image where the target to be located is located.

[0141] The positioning device disclosed in the above embodiments of this application first acquires an image of the target area and the world coordinate system coordinates corresponding to each grid point in the prior area of ​​the target area. Then, it calibrates the monocular camera to generate a calibration matrix. Next, it transforms the world coordinate system coordinates corresponding to each grid point to the pixel coordinate system corresponding to the image to generate an inverse mapping matrix. Finally, it determines the world coordinate system coordinates corresponding to each pixel point in the image and the world coordinate system coordinates corresponding to the target to be located. Thus, by transforming the world coordinate system coordinates of the prior area to the pixel coordinate system, a mapping relationship between the coordinates of each pixel point in the image and the world coordinate system coordinates is established. Therefore, the world coordinate system coordinates of the target to be located within the perception area of ​​the monocular camera can be obtained using only the image acquired by the monocular camera, thereby achieving accurate positioning and improving the accuracy and reliability of monocular vision positioning.

[0142] In one possible implementation of Embodiment 3 of this application, the calibration matrix includes an intrinsic parameter matrix and an extrinsic parameter matrix. The world coordinate system coordinates corresponding to each grid point in the prior region are obtained through a depth sensing device. The calibration module 502 may specifically include the following sub-modules:

[0143] The first acquisition submodule is used to acquire the intrinsic parameter matrix of the monocular camera.

[0144] The first determination submodule is used to perform target detection on the image to determine at least 5 calibrated targets within the prior region.

[0145] The second determination submodule is used to determine the world coordinate system coordinates and pixel coordinates corresponding to at least 5 calibration targets.

[0146] The third determination submodule is used to determine the extrinsic parameter matrix between the monocular camera and the depth sensing device based on the intrinsic parameter matrix, the world coordinate system coordinates and pixel coordinates corresponding to at least 5 calibration targets.

[0147] In one possible implementation of Embodiment 3 of this application, the first acquisition submodule may specifically include the following units:

[0148] The calibration unit is used to calibrate the monocular camera using the Zhang Zhengyou calibration method to generate the intrinsic parameter matrix of the monocular camera.

[0149] In one possible implementation of Embodiment 3 of this application, the first determining submodule may specifically include the following units:

[0150] The first determining unit is used to perform target detection on the image using the YOLOv5 algorithm to determine at least 5 calibrated targets in the prior region.

[0151] In one possible implementation of Embodiment 3 of this application, the calibration module 502 may further include the following sub-modules, used before determining the world coordinate system coordinates and pixel coordinates corresponding to at least 5 calibration targets:

[0152] The time synchronization submodule is used to synchronize the time of the monocular camera and the depth sensing device.

[0153] In one possible implementation of Embodiment 3 of this application, the conversion module 503 may specifically include the following sub-modules:

[0154] The first generation submodule is used to transform the world coordinates of each grid point to the pixel coordinates of the image using a calibration matrix, so as to generate a sparse pixel matrix.

[0155] The second generation submodule is used to upsample the sparse pixel matrix and generate an inverse mapping matrix.

[0156] In one possible implementation of Embodiment 3 of this application, the first generation submodule may specifically include the following units:

[0157] The second determining unit is used to transform the world coordinate system coordinates corresponding to each grid point to the pixel coordinate system corresponding to the image, and to determine the mapping relationship between each pixel point and each grid point.

[0158] The first generation unit is used to generate a sparse pixel matrix based on the mapping relationship between each pixel and each grid point.

[0159] In one possible implementation of Embodiment 3 of this application, the second determining unit may specifically include the following sub-units:

[0160] The first transformation sub-unit is used to transform the world coordinate system coordinates of each grid point to the preset camera coordinate system through rigid body transformation.

[0161] The second transformation subunit is used to transform the preset camera coordinate system to the preset image coordinate system through perspective transformation.

[0162] The third transformation subunit is used to transform the preset image coordinate system to the pixel coordinate system through affine transformation based on the camera intrinsic parameters.

[0163] The first determining sub-unit is used to determine the mapping relationship between each pixel and each grid point based on the world coordinate system coordinates and pixel coordinate system corresponding to each grid point.

[0164] In one possible implementation of Embodiment 3 of this application, the second generation submodule may specifically include the following units:

[0165] The third determining unit is used to determine, among all pixels, pixels that have corresponding world coordinate system coordinates as non-nullable points, and pixels that do not have corresponding world coordinate system coordinates as nullable points.

[0166] The first processing unit iterates through each pixel, determining whether at least one of its surrounding pixels is a non-nullable pixel. If at least one of the surrounding pixels is a non-nullable pixel, the average world coordinates of the surrounding pixels are assigned to the pixel. If all surrounding pixels are nullable pixels, the next pixel is then iterated through.

[0167] The second processing unit is used to repeat the above traversal processing steps until all pixels are non-nullable.

[0168] The second generation unit is used to generate an inverse mapping matrix based on the world coordinate system coordinates corresponding to each pixel after traversal processing.

[0169] In one possible implementation of Embodiment 3 of this application, the surrounding pixels of any pixel are the eight neighboring pixels of any pixel.

[0170] The positioning device disclosed in the above embodiments of this application first acquires an image of the target area and the world coordinate system coordinates corresponding to each grid point in the prior area. Then, based on the image and the world coordinate system coordinates corresponding to each grid point, the monocular camera is calibrated to generate a calibration matrix. Next, using the calibration matrix, the world coordinate system coordinates corresponding to each grid point are transformed to the pixel coordinate system corresponding to the image to generate a sparse pixel matrix. The sparse pixel matrix is ​​further upsampled to generate an inverse mapping matrix. Finally, based on the inverse mapping matrix, the world coordinate system coordinates corresponding to each pixel point in the image are determined, and the world coordinate system coordinates corresponding to the target to be located are determined based on the pixel point where the target to be located is located in the image. Thus, by upsampling the sparse pixel matrix, a complete pixel matrix, i.e., the inverse mapping matrix, is generated. Each pixel coordinate system coordinate in the inverse mapping matrix corresponds to a world coordinate system coordinate, thereby obtaining a complete mapping relationship between all pixel coordinate system coordinates in the image and the world coordinate system coordinates, further improving the accuracy and reliability of monocular vision positioning.

[0171] The positioning device provided in this application embodiment can be applied in the foregoing method embodiment. For details, please refer to the description of the above method embodiment, which will not be repeated here.

[0172] Figure 6 This is a schematic diagram of the terminal device provided in Embodiment 4 of this application. Figure 6 As shown, the terminal device 600 of this embodiment includes: at least one processor 610 ( Figure 6 (Only one is shown) a processor, a memory 620, and a computer program 621 stored in the memory 620 and executable on the at least one processor 610, wherein the processor 610 executes the computer program 621 to implement the steps in the above-described positioning method embodiments.

[0173] The terminal device 600 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. This terminal device may include, but is not limited to, a processor 610 and a memory 620. Those skilled in the art will understand that... Figure 6 This is merely an example of terminal device 600 and does not constitute a limitation on terminal device 600. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0174] The processor 610 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0175] In some embodiments, the memory 620 may be an internal storage unit of the terminal device 600, such as a hard disk or memory of the terminal device 600. In other embodiments, the memory 620 may be an external storage device of the terminal device 600, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device 600. Furthermore, the memory 620 may include both internal and external storage units of the terminal device 600. The memory 620 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 620 can also be used to temporarily store data that has been output or will be output.

[0176] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0177] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0178] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0179] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0180] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0181] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0182] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0183] The implementation of all or part of the processes in the methods of the above embodiments can also be accomplished by a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the various method embodiments described above.

[0184] The embodiments described above are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A positioning method, characterized in that, include: The image of the target area is acquired, and the world coordinate system coordinates corresponding to each grid point in the prior area are acquired. The image is captured by a monocular camera on the target area. The image includes multiple pixels and contains the target to be located. The target area includes the prior area, which includes multiple grid points. The world coordinate system coordinates corresponding to each grid point in the prior area are acquired by a depth sensing device. Based on the image and the world coordinate system coordinates corresponding to each of the grid points, the monocular camera is calibrated to generate a calibration matrix, which includes an intrinsic parameter matrix and an extrinsic parameter matrix. The world coordinate system of each grid point is transformed to the preset camera coordinate system through rigid body transformation. The preset camera coordinate system is transformed to the preset image coordinate system through perspective transformation; Based on the camera intrinsic parameters, the preset image coordinate system is transformed to the pixel coordinate system through an affine transformation; Based on the world coordinate system coordinates and pixel coordinate system corresponding to each grid point, determine the mapping relationship between each pixel point and each grid point; A sparse pixel matrix is ​​generated based on the mapping relationship between each pixel and each grid point. Among the pixels, pixels with corresponding world coordinate system coordinates are defined as non-nullable points, and pixels without corresponding world coordinate system coordinates are defined as nullable points. Each pixel is traversed to determine whether there is at least one non-nullable pixel among the surrounding pixels of each pixel; if there is at least one non-nullable pixel among the surrounding pixels of any pixel, the mean value of the world coordinates of the surrounding pixels of any pixel is assigned to the pixel; if all the surrounding pixels of any pixel are nullable pixels, the next pixel is traversed. Repeat the above traversal process until all pixels are non-nullable. Based on the world coordinates of each pixel after traversal processing, an inverse mapping matrix is ​​generated. Based on the inverse mapping matrix, the world coordinate system coordinates corresponding to each pixel in the image are determined, and based on the pixel where the target to be located is located in the image, the world coordinate system coordinates corresponding to the target to be located are determined.

2. The positioning method as described in claim 1, characterized in that, The step of calibrating the monocular camera and generating a calibration matrix based on the image and the world coordinate system coordinates corresponding to each of the grid points includes: Obtain the intrinsic parameter matrix of the monocular camera; Target detection is performed on the image to identify at least five calibrated targets within the prior region; Determine the world coordinate system coordinates and pixel coordinates corresponding to the at least 5 calibration targets; The extrinsic parameter matrix between the monocular camera and the depth sensing device is determined based on the intrinsic parameter matrix, the world coordinate system coordinates and pixel coordinates corresponding to the at least 5 calibration targets.

3. The positioning method as described in claim 2, characterized in that, The step of obtaining the intrinsic parameter matrix of the monocular camera includes: The monocular camera is calibrated using Zhang Zhengyou's calibration method to generate the intrinsic parameter matrix of the monocular camera.

4. The positioning method as described in claim 2, characterized in that, The step of performing target detection on the image to determine at least five labeled targets within the prior region includes: The image is subjected to target detection using the YOLOv5 algorithm to identify at least five calibrated targets within the prior region.

5. The positioning method as described in claim 2, characterized in that, Before determining the world coordinate system coordinates and pixel coordinates corresponding to the at least five calibration targets, the method further includes: The monocular camera and the depth sensing device are synchronized in time.

6. The positioning method as described in claim 1, characterized in that, The pixels surrounding any given pixel are the pixels in the eight-neighborhood of that given pixel.

7. A positioning device, characterized in that, include: The acquisition module is used to acquire an image of the target area and to acquire the world coordinate system coordinates corresponding to each grid point in the prior area. The image is captured by a monocular camera on the target area. The image includes multiple pixels and contains the target to be located. The target area includes the prior area, which includes multiple grid points. The world coordinate system coordinates corresponding to each grid point in the prior area are acquired by a depth sensing device. The calibration module is used to calibrate the monocular camera based on the image and the world coordinate system coordinates corresponding to each of the grid points to generate a calibration matrix, the calibration matrix including an intrinsic parameter matrix and an extrinsic parameter matrix; The first transformation subunit is used to transform the world coordinate system coordinates of each grid point to the preset camera coordinate system through rigid body transformation. The second transformation subunit is used to transform the preset camera coordinate system to the preset image coordinate system through perspective transformation; The third transformation subunit is used to transform the preset image coordinate system to the pixel coordinate system through an affine transformation based on the camera intrinsic parameters. The first determining subunit is used to determine the mapping relationship between each pixel and each grid point based on the world coordinate system coordinates and the pixel coordinate system corresponding to each grid point; The first generation unit is used to generate a sparse pixel matrix based on the mapping relationship between each pixel and each grid point; The third determining unit is used to determine, among the various pixels, pixels that have corresponding world coordinate system coordinates as non-nullable points, and pixels that do not have corresponding world coordinate system coordinates as nullable points. The first processing unit is used to traverse each pixel and determine whether there is at least one non-null value among the surrounding pixels of each pixel; if there is at least one non-null value among the surrounding pixels of any pixel, the mean value of the world coordinate system coordinates corresponding to the surrounding pixels of any pixel is assigned to the pixel; if all the surrounding pixels of any pixel are null value points, the next pixel is traversed. The second processing unit is used to repeat the above traversal processing steps until all pixels are non-nullable. The second generation unit is used to generate an inverse mapping matrix based on the world coordinate system coordinates corresponding to each pixel after traversal processing. The determination module is used to determine the world coordinate system coordinates corresponding to each pixel in the image based on the inverse mapping matrix, and to determine the world coordinate system coordinates corresponding to the target to be located based on the pixel in the image where the target to be located is located.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images