A method and device for searching point cloud data based on picture comparison

Abstract

The application provides a method and device for searching point cloud data based on picture comparison, selects a first standard positioning point and a second standard positioning point on the boundary of a to-be-identified image, maps coordinates in a pixel coordinate system to a world coordinate system based on two random depths, positions a line capable of indicating the boundary of the to-be-identified image in the world coordinate system and projects the line in an xoy plane to obtain a first mapping line and a second mapping line, determines a sector displayed by the to-be-identified image based on the first mapping line and the second mapping line, and maps the sector to a radar coordinate system to form point cloud data, so as to mark the point cloud data associated with the to-be-identified image.

Description

Technical Field

[0001] This invention relates to the field of point cloud data processing technology, and in particular to a method and apparatus for finding point cloud data based on image comparison. Background Technology

[0002] Point cloud data is a data format used to represent a set of discrete points in three-dimensional space. Each point contains coordinate information and possibly other attributes, such as color, normal vector, or intensity value. Point cloud data is commonly used in computer graphics, robot perception, geographic information systems, and other fields. There are many ways to generate point cloud data, among which common methods include lidar, a commonly used point cloud data acquisition technology. It scans the surface of an object by emitting a laser beam, then receives the reflected light signal, and calculates the three-dimensional coordinate information of each point on the object's surface based on the signal's time delay and phase difference. LiDAR has advantages such as high precision, high speed, and non-contact operation, and is therefore widely used in topographic surveying, building inspection, road design, and other fields.

[0003] In scenarios with labeled images and point cloud data, there is a need to search for point cloud data based on images. On the one hand, it is necessary to search for point cloud data related to the content displayed in the image. On the other hand, it is necessary to find the location of the target object in the image within the point cloud data.

[0004] In existing technologies, a ray is marked in the point cloud data based on the specified annotation points in the image. However, this method cannot perceive depth position, nor can it mark the range of point cloud data associated with the content displayed in the image. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for finding point cloud data based on image comparison, so as to eliminate or improve one or more defects existing in the prior art and solve the problem that the prior art cannot mark the image display range in point cloud data.

[0006] One aspect of the present invention provides a method for finding point cloud data based on image comparison, the method comprising the following steps:

[0007] Acquire images and point cloud data to be identified for the same target scene, wherein the images to be identified are captured by a set camera in the target scene;

[0008] A first mapping is established between the pixel coordinate system and the imaging plane coordinate system of the image to be identified, a second mapping is established between the imaging plane coordinate system and the camera coordinate system of the set camera, and a third mapping is established between the camera coordinate system and the world coordinate system;

[0009] Based on the pixel coordinate system, obtain the first standard positioning point and its first pixel coordinates in the upper left corner of the image to be identified, and the second standard positioning point and its second pixel coordinates in the lower right corner of the image to be identified;

[0010] Based on the continuous first, second, and third mappings, find the first spatial point of the first standard positioning point at a first random depth and the second spatial point at a second random depth in the world coordinate system, and find the third spatial point of the second standard positioning point at the first random depth and the fourth spatial point at the second random depth in the world coordinate system.

[0011] The line connecting the first spatial point and the second spatial point is projected onto the xoy plane of the world coordinate system to obtain a first mapping line. The line connecting the third spatial point and the fourth spatial point is projected onto the xoy plane of the world coordinate system to obtain a second mapping line. The first mapping line and the second mapping line intersect at a first intersection point. Based on the first intersection point, the first mapping line, and the second mapping line, the first angle range of the image to be identified on the xoy plane of the world coordinate system is calculated.

[0012] A fourth mapping is established between the world coordinate system and the radar coordinate system of the point cloud data. Based on the fourth mapping, the first intersection point and the first angle range are transformed to the second intersection point and the second angle range in the radar coordinate system. The sector area in the radar coordinate system is marked according to the second intersection point and the second angle range. The point cloud data in the sector area under the horizontal view is marked and associated with the image to be identified.

[0013] In some embodiments, the method further includes:

[0014] Obtain the points to be identified in the image to be identified;

[0015] Based on the continuous first, second, and third mappings, the third pixel coordinates of the point to be identified are mapped to the first ray in the world coordinate system; based on the fourth mapping, the first ray is mapped to the second ray in the radar coordinate system.

[0016] Find one or more candidate points in the point cloud data that are within the first set distance range of the second ray, and take the point in the candidate points that is closest to the second intersection point as the target point in the point cloud data corresponding to the point to be identified.

[0017] In some embodiments, the method further includes:

[0018] When no candidate point is found within the first set distance range of the second ray, one or more candidate points within the second set distance range, the third set distance range, ... the Nth set distance range of the second ray are searched one by one based on exponential detection; the point closest to the second intersection point within the distance range of the first candidate point is taken as the target point in the point cloud data corresponding to the point to be identified.

[0019] In some embodiments, the second set distance range is twice the first set distance range, the third set distance range is twice the second set distance range, and so on.

[0020] In some embodiments, the first pixel coordinates are (1,1), the second pixel coordinates are (U-1,V-1), U is the pixel width of the image to be identified, and V is the pixel height of the image to be identified.

[0021] In some embodiments, the method includes:

[0022] For a point (u, v) in the image to be identified within the pixel coordinate system, its projection into the camera coordinate system has a depth of Z. c The coordinates of the point at (X) are c ,Y c Z c ), then (u,v) and (X) c ,Y c Z c The relationship established based on the continuous first and second mappings is as follows:

[0023] ;

[0024] in, This represents the focal length along the x-axis of the camera coordinate system. The focal length, expressed in pixels, represents the focal length along the y-axis of the camera coordinate system. This represents the abscissa of the center point of the image to be identified in the pixel coordinate system. The vertical coordinate of the center point of the image to be identified in the pixel coordinate system is represented.

[0025] In some embodiments, the method includes:

[0026] For the point coordinates (X) in the camera coordinate system c ,Y c Z c ), whose corresponding point coordinates in the world coordinate system are (Xω,Yω,Zω), then (X c ,Y c Z cThe relationship between (Xω,Yω,Zω) and (Xω,Yω,Zω) based on the third mapping is as follows:

[0027] ;

[0028] Where R represents the rotation matrix and t represents the translation matrix.

[0029] In some embodiments, the method further includes: visualizing the point cloud data associated with the image to be identified within the sector area, and the target point in the point cloud data corresponding to the point to be identified.

[0030] On the other hand, the present invention also provides an apparatus for finding point cloud data based on image comparison, including a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the apparatus implements the steps of the method described above.

[0031] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0032] The beneficial effects of the present invention are at least as follows:

[0033] The method and apparatus for finding point cloud data based on image comparison of the present invention selects a first standard positioning point and a second standard positioning point on the boundary of the image to be identified, and maps the coordinates in the pixel coordinate system to the world coordinate system based on two random depths to locate the line that can indicate the boundary of the content displayed in the image to be identified in the world coordinate system, and projects it in the xoy plane to obtain the first mapping line and the second mapping line; based on the first mapping line and the second mapping line, the sector area displayed by the image to be identified is determined and mapped to the radar coordinate system that forms the point cloud data to mark the point cloud data associated with the image to be identified.

[0034] Furthermore, for the point to be identified in the image, it is mapped to a ray in the radar coordinate system. The point closest to the second focus within a set distance range around the ray, i.e. the point closest to the origin of the sector area, is used as the point in the point cloud data corresponding to the point to be identified.

[0035] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0036] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0037] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0038] Figure 1 This is a schematic diagram of the method for finding point cloud data based on image comparison according to an embodiment of the present invention.

[0039] Figure 2 This is a schematic diagram showing the relationship between the pixel coordinate system and the imaging plane coordinate system.

[0040] Figure 3 This is a schematic diagram showing the mapping relationship between points in the imaging plane coordinate system and points in the world coordinate system.

[0041] Figure 4 This is a schematic diagram of the first and second standard positioning points selected in the method for finding point cloud data based on image comparison according to an embodiment of the present invention.

[0042] Figure 5 for Figure 4 A schematic diagram of the ray marked by the first standard positioning point in the world coordinate system and the ray marked by the second standard positioning point in the world coordinate system.

[0043] Figure 6 This is a schematic diagram of the first and second mapping lines in the method for finding point cloud data based on image comparison according to an embodiment of the present invention.

[0044] Figure 7 This is a flowchart illustrating a method for searching point cloud data based on image comparison, according to an embodiment of the present invention, to query target points in point cloud data corresponding to marked points to be identified.

[0045] Figure 8 This is a schematic diagram illustrating the method for searching point cloud data based on image comparison according to an embodiment of the present invention, which queries the target point in the point cloud data corresponding to the marked point to be identified. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0047] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0048] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0049] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0050] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0051] To associate images with point cloud data, mark the point cloud data within the image display area, and find the matching points in the point cloud data for the point to be identified in the image, this invention provides a method for finding point cloud data based on image comparison. This method includes the following steps S101~S106:

[0052] Step S101: Obtain the image to be recognized and point cloud data generated for the same target scene. The image to be recognized is taken by a set camera in the target scene.

[0053] Step S102: Establish a first mapping between the pixel coordinate system of the image to be identified and the imaging plane coordinate system, a second mapping between the imaging plane coordinate system and the camera coordinate system of the camera, and a third mapping between the camera coordinate system and the world coordinate system.

[0054] Step S103: Based on the pixel coordinate system, obtain the first standard positioning point and its first pixel coordinates in the upper left corner of the image to be recognized, and the second standard positioning point and its second pixel coordinates in the lower right corner of the image to be recognized.

[0055] Step S104: Based on the continuous first mapping, second mapping and third mapping, find the first spatial point of the first standard positioning point at the first random depth and the second spatial point at the second random depth in the world coordinate system, and find the third spatial point of the second standard positioning point at the first random depth and the fourth spatial point at the second random depth in the world coordinate system.

[0056] Step S105: Project the line connecting the first spatial point and the second spatial point onto the xoy plane of the world coordinate system to obtain the first mapping line. Project the line connecting the third spatial point and the fourth spatial point onto the xoy plane of the world coordinate system to obtain the second mapping line. The first mapping line and the second mapping line intersect at the first intersection point. Calculate the first angle range of the image to be identified on the xoy plane of the world coordinate system based on the first intersection point, the first mapping line, and the second mapping line.

[0057] Step S106: Establish the fourth mapping between the world coordinate system and the radar coordinate system of the point cloud data. Based on the fourth mapping, transform the first intersection point and the first angle range to the second intersection point and the second angle range in the radar coordinate system. Mark the sector area in the radar coordinate system according to the second intersection point and the second angle range. Mark the point cloud data in the sector area under the horizontal view and associate it with the image to be identified.

[0058] In step S101, the image to be identified and the point cloud data are generated based on the same scene, which is a prerequisite for identification and association. The image to be identified is captured by a camera, and the point cloud data can be generated by radar scanning or acquired by a depth camera. This embodiment discusses point cloud data generated by radar scanning. For the image to be identified, it involves four types of coordinate systems in the vision system: pixel coordinate system, imaging plane coordinate system, camera coordinate system, and world coordinate system.

[0059] In the pixel coordinate system, a point (u, v) is measured in pixels, representing discrete image coordinates or pixel coordinates, with the origin at the top left corner of the image. In the imaging plane coordinate system, a point (x, y) is measured in meters or millimeters, representing continuous image coordinates or spatial coordinates, established with the intersection of the image's diagonals as the reference origin. In the camera coordinate system, a point (Xc, Yc, Zc) represents the camera's coordinates from its own angle, with the origin at the camera's optical center and the Z-axis parallel to the camera's optical axis, i.e., the camera's lens direction. In the world coordinate system, a point (Xω, Yω, Zω) serves as a reference for the target object's position. The point's position can be freely set for computational convenience, and can be located on the robot's base or the robot's front actuator. The camera coordinate system and the world coordinate system can be transformed through rotation and translation. Furthermore, the world coordinate system and the radar coordinate system that generates the point cloud data can also be directly transformed through rotation and translation.

[0060] In step S102, the transformation relationship between a point (u,v) in the pixel coordinate system and a point (x,y) in the imaging plane coordinate system is as follows: Figure 2 As shown, the formula for calculating scaling and translation is:

[0061] ; (1)

[0062] Where α and β are scaling factors, in pixels per meter. This represents the abscissa of the center point of the image to be identified in the pixel coordinate system. The vertical coordinate of the center point of the image to be identified in the pixel coordinate system is represented.

[0063] The focal length of a point (x, y) in the imaging plane coordinate system is... The corresponding depth Z in the world coordinate system c The coordinates of the point (X) c ,Y c Z c ), conversion relationship as Figure 2 As shown, then:

[0064] ; (2)

[0065] but:

[0066] ; (3)

[0067] After sorting, we can obtain:

[0068] ; (4)

[0069] like Figure 3 As shown, between the camera coordinate system and the pixel coordinate system, for a point (u,v) in the image to be recognized in the pixel coordinate system, its projection into the camera coordinate system has a depth of Z. c The coordinates of the point at (X) are c ,Y c Z c ), then (u,v) and (X) c ,Y c Z c The relationship established based on the continuous first and second mappings is as follows:

[0070] ; (5)

[0071] Or adjust to: ; (6)

[0072] in, This represents the focal length along the x-axis of the camera coordinate system. The focal length, expressed in pixels, represents the focal length along the y-axis of the camera coordinate system. This represents the x-coordinate of the center point of the image to be identified in the pixel coordinate system. This represents the ordinate of the center point of the image to be identified in the pixel coordinate system.

[0073] The camera's projection matrix converts coordinates in the camera coordinate system to pixel coordinates, while the inverse matrix converts coordinates in pixel coordinates to camera coordinates. Here, the inverse of the projection matrix is ​​applied to the two 3D coordinates of the top-left corner and the two 3D coordinates of the bottom-right corner to transform these points into points in the camera coordinate system.

[0074] Based on equation (5) or (6), coordinate transformation between a point in the image coordinate system and a point at a specified depth Zc in the camera coordinate system can be achieved, i.e., a continuous first mapping and a second mapping. For example, take two Zc points. c If the values ​​are 10 and 50, then the two three-dimensional coordinates of the upper left corner are (10,10,10) and (50,50,50), respectively, and the two three-dimensional coordinates of the lower right corner are (10*(x-1), 10*(y-1), 10), (50*(x-1), 50*(y-1), 50).

[0075] Furthermore, for the point coordinates (X...) in the camera coordinate system... c ,Y c Z c ), whose corresponding point coordinates in the world coordinate system are (Xω,Yω,Zω), then (X c ,Y c Z c The relationship between (Xω,Yω,Zω) and (Xω,Yω,Zω) based on the third mapping is as follows:

[0076] ; (7)

[0077] Where R represents the rotation matrix and t represents the translation matrix. Based on the camera's extrinsic parameters... It can convert points in the camera coordinate system to the world coordinate system.

[0078] Equation (7) manages the camera coordinate system and the world coordinate system, i.e., the third mapping.

[0079] In steps S103 and S104, such as Figure 4 As shown, the first standard positioning point and the second standard positioning point are located at the upper left and lower right positions in the image to be identified. In some embodiments, the first pixel coordinate is p (1,1), the second pixel coordinate is q (U-1,V-1), U is the pixel width of the image to be identified, and V is the pixel height of the image to be identified.

[0080] By setting two random depths, we can find the corresponding points in the world coordinate system at those two depths, corresponding to the first standard positioning point p and the second standard positioning point q. Figure 5 As shown, the first standard positioning point p corresponds to p1 and p2, and the second standard positioning point q corresponds to q1 and q2.

[0081] p1 and p2, q1 and q2 are the points corresponding to p and q at two depths in the world coordinate system calculated using the above equations (5), (6), and (7).

[0082] In step S105, as Figure 6 As shown, the line connecting p1 and p2 is projected onto the xoy plane to obtain the first mapping line, and the line connecting q1 and q2 is projected onto the xoy plane to obtain the second mapping line, which intersects at the first intersection point M. The angle formed by the first mapping line, the first intersection point, and the second mapping line is the display angle of the image to be recognized in the xoy plane. This application only cares about the rotation radian of each line in the xoy plane and the origin. Through two three-dimensional world coordinate points on each line, using only the x and y coordinates, the function representation of this line can be obtained. The intersection point of the two lines can be calculated in the xoy plane, and this intersection point is the origin of the sector. Because the coordinates of the points involved in the calculation are all in the world coordinate system, the origin of the sector is also based on the world coordinate system. At this time, the point representing the slope of the two lines can also be obtained. The y / x of this point is the slope of the original line, let's assume it is p1 or p2.

[0083] In step S106, the intersection point M and the included angle obtained in step S105 are transformed into the radar coordinate system through a fourth mapping, specifically through a four-dimensional translation and rotation matrix. This ultimately yields the range of the horizontal viewing angle within the radar coordinate system, and the point cloud data within this range is then incorporated into the image to be identified.

[0084] Rendering point clouds is based on the radar coordinate system. A transformation matrix from the world coordinate system to the radar coordinate system can be obtained from the position of each radar point. The origin of the sector needs to be calculated into point coordinates in the radar coordinate system using the transformation matrix. Similarly, p1 and p2 also need to be transformed using the rotation parameters of the transformation matrix to obtain their coordinates in the radar coordinate system. The rotation radian value of each line is then obtained using the arctan(y / x) value of each point's current coordinates. With the origin of the sector and the rotation radians of the two sides, the sector block in the radar coordinate system can be drawn to indicate the associated point cloud data.

[0085] In some embodiments, such as Figure 7 and Figure 8 As shown, the method further includes steps S201 to S203:

[0086] Step S201: Obtain the points to be identified in the image to be identified.

[0087] Step S202: Based on the continuous first mapping, second mapping and third mapping, map the third pixel coordinates of the point to be identified to the first ray in the world coordinate system; based on the fourth mapping, map the first ray to the second ray in the radar coordinate system.

[0088] Step S203: Find one or more candidate points in the point cloud data that are within the first set distance range of the second ray, and take the point that is closest to the second intersection point among the candidate points as the target point in the point cloud data corresponding to the point to be identified.

[0089] In some embodiments, the method further includes:

[0090] When no candidate point is found within the first set distance range of the second ray, one or more candidate points within the second set distance range, third set distance range, ... Nth set distance range of the second ray are searched one by one based on exponential detection; the point closest to the second intersection point within the distance range of the first candidate point is taken as the target point in the point cloud data corresponding to the point to be identified.

[0091] In some embodiments, the second set distance range is twice the first set distance range, the third set distance range is twice the second set distance range, and so on. For example, the first set distance range is 0.1m, the second set distance range is 0.2m, and the third set distance range is 0.4m.

[0092] In some embodiments, the method further includes: visualizing the point cloud data of the image to be identified within the sector area, and the target point in the point cloud data corresponding to the identification point.

[0093] On the other hand, the present invention also provides an apparatus for finding point cloud data based on image comparison, including a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the apparatus implements the steps of the method described above.

[0094] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0095] In summary, the method and apparatus for finding point cloud data based on image comparison of the present invention selects a first standard positioning point and a second standard positioning point on the boundary of the image to be identified, maps the coordinates in the pixel coordinate system to the world coordinate system based on two random depths, so as to locate the line that can indicate the boundary of the content displayed in the image to be identified in the world coordinate system, and projects it in the xoy plane to obtain the first mapping line and the second mapping line; based on the first mapping line and the second mapping line, the sector area displayed by the image to be identified is determined and mapped to the radar coordinate system that forms the point cloud data, so as to mark the point cloud data associated with the image to be identified.

[0096] Furthermore, for the point to be identified in the image, it is mapped to a ray in the radar coordinate system. The point closest to the second focus within a set distance range around the ray, i.e. the point closest to the origin of the sector area, is used as the point in the point cloud data corresponding to the point to be identified.

[0097] Corresponding to the above method, the present invention also provides an apparatus / system including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system performs the steps of the method as described above.

[0098] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0099] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0100] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0101] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0102] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for finding point cloud data based on image comparison, characterized in that, The method includes the following steps: Acquire images and point cloud data to be identified for the same target scene, wherein the images to be identified are captured by a set camera in the target scene; A first mapping is established between the pixel coordinate system and the imaging plane coordinate system of the image to be identified, a second mapping is established between the imaging plane coordinate system and the camera coordinate system of the set camera, and a third mapping is established between the camera coordinate system and the world coordinate system; Based on the pixel coordinate system, obtain the first standard positioning point and its first pixel coordinates in the upper left corner of the image to be identified, and the second standard positioning point and its second pixel coordinates in the lower right corner of the image to be identified; Based on the continuous first, second, and third mappings, find the first spatial point of the first standard positioning point at a first random depth and the second spatial point at a second random depth in the world coordinate system, and find the third spatial point of the second standard positioning point at the first random depth and the fourth spatial point at the second random depth in the world coordinate system. The line connecting the first spatial point and the second spatial point is projected onto the xoy plane of the world coordinate system to obtain a first mapping line. The line connecting the third spatial point and the fourth spatial point is projected onto the xoy plane of the world coordinate system to obtain a second mapping line. The first mapping line and the second mapping line intersect at a first intersection point. Based on the first intersection point, the first mapping line, and the second mapping line, the first angle range of the image to be identified on the xoy plane of the world coordinate system is calculated. A fourth mapping is established between the world coordinate system and the radar coordinate system of the point cloud data. Based on the fourth mapping, the first intersection point and the first angle range are transformed to the second intersection point and the second angle range in the radar coordinate system. The sector area in the radar coordinate system is marked according to the second intersection point and the second angle range. The point cloud data in the sector area under the horizontal view is marked and associated with the image to be identified.

2. The method for finding point cloud data based on image comparison according to claim 1, characterized in that, The method further includes: Obtain the points to be identified in the image to be identified; Based on the continuous first, second, and third mappings, the third pixel coordinates of the point to be identified are mapped to the first ray in the world coordinate system; based on the fourth mapping, the first ray is mapped to the second ray in the radar coordinate system. Find one or more candidate points in the point cloud data that are within the first set distance range of the second ray, and take the point in the candidate points that is closest to the second intersection point as the target point in the point cloud data corresponding to the point to be identified.

3. The method for finding point cloud data based on image comparison according to claim 2, characterized in that, The method further includes: When no candidate point is found within the first set distance range of the second ray, one or more candidate points within the second set distance range, the third set distance range, ... the Nth set distance range of the second ray are searched one by one based on exponential detection; the point closest to the second intersection point within the distance range of the first candidate point is taken as the target point in the point cloud data corresponding to the point to be identified.

4. The method for finding point cloud data based on image comparison according to claim 3, characterized in that, The second set distance range is twice the first set distance range, the third set distance range is twice the second set distance range, and so on.

5. The method for finding point cloud data based on image comparison according to claim 1, characterized in that, The first pixel coordinates are (1,1), the second pixel coordinates are (U-1,V-1), U is the pixel width of the image to be identified, and V is the pixel height of the image to be identified.

6. The method for finding point cloud data based on image comparison according to claim 1, characterized in that, In the method: For a point (u, v) in the image to be identified within the pixel coordinate system, its projection into the camera coordinate system has a depth of Z. c The coordinates of the point at (X) are c ,Y c Z c ), then (u,v) and (X) c ,Y c Z c The relationship established based on the continuous first and second mappings is as follows: ； in, This represents the focal length along the x-axis of the camera coordinate system. The focal length, expressed in pixels, represents the focal length along the y-axis of the camera coordinate system. This represents the abscissa of the center point of the image to be identified in the pixel coordinate system. The vertical coordinate of the center point of the image to be identified in the pixel coordinate system is represented.

7. The method for finding point cloud data based on image comparison according to claim 6, characterized in that, In the method: For the point coordinates (X) in the camera coordinate system c ,Y c Z c ), whose corresponding point coordinates in the world coordinate system are (Xω,Yω,Zω), then (X c ,Y c Z c The relationship between (Xω,Yω,Zω) and (Xω,Yω,Zω) based on the third mapping is as follows: ； Where R represents the rotation matrix and t represents the translation matrix.

8. The method for finding point cloud data based on image comparison according to any one of claims 2 to 3, characterized in that, The method further includes: The point cloud data associated with the image to be identified within the sector area, and the target point in the point cloud data corresponding to the point to be identified are visualized.

9. A device for searching point cloud data based on image comparison, comprising a processor and a memory, characterized in that, The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 8.

Images