Point cloud coarse registration method and apparatus, and device

Abstract

Provided in the embodiments of the present disclosure are a point cloud coarse registration method and apparatus, and a device. The method comprises: using a plurality of acquisition devices to simultaneously perform, on the same horizontal plane, point cloud acquisition on a target object, so as to obtain a plurality of pieces of point cloud data of the target object, the point cloud data obtained by adjacent acquisition devices overlapping partially; respectively performing discretization projection on the plurality of pieces of point cloud data, so as to obtain a bird's eye view corresponding to each piece of point cloud data (S102); respectively performing two-dimensional image scale-invariant local feature detection on the plurality of bird's eye views obtained, so as to obtain feature data corresponding to each bird's eye view (S103); performing feature matching on the plurality of pieces of feature data obtained, so as to obtain a point cloud transformation matrix (S104); and, on the basis of the point cloud transformation matrix, performing coarse registration on the plurality of pieces of point cloud data of the target object (S105).

Description

Point cloud coarse registration method, device and equipment

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present disclosure claims priority to Chinese patent application CN202311540389.4, filed on November 17, 2023, entitled “Point cloud coarse registration method, device and apparatus,” the entire contents of which are incorporated by reference into the present disclosure. Technical Field

[0003] The present disclosure relates to the field of computer vision technology, and more particularly to a method, apparatus, and device for coarse point cloud registration. Background Art

[0004] With the rapid development of high-precision sensors such as LiDAR, point clouds have gradually become the primary data format for representing the three-dimensional world. Point cloud registration is crucial in many applications, such as 3D modeling, object detection and tracking, and augmented reality. In practical applications, alignment errors between point clouds are inevitable due to factors such as sensor errors, noise, and non-rigid deformation. Furthermore, the large number of point clouds collected by sensors imposes a significant time burden on point cloud registration algorithms. Related coarse point cloud registration methods suffer from technical issues such as low efficiency and time consumption.

[0005] Summary of the Invention

[0006] The embodiments of the present disclosure provide a point cloud coarse registration method, apparatus, and device to solve the technical problems of low efficiency and long time consumption of existing point cloud coarse registration methods.

[0007] In a first aspect, an embodiment of the present disclosure provides a point cloud coarse registration method, comprising: using multiple acquisition devices to simultaneously perform point cloud acquisition on a target object on the same horizontal plane, obtaining multiple point cloud data of the target object, and the point cloud data obtained by adjacent acquisition devices partially overlap; performing discretization projection on the multiple point cloud data respectively to obtain a bird's-eye view corresponding to each point cloud data; performing two-dimensional image scale-invariant local feature detection on the multiple bird's-eye views obtained respectively to obtain feature data corresponding to each bird's-eye view; performing feature matching on the multiple feature data obtained to obtain a point cloud transformation matrix; and coarsely registering the multiple point cloud data of the target object according to the point cloud transformation matrix.

[0008] In a second aspect, an embodiment of the present disclosure provides a point cloud coarse registration device, including: an acquisition module, configured to use multiple acquisition devices to simultaneously perform point cloud acquisition on a target object on the same horizontal plane, to obtain multiple point cloud data of the target object, and the point cloud data obtained by adjacent acquisition devices partially overlap; a projection module, configured to perform discretized projection on multiple point cloud data respectively, to obtain a bird's-eye view corresponding to each point cloud data; a detection module, configured to perform two-dimensional image scale-invariant local feature detection on the multiple bird's-eye views obtained respectively, to obtain feature data corresponding to each bird's-eye view; a matching module, configured to perform feature matching on the multiple feature data obtained, to obtain a point cloud transformation matrix; and a registration module, configured to perform coarse registration on multiple point cloud data of the target object according to the point cloud transformation matrix.

[0009] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: at least one processor and a memory; the memory stores computer-executable instructions; and at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the point cloud coarse registration method as described in any one of the first aspects.

[0010] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement the point cloud coarse registration method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0012] FIG1 is a flow chart of a point cloud coarse registration method provided by one embodiment of the present disclosure;

[0013] FIG2 is a schematic diagram of a bird's-eye view provided by an embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram of the direction of the SIFT descriptor;

[0015] Figure 4 is a schematic diagram of a random KD tree;

[0016] FIG5 is a schematic diagram of a point cloud coarse registration result provided by an embodiment of the present disclosure;

[0017] FIG6 is a schematic structural diagram of a point cloud coarse registration device provided by an embodiment of the present disclosure;

[0018] FIG7 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

[0019] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0020] The present disclosure is further described in detail below through specific embodiments in conjunction with the accompanying drawings. Similar elements in different embodiments use associated similar element numbers. In the following embodiments, many detailed descriptions are intended to enable the present disclosure to be better understood. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other elements, materials, and methods. In some cases, some operations related to the present disclosure are not shown or described in the specification. This is to avoid the core part of the present disclosure being overwhelmed by too much description. For those skilled in the art, it is not necessary to describe these related operations in detail. They can fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0021] In addition, the features, operations, or characteristics described in the specification may be combined in any appropriate manner to form various embodiments. Furthermore, the steps or actions in the method description may be reordered or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various sequences in the specification and drawings are provided solely for the purpose of clearly describing a particular embodiment and are not intended to be mandatory, unless otherwise specified.

[0022] The serial numbers assigned to components herein, such as "first," "second," etc., are used solely to distinguish the objects being described and do not convey any sequential or technical meaning. References to "connection" and "coupling" in this disclosure, unless otherwise specified, include both direct and indirect connections (couplings).

[0023] With the rapid development of high-precision sensors such as LiDAR (Light Detection and Ranging), point clouds have gradually become the primary data format for representing the three-dimensional world. However, since a single sensor can only scan and acquire data within a limited field of view, it is necessary to register the data collected by multiple sensors to generate a complete three-dimensional scene. Point cloud registration technology estimates the transformation matrix between two scanned point clouds. Based on this transformation matrix, multiple partial scanned point clouds of the same three-dimensional scene or object can be merged into a complete three-dimensional point cloud. Point cloud registration is crucial in many applications, such as 3D modeling, object detection and tracking, and augmented reality. From the most classic point cloud registration algorithm—iterative closest point (ICP)—to later improved ICP algorithms, it can be seen that such algorithms generally require coarse registration of the point clouds to obtain a good initial value. Therefore, coarse point cloud registration is an important prerequisite for achieving high-quality 3D reconstruction and object recognition.

[0024] Most existing coarse registration methods reduce geometric reprojection errors through a two-step process: correspondence search and transformation estimation. These two steps are performed alternately until the geometric reprojection error is minimized. However, in practical applications, alignment errors between point clouds are inevitable due to factors such as sensor errors, noise, and non-rigid deformation. Furthermore, the large number of point clouds collected by sensors imposes a significant time burden on point cloud registration algorithms. Existing coarse point cloud registration methods are inefficient and time-consuming, making them difficult to meet the demands of applications with high real-time requirements.

[0025] Point cloud registration is also called point cloud stitching. The overall idea is to utilize the feature information of multiple point cloud data, match them through algorithms to identify the overlapping parts of the point cloud data, and then transform the point cloud data originally belonging to one coordinate system to another coordinate system through point cloud transformation, thereby restoring the original appearance of the entire object. The original point cloud data can be collected by a 3D laser scanning device, and its point position information can include 3D coordinate information, reflection intensity information, etc. Usually, multiple devices collect objects from different angles at the same time. Therefore, the point cloud data collected by a sensor usually only contains part of the object's information. Therefore, in order to perform point cloud registration, the point cloud data collected by adjacent devices must have overlapping parts.

[0026] To improve the efficiency of point cloud coarse registration, shorten its time consumption, and enhance its real-time performance, thus meeting the needs of application scenarios with high real-time requirements, this disclosure reduces the three-dimensional registration problem to two dimensions through a bird's-eye view. Compared to three-dimensional space, operations such as feature extraction and feature matching on a two-dimensional plane have lower time complexity, thereby fundamentally improving registration efficiency. The method provided by this disclosure will be further elaborated through specific examples below.

[0027] FIG1 is a flow chart of a point cloud coarse registration method provided by an embodiment of the present disclosure. As shown in FIG1 , the point cloud coarse registration method provided by this embodiment may include steps S101 to S105.

[0028] S101. Use multiple acquisition devices to simultaneously acquire point clouds of a target object on the same horizontal plane, obtaining multiple point cloud data of the target object, with the point cloud data obtained by adjacent acquisition devices partially overlapping. In this embodiment, the target object can be, for example, an object or a scene. To achieve point cloud registration, in this embodiment, the acquisition content is kept intersecting, ensuring point cloud overlap, i.e., point cloud data obtained by adjacent acquisition devices partially overlap.

[0029] S102: Discretize and project the multiple point cloud data to obtain a bird's-eye view corresponding to each point cloud data. In this embodiment, the point cloud bird's-eye view is obtained by discretizing and projecting the multiple point cloud data obtained in step S101 on the XOY coordinate plane.

[0030] S103: Perform two-dimensional image scale-invariant local feature detection on each of the multiple bird's-eye views to obtain feature data corresponding to each bird's-eye view. The feature data corresponding to each bird's-eye view is obtained by performing two-dimensional image scale-invariant feature transform (SIFT) local feature detection on the bird's-eye view obtained in step S102.

[0031] S104: Perform feature matching on the obtained multiple feature data to obtain a point cloud transformation matrix.

[0032] The feature data obtained in step S103 is further screened and matched to obtain well-matched features, and wrong matches are eliminated, and then a point cloud transformation matrix is ​​generated based on the correct matching results.

[0033] S105 , performing coarse registration on multiple point cloud data of the target object according to the point cloud transformation matrix.

[0034] The source point cloud is transformed into a target matrix using the point cloud transformation matrix generated in step S104, completing the coarse registration. Coarse registration is performed on point clouds collected by the radar at different locations on the same horizontal plane. Since the height error of each point cloud comes only from the difference in the optical center position of different radars, only the X and Y dimensions of the point cloud are operated on.

[0035] The point cloud coarse registration method provided in this embodiment uses multiple acquisition devices to simultaneously perform point cloud acquisition on the target object on the same horizontal plane, thereby obtaining multiple point cloud data of the target object, and the point cloud data obtained by adjacent acquisition devices partially overlap; the multiple point cloud data are discretized and projected respectively to obtain a bird's-eye view corresponding to each point cloud data; the obtained multiple bird's-eye views are respectively subjected to two-dimensional image scale-invariant local feature detection to obtain feature data corresponding to each bird's-eye view; feature matching is performed on the obtained multiple feature data to obtain a point cloud transformation matrix; and the multiple point cloud data of the target object are coarsely registered according to the point cloud transformation matrix. Through discretized projection, the three-dimensional point cloud is converted into a two-dimensional bird's-eye view, and feature extraction, feature matching and other operations are performed on the two-dimensional plane, reducing the registration problem from three dimensions to two dimensions. Compared with three-dimensional space, the time complexity and the spatial complexity of the calculation are reduced, thereby fundamentally improving the matching efficiency and shortening the matching time.

[0036] On the basis of the above embodiment, the following will further elaborate on how to create a bird's-eye view of each point cloud data. On the basis of the above embodiment, in the point cloud coarse registration method provided in this embodiment, a plurality of point cloud data are discretized and projected respectively to obtain a bird's-eye view corresponding to each point cloud data, including steps S1021 to S1023.

[0037] S1021. Set a region of interest in each point cloud data and determine the actual scale corresponding to each pixel in the bird's-eye view.

[0038] In order to obtain a bird's-eye view of the point cloud, we first set the region of interest (x min :x max ,y min :y max , z min :z max ) and set an origin within the region of interest. Next, calculate the point cloud coordinates of this origin in the bird's-eye view. Next, subtract the point cloud values ​​corresponding to the origin from the X and Y values ​​of all point clouds in the region of interest, converting the X and Y values ​​of all point clouds in the region to positive values. Finally, define the actual scale res corresponding to each pixel in the bird's-eye view. For example, you can set one pixel to represent the actual size of a 0.1*0.1 area within the region of interest.

[0039] S1022: Determine the pixel coordinates of each point cloud in the region of interest in the bird's-eye view according to the correspondence between the pixels and the actual scale.

[0040] Based on the correspondence between pixels and actual scales determined in step S1021, combined with the difference between the point cloud axis direction and the image axis direction, the pixel coordinates of each point cloud in the region of interest in the bird's-eye view are calculated. If the calculated pixel coordinates are decimals, they are rounded according to the following unified scale:

[0041] Where: (x 2D ,y 2D ) is the image coordinate, (x 3D ,y 3D ) is the point cloud coordinate, res refers to the scale ratio, x min is the minimum value of the X axis of the region of interest, y min is the minimum value of the Y axis of the region of interest.

[0042] S1023 , determining the pixel value of each pixel in the bird's-eye view according to the maximum value of multiple point cloud height values ​​corresponding to each pixel of the bird's-eye view and the height of the detection range, and obtaining the bird's-eye view corresponding to each point cloud data.

[0043] Since the resolution of the bird's-eye view is only related to the size of the correspondence between pixels and the actual scale, after determining the projection relationship between the point cloud and the bird's-eye view, it is necessary to calculate the pixel value of each pixel in the bird's-eye view. The pixel value represents the height information of the actual point cloud at that pixel position. Since the relationship between the pixels and the point cloud in the bird's-eye view is one-to-many, in this embodiment, the maximum value of the point cloud height value within a single pixel is divided by the height of the detection range, and the result is mapped to the range of 0 to 255. Specifically, the pixel value can be determined according to the following expression:

[0044] Where: H i is the height value of the i-th unit in the top view; S i is the set of points whose top view projection falls on the i-th unit; H j is the height value of the jth point in the unit; H min and H max They are the lowest and highest heights of the intercepted point cloud space respectively.

[0045] Please refer to Figure 2, which is a bird's-eye view diagram provided by one embodiment of the present disclosure. As shown in Figure 2, the first row shows the original point cloud data collected by the radar in a three-dimensional coordinate system; the second row shows a simulated perspective, projecting the point cloud data onto the XOY plane and rasterizing it according to a manually set resolution; the third row calculates the maximum Z coordinate of each point cloud in each grid cell proportionally to the pixel value, thereby converting the three-dimensional point cloud into a two-dimensional depth map.

[0046] The point cloud coarse registration method provided in this embodiment, based on the above embodiments, further improves the accuracy of the bird's-eye view and helps improve the accuracy of point cloud coarse registration by setting a region of interest in each point cloud data and determining the actual scale corresponding to each pixel in the bird's-eye view; determining the pixel coordinates of each point cloud in the region of interest in the bird's-eye view based on the correspondence between the pixel and the actual scale; and determining the pixel value of each pixel in the bird's-eye view based on the maximum value of multiple point cloud height values ​​corresponding to each bird's-eye view pixel and the height of the detection range.

[0047] Based on any of the above embodiments, the following further details how to extract feature data from a bird's-eye view image for matching. Based on the above embodiments, the point cloud coarse registration method provided in this embodiment performs two-dimensional image scale-invariant local feature detection on each of the multiple bird's-eye view images to obtain feature data corresponding to each bird's-eye view image. Specifically, this method may include steps S1031 to S1034.

[0048] S1031. Use a Gaussian filter function to repeatedly downsample each bird's-eye view image.

[0049] First, we perform scale space extrema detection. Specifically, we search for image locations at all scales and use Gaussian differential functions to identify potential scale- and rotation-invariant points of interest. The scale space L(x, y, σ) can be defined as the convolution operation of the scale-variable Gaussian function G(x, y, σ) with the original image I(x, y), as follows: L(x,y,σ)=I(x,y)*G(x,y,σ)

[0050] Among them, G(x,y,σ) is the Gaussian kernel function; (x,y) refers to the coordinates of the space; L(x,y,σ) is the scale space of the image; I(x,y) is the original image; the size of σ reflects the smoothness of the image; * is the convolution operator.

[0051] By repeatedly downsampling the bird's-eye view image using a Gaussian filter function, we can directly obtain and output a series of trapezoidal images of varying sizes, known as a Gaussian pyramid. By subtracting the images within each group of the Gaussian pyramid, we can obtain a Gaussian difference pyramid.

[0052] S1032: Compare each pixel with its adjacent points in the image domain and the scale space domain to obtain dual local extreme points.

[0053] To find the extreme points of the Gaussian function, each pixel is compared with its neighbors in both the image and scale space domains. In the two-dimensional image space, each pixel is compared with its eight neighbors; in the same scale space, the central pixel is compared with its 18 neighbors. This yields dual local extreme points in both the scale space and the two-dimensional image space. By comparing each pixel with its neighbors in both the image and scale space domains, dual local extreme points can be found.

[0054] S1033. Remove unstable and misdetected extreme points from the dual local extreme points to obtain stable extreme points. Because noise and edges can cause sudden changes in Gaussian values, it is necessary to further confirm and filter the previously obtained local extreme points to remove unstable and misdetected extreme points. At the same time, it is also necessary to determine the exact location of the extreme points obtained in the downsampled image in the original image. By removing unstable and misdetected extreme points from the dual local extreme points, stable extreme points can be obtained.

[0055] S1034. Use gradient solution to determine the direction of the stable extreme point, and obtain feature data corresponding to each bird's-eye view.

[0056] After obtaining a stable extreme point, its direction is then determined. The key point direction determination process can specifically include: extracting stable extreme points in different scale spaces to ensure scale invariance. To achieve rotational invariance of the key point to the image, a gradient solution is used to determine the key point direction. Define L(x,y) as the original image space function, then the gradient amplitude of any key point is as follows:

[0057] The gradient directions are as follows:

[0058] Keypoint description: Keypoint description mathematically represents a keypoint and its surrounding contributing pixels. It is a key step in image feature point matching. By dividing the area around each keypoint and assigning eight directions to each subregion, a 128-dimensional vector is constructed as a SIFT descriptor. Each vector represents a description of a keypoint. The construction of a keypoint descriptor ensures that feature points have fixed information such as position, scale, and orientation.

[0059] Please refer to Figure 3, which shows a schematic diagram of the SIFT descriptor's orientation. As shown in Figure 3, for each keypoint, a 4*4*8-dimensional vector is constructed to calculate the gradient magnitude and direction of the surrounding area. The direction with the maximum gradient magnitude is selected as the primary direction. Furthermore, directions with a relative gradient magnitude that is 80% higher are selected as secondary directions to improve the robustness of the algorithm and the stability of feature matching.

[0060] The point cloud coarse registration method provided in this embodiment, based on the above embodiment, further utilizes a Gaussian filter function to repeatedly downsample each bird's-eye view image; compares each pixel with its adjacent points in the image domain and the scale space domain to obtain dual local extreme points; removes unstable and misdetected extreme points from the dual local extreme points to obtain stable extreme points; uses gradient solution to determine the direction of the stable extreme points to obtain feature data corresponding to each bird's-eye view image. The stable features help improve the robustness and stability of point cloud matching.

[0061] Based on any of the above embodiments, the following further details how to generate a point cloud transformation matrix for point cloud registration. Based on the above embodiments, the point cloud coarse registration method provided in this embodiment performs feature matching on the obtained multiple feature data to obtain a point cloud transformation matrix, which can specifically include steps S1041 to S1042.

[0062] S1041. Perform feature matching on the obtained multiple feature data using the FLANN matching method to obtain a matching result.

[0063] In this embodiment, a random KD tree algorithm can be first used to perform a preset number of checks on the obtained multiple feature data to obtain an initial matching point set. Considering the requirements of registration rate and computational cost, the number of checks can be slightly reduced. First, five random KD tree indexing algorithms are used, and 50 checks are performed on each tree. The more checks, the higher the accuracy, but the higher the computational cost. Please refer to Figure 4, which is a schematic diagram of a random KD tree. As shown in Figure 4, multiple random KD trees are established, starting from the N with the highest variance. d Several dimensions are randomly selected from the dimension to be used for partitioning. When searching the random KD forest, all random KD trees will share a priority queue.

[0064] Then, the suboptimal matching point sets in the initial matching point set are eliminated, and the optimal matching point set is retained to obtain the matching result. In an exemplary embodiment, the Lowe's ratio test with a multiplier of 0.7 can be applied to eliminate the suboptimal matching point sets and retain only the optimal matching point set.

[0065] S1042. Use the RANSAC algorithm to optimize the matching results and obtain the point cloud transformation matrix.

[0066] In this embodiment, the RANSAC algorithm is used to optimize the matching results and obtain the homography point cloud transformation matrix. In one exemplary embodiment, the two-dimensional coordinates of the well-matched key points obtained in step S1041 are placed into two lists of floating-point coordinate pairs. One list contains the coordinates of the key points in the query image, and the other list contains the coordinates of the matched key points in the scene. RANSAC is used to eliminate incorrect matches. The point cloud transformation matrix is ​​generated based on the results of SIFT feature extraction on the bird's-eye view image and screening for correct matches.

[0067] In an optional implementation, the RANSAC algorithm is used to optimize the matching results to obtain a point cloud transformation matrix, which may specifically include: placing the two-dimensional coordinates of the matched key points in the matching results into two lists, one list being used to store the key point coordinates in the query image, and the other list being used to store the key point coordinates in the matching image; using the RANSAC algorithm to eliminate erroneous matches in the list; and obtaining the point cloud transformation matrix based on the matching results from which the erroneous matches have been eliminated.

[0068] The point cloud coarse registration method provided in this embodiment builds on the previous embodiment by further employing the FLANN matching method to perform feature matching on the obtained multiple feature data to obtain matching results. The matching results are then optimized using the RANSAC algorithm to generate a point cloud transformation matrix. An accurate point cloud transformation matrix helps improve the accuracy of point cloud coarse registration.

[0069] Based on any of the above embodiments, the following further details how to perform coarse registration of multiple point cloud data of a target object based on a point cloud transformation matrix. Based on the above embodiments, the point cloud coarse registration method provided in this embodiment can achieve coarse registration of multiple point cloud data using a perspective transformation algorithm based on a point cloud transformation matrix.

[0070] The generated point cloud transformation matrix is ​​used to transform the source point cloud into a target matrix to complete the registration. Point clouds collected by radar at different locations on the same horizontal plane are coarsely registered. The height error of each point cloud comes only from the difference in the optical center position of each radar, so only the X and Y dimensions of the point cloud are operated on. In this embodiment, perspective transformation is used to achieve coarse registration. When multiple radars scan an object, each radar's point cloud contains information about a portion of the object. Using this series of transformations, a bird's-eye view image from the same perspective can be obtained, facilitating registration and stitching.

[0071] Let the coordinates of the original image be (u,v), the corresponding homogeneous coordinates be (U,V,W), and the coordinates of the transformed image be (x,y). Then we have:

[0072] At the same time, the conversion relationship between two-dimensional coordinates and three-dimensional coordinates:

[0073] Arranging the above matrix equations and corresponding relationships to solve the two-dimensional coordinates, the results are as follows:

[0074] The above only represents the correspondence between a set of feature points. After feature matching, there are usually n sets of corresponding points: (u1, v1) corresponds to (x1, y1), (u2, v2) corresponds to (x2, y2), ..., (u n ,v n ) corresponds to (x n ,y n ), then we can get the system of equations in matrix form:

[0075] Let AT=B, then the least squares solution is: T=(A T A) -1 A T B, and when n=4, the solution obtained is an exact solution, and if n>4, the solution obtained is a least squares solution.

[0076] The T obtained at this time is the transformation matrix of the perspective transformation:

[0077] Following the above process, the point cloud is operated, and finally the Z-axis information is completed to complete the point cloud coarse registration. Please refer to Figure 5 for the effect, which is a schematic diagram of the point cloud coarse registration result provided by one embodiment of the present disclosure. As shown in Figure 5, the result of coarse registration of two point clouds is achieved.

[0078] In summary, the point cloud coarse registration method provided by the present disclosure is not only highly efficient in operation, but also has reduced computational costs. By using the bird's-eye view technology, the three-dimensional registration problem is reduced to two dimensions, and the time complexity of operations such as feature extraction and feature matching on the two-dimensional plane is reduced compared to three-dimensional space, thereby fundamentally improving the overall efficiency. Taking the example used in the present disclosure as an example, the point cloud data volume of the source point cloud and the target point cloud is basically around 130,000, and the registration time is around 0.5 seconds, and the algorithm runs at a considerable speed. The operation of converting to a bird's-eye view is regarded as a way of downsampling the point cloud, which reduces the spatial complexity of the calculation. At the same time, the time complexity of feature search and matching of two-dimensional images is reduced compared to three-dimensional images, which reduces the demand for hardware processors for the method disclosed in the present disclosure, and can also meet the needs when using a CPU, which can reduce costs to a certain extent.

[0079] FIG6 is a schematic diagram of the structure of a point cloud coarse registration device according to an embodiment of the present disclosure. As shown in FIG6 , the point cloud coarse registration device 60 provided in this embodiment may include: an acquisition module 601 , a projection module 602 , a detection module 603 , a matching module 604 , and a registration module 605 .

[0080] The acquisition module 601 is configured to use multiple acquisition devices to simultaneously acquire point clouds of the target object on the same horizontal plane, thereby obtaining multiple point cloud data of the target object, and the point cloud data obtained by adjacent acquisition devices partially overlap; the projection module 602 is configured to perform discretized projection on the multiple point cloud data respectively, thereby obtaining a bird's-eye view corresponding to each point cloud data; the detection module 603 is configured to perform two-dimensional image scale-invariant local feature detection on the multiple bird's-eye views obtained respectively, thereby obtaining feature data corresponding to each bird's-eye view; the matching module 604 is configured to perform feature matching on the multiple feature data obtained, thereby obtaining a point cloud transformation matrix; the registration module 605 is configured to perform coarse registration on the multiple point cloud data of the target object according to the point cloud transformation matrix.

[0081] The device of this embodiment can be used to execute the technical solution of the method embodiment shown in Figure 1. Its implementation principle and technical effects are similar and will not be repeated here.

[0082] In an optional embodiment, the projection module 602 is used to perform discretized projection on multiple point cloud data respectively to obtain a bird's-eye view corresponding to each point cloud data, which may specifically include: setting a region of interest in each point cloud data and determining the actual scale corresponding to each pixel in the bird's-eye view; determining the pixel coordinates of each point cloud in the region of interest in the bird's-eye view based on the correspondence between the pixel and the actual scale; determining the pixel value of each pixel in the bird's-eye view based on the maximum value of multiple point cloud height values ​​corresponding to each bird's-eye view pixel and the height of the detection range, and obtaining a bird's-eye view corresponding to each point cloud data.

[0083] In an optional embodiment, the detection module 603 is used to perform two-dimensional image scale-invariant local feature detection on the multiple bird's-eye views obtained to obtain feature data corresponding to each bird's-eye view. Specifically, it may include: repeatedly downsampling each bird's-eye view using a Gaussian filter function; comparing each pixel with its adjacent points in the image domain and the scale space domain to obtain dual local extreme points; removing unstable and misdetected extreme points from the dual local extreme points to obtain stable extreme points; using gradient solution to determine the direction of the stable extreme points to obtain feature data corresponding to each bird's-eye view.

[0084] In an optional embodiment, the matching module 604 is used to perform feature matching on the obtained multiple feature data to obtain a point cloud transformation matrix, which may specifically include: using the FLANN matching method to perform feature matching on the obtained multiple feature data to obtain matching results; using the RANSAC algorithm to optimize the matching results to obtain a point cloud transformation matrix.

[0085] In an optional embodiment, the matching module 604 is used to perform feature matching on the obtained multiple feature data using the FLANN matching method to obtain a matching result, which may specifically include: using a random KD tree algorithm to perform a preset number of checks on the obtained multiple feature data to obtain an initial matching point set; eliminating the suboptimal matching point set in the initial matching point set, retaining the optimal matching point set, and obtaining a matching result.

[0086] In an optional embodiment, the matching module 604 is used to optimize the matching results using the RANSAC algorithm to obtain a point cloud transformation matrix, which may specifically include: placing the two-dimensional coordinates of the matched key points in the matching results into two lists, one list being used to store the key point coordinates in the query image, and the other list being used to store the key point coordinates in the matching image; using the RANSAC algorithm to eliminate erroneous matches in the list; and obtaining the point cloud transformation matrix based on the matching results from which the erroneous matches have been eliminated.

[0087] In an optional implementation, the registration module 605 is configured to perform coarse registration on multiple point cloud data of the target object according to the point cloud transformation matrix, which may specifically include: using a perspective transformation algorithm according to the point cloud transformation matrix to achieve coarse registration of multiple point cloud data.

[0088] An embodiment of the present disclosure further provides an electronic device, as shown in FIG7 . The present disclosure is described using FIG7 as an example only, and does not limit the present disclosure to this example. FIG7 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. As shown in FIG7 , the electronic device 70 provided by this embodiment may include: a memory 701, a processor 702, and a bus 703. The bus 703 is used to connect various components.

[0089] The memory 701 stores a computer program, which can implement the technical solution of any of the above method embodiments when executed by the processor 702.

[0090] The memory 701 and the processor 702 are electrically connected, directly or indirectly, to enable data transmission or interaction. For example, these elements may be electrically connected to each other via one or more communication buses or signal lines, such as bus 703. The memory 701 stores a computer program for implementing the point cloud coarse registration method, including at least one software function module that can be stored in the memory 701 in the form of software or firmware. The processor 702 executes various functional applications and data processing by running the software program and modules stored in the memory 701.

[0091] The memory 701 may be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory 701 is used to store programs, and the processor 702 executes the programs after receiving execution instructions. In an exemplary embodiment, the software programs and modules in the memory 701 may also include an operating system, which may include various software components and / or drivers for managing system tasks (e.g., memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components.

[0092] The processor 702 can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 702 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The various methods, steps and logic block diagrams disclosed in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. It can be understood that the structure of Figure 7 is only for illustration and can also include more or fewer components than shown in Figure 7, or have a configuration different from that shown in Figure 7. The components shown in Figure 7 can be implemented in hardware and / or software.

[0093] The embodiments of the present disclosure further provide a computer-readable storage medium on which a computer program is stored. The computer program is executed by a processor to implement the technical solution of any of the above method embodiments.

[0094] The point cloud coarse registration method, device and equipment provided by the embodiments of the present disclosure use multiple acquisition devices to simultaneously perform point cloud acquisition on the target object on the same horizontal plane, thereby obtaining multiple point cloud data of the target object, and the point cloud data obtained by adjacent acquisition devices partially overlap; the multiple point cloud data are discretized and projected respectively to obtain a bird's-eye view corresponding to each point cloud data; the obtained multiple bird's-eye views are respectively subjected to two-dimensional image scale-invariant local feature detection to obtain feature data corresponding to each bird's-eye view; feature matching is performed on the obtained multiple feature data to obtain a point cloud transformation matrix; and the multiple point cloud data of the target object are coarsely registered according to the point cloud transformation matrix. Through discretized projection, the three-dimensional point cloud is converted into a two-dimensional bird's-eye view, and feature extraction, feature matching and other operations are performed on the two-dimensional plane, reducing the registration problem from three dimensions to two dimensions. Compared with three-dimensional space, the time complexity and the spatial complexity of the calculation are reduced, thereby fundamentally improving the matching efficiency and shortening the matching time.

[0095] The various embodiments in the present disclosure are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0096] The scope of protection of the present disclosure is not limited to the above-described embodiments. Obviously, those skilled in the art may make various modifications and variations to the present disclosure without departing from the scope and spirit of the present disclosure. If such modifications and variations fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include such modifications and variations.

Claims

1. A point cloud coarse registration method, comprising: Using multiple acquisition devices to simultaneously perform point cloud acquisition on a target object on the same horizontal plane, to obtain multiple point cloud data of the target object, and the point cloud data obtained by adjacent acquisition devices partially overlap; Performing discretization projection on the plurality of point cloud data respectively to obtain a bird's-eye view corresponding to each point cloud data; Performing two-dimensional image scale-invariant local feature detection on the obtained multiple bird's-eye views respectively to obtain feature data corresponding to each bird's-eye view; Perform feature matching on the obtained multiple feature data to obtain the point cloud transformation matrix; The plurality of point cloud data of the target object are roughly aligned according to the point cloud transformation matrix.

2. The method according to claim 1, wherein: The step of performing discretization projection on the plurality of point cloud data to obtain a bird's-eye view corresponding to each point cloud data comprises: Set the region of interest in each point cloud data and determine the actual scale corresponding to each pixel in the bird's-eye view; Determine the pixel coordinates of each point cloud in the bird's-eye view in the region of interest according to the correspondence between the pixel and the actual scale; According to the maximum value of multiple point cloud height values ​​corresponding to each bird's-eye view pixel and the height of the detection range, the pixel value of each pixel in the bird's-eye view is determined to obtain the bird's-eye view corresponding to each point cloud data.

3. The method according to claim 1, wherein: The two-dimensional image scale-invariant local feature detection is performed on the obtained multiple bird's-eye views to obtain feature data corresponding to each bird's-eye view, including: The Gaussian filter function is used to repeatedly downsample each bird's-eye view image; Compare each pixel with its neighboring points in the image domain and scale space domain to obtain dual local extreme points; Removing unstable and misdetected extreme points from the dual local extreme points to obtain stable extreme points; The direction of the stable extreme point is determined by gradient solution, and the characteristic data corresponding to each bird's-eye view is obtained.

4. The method according to claim 1, wherein: The performing feature matching on the obtained multiple feature data to obtain the point cloud transformation matrix includes: The FLANN matching method is used to perform feature matching on the obtained multiple feature data to obtain a matching result; The matching result is optimized using the RANSAC algorithm to obtain a point cloud transformation matrix.

5. The method according to claim 4, wherein: The FLANN matching method is used to perform feature matching on the obtained multiple feature data to obtain matching results, including: A random KD tree algorithm is used to perform a preset number of checks on the obtained multiple feature data to obtain an initial matching point set; The suboptimal matching point set in the initial matching point set is eliminated, and the optimal matching point set is retained to obtain a matching result.

6. The method according to claim 4, wherein: The using of the RANSAC algorithm to optimize the matching result and obtain the point cloud transformation matrix includes: Putting the two-dimensional coordinates of the key points matched in the matching results into two lists, one of which is used to store the coordinates of the key points in the query image, and the other is used to store the coordinates of the key points in the matching image; Using the RANSAC algorithm to remove incorrect matches from the list; The point cloud transformation matrix is ​​obtained based on the matching results after eliminating the wrong matches.

7. The method according to any one of claims 1 to 6, wherein: The coarsely registering the plurality of point cloud data of the target object according to the point cloud transformation matrix comprises: A perspective transformation algorithm is used according to the point cloud transformation matrix to achieve rough alignment of multiple point cloud data.

8. A point cloud coarse registration device, comprising: A collection module is configured to use multiple collection devices to simultaneously collect point cloud data of the target object on the same horizontal plane, so as to obtain multiple point cloud data of the target object and the point cloud data obtained by adjacent collection devices partially overlap; A projection module is configured to perform discretization projection on the plurality of point cloud data respectively to obtain a bird's-eye view corresponding to each point cloud data; A detection module is configured to perform two-dimensional image scale-invariant local feature detection on the obtained multiple bird's-eye views respectively to obtain feature data corresponding to each bird's-eye view; A matching module is configured to perform feature matching on the obtained multiple feature data to obtain a point cloud transformation matrix; The registration module is configured to perform coarse registration on the multiple point cloud data of the target object according to the point cloud transformation matrix.

9. An electronic device, comprising: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the point cloud coarse registration method according to any one of claims 1 to 7.

10. A computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the point cloud coarse registration method according to any one of claims 1 to 7.

Images