Point cloud data registration method and apparatus, electronic device, and computer program product

Abstract

Embodiments of the present disclosure disclose a point cloud data registration method and device, electronic equipment and computer program product, the method comprising: obtaining at least two pieces of point cloud data to be registered; the at least two pieces of point cloud data are collected for the same collection area, and at least three calibration balls are fixedly placed in the collection area during the collection process; converting the point cloud data into image data; identifying the image area corresponding to the calibration ball from the image data; matching the calibration ball identified in the image data corresponding to the at least two pieces of point cloud data based on the relative position relationship of the at least three calibration balls in the image data; and registering the at least two pieces of point cloud data based on the matching result. This technical solution directly identifies the image, and then registers the point cloud data based on the information identified from the image, which can reduce the labor cost, improve the processing efficiency of the point cloud data, and also improve the accuracy and efficiency of the point cloud data registration.

Description

Technical Field

[0001] This disclosure relates to the field of geographic information technology, specifically to a point cloud data registration method, apparatus, electronic device, and computer program product. Background Technology

[0002] The rapid development of 3D laser scanning technology has made point cloud data processing increasingly important, and point cloud registration is the most fundamental step in this process, directly affecting subsequent feature extraction, 3D modeling, and other processes, thus holding significant importance. Typically, due to the wide area of ​​the field scanning area, the limited field of view of the scanning instrument, and object occlusion, it is difficult to obtain complete object surface data from a single viewpoint. The purpose of point cloud registration is to obtain complete scene data by transforming point cloud data acquired from multiple viewpoints in different coordinate systems into a single coordinate system. This process, due to the need to process massive amounts of point cloud data, not only places stringent demands on computer performance but also requires extensive manual inspection and point selection, making it inefficient and unsuitable for processing large-scale point cloud data.

[0003] Therefore, it is necessary to propose a point cloud data registration scheme that can reduce the difficulty of point cloud data processing, so as to quickly and automatically register point cloud data. Summary of the Invention

[0004] This disclosure provides a point cloud data registration method, apparatus, electronic device, and computer program product.

[0005] In a first aspect, this disclosure provides a point cloud data registration method, which includes:

[0006] Acquire at least two sets of point cloud data to be registered; the at least two sets of point cloud data were acquired from the same acquisition area, and at least three calibration balls were fixedly placed in the acquisition area during the acquisition process;

[0007] Convert the point cloud data into image data;

[0008] Identify the image region corresponding to the calibration ball from the image data;

[0009] Based on the relative positional relationship of the at least three calibration spheres in the image data, the calibration spheres identified in the image data corresponding to the at least two point cloud data are matched;

[0010] The at least two sets of point cloud data are registered based on the matching results.

[0011] Further, identifying the image region corresponding to the calibration ball from the image data includes:

[0012] The spherical candidate region corresponding to the calibration sphere is identified by detecting circular regions from the image data;

[0013] Image regions corresponding to the calibration sphere are obtained by filtering from the spherical candidate regions based on the attribute information of the calibration sphere.

[0014] Further, based on the attribute information of the calibration sphere, image regions corresponding to the calibration sphere are selected from the spherical candidate regions, including:

[0015] Based on the color of the calibration ball and / or the distance threshold of the calibration ball relative to the point cloud data acquisition point, spherical candidate regions that are not corresponding to the calibration ball will be eliminated;

[0016] Based on the radius of the calibration sphere, the image region corresponding to the calibration sphere is selected from the remaining spherical candidate regions.

[0017] Further, based on the relative positional relationship of the at least three calibration spheres in the image data, matching the calibration spheres identified in the image data corresponding to the at least two point cloud data sets includes:

[0018] Determine a triangle formed by the centers of any three calibration spheres in the image data;

[0019] Based on the geometric information of the triangle, target triangle pairs that match are selected from the image data corresponding to the at least two point cloud data sets;

[0020] Determine the one-to-one correspondence between the vertices of the target triangle pair;

[0021] Based on the one-to-one correspondence, the positional correspondence of the same calibration ball in the image data corresponding to the at least two sets of point cloud data is determined.

[0022] Further, based on the geometric information of the triangle, matching target triangle pairs are selected from the image data corresponding to the at least two point cloud data sets, including:

[0023] Calculate the average side length of the triangle, and calculate the side length error of the triangle based on the average side length;

[0024] Identify candidate triangle pairs with equal side length errors in the image data corresponding to the at least two point cloud data sets;

[0025] The target triangle pair is selected from the candidate triangle pairs based on the geometric information of the two triangles in the candidate triangle pair.

[0026] Further, registration is performed on the at least two point cloud datasets based on the matching results, including:

[0027] Based on the positional correspondence of the same calibration sphere, the coordinate transformation parameters between the coordinate systems of the at least two point cloud data sets are determined.

[0028] Secondly, this invention provides a map creation method, which uses the method described in the first aspect to register the collected point cloud data and creates a map based on the registered point cloud data.

[0029] Thirdly, embodiments of the present invention provide a point cloud data registration device, comprising:

[0030] The acquisition module is configured to acquire at least two sets of point cloud data to be registered; the at least two sets of point cloud data are acquired from the same acquisition area, and at least three calibration balls are fixedly placed in the acquisition area during the acquisition process;

[0031] The conversion module is configured to convert the point cloud data into image data;

[0032] The recognition module is configured to identify the image region corresponding to the calibration ball from the image data;

[0033] The matching module is configured to match the calibration spheres identified in the image data corresponding to the at least two point cloud data sets based on the relative positional relationship of the at least three calibration spheres in the image data;

[0034] The registration module is configured to register the at least two sets of point cloud data based on the matching results.

[0035] The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above function.

[0036] In one possible design, the above-described device includes a memory and a processor. The memory stores one or more computer instructions that support the device in performing the corresponding methods described above, and the processor is configured to execute the computer instructions stored in the memory. The device may also include a communication interface for communicating with other devices or communication networks.

[0037] Fourthly, embodiments of this disclosure provide an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described in any of the above aspects.

[0038] Fifthly, embodiments of this disclosure provide a computer-readable storage medium for storing computer instructions used by any of the above-described devices, which, when executed by a processor, are used to implement the methods described in any of the above aspects.

[0039] In a sixth aspect, embodiments of this disclosure provide a computer program product comprising computer instructions which, when executed by a processor, are used to implement the methods described in any of the preceding aspects.

[0040] The technical solutions provided in this disclosure may have the following beneficial effects:

[0041] This disclosure proposes a point cloud data registration method based on target spheres. The method first generates image data from point cloud data; then, it automatically detects target spheres on the image data; next, it matches the detected target spheres to obtain corresponding target spheres, i.e., the positional correspondence of the same target sphere in different image data; finally, it registers multiple point cloud data sets based on the corresponding target spheres. This method does not directly process massive amounts of point cloud data, but rather converts the point cloud data into image data and processes it from an image perspective, avoiding the processing of large amounts of point cloud data. Furthermore, unlike traditional point cloud data registration algorithms, this method does not require the use of normals, curvature, or other computational features to filter corresponding point pairs. For massive point cloud data, normal calculations consume significant resources, affecting data processing performance, and the registration results are also constrained by the quality of the normals. Therefore, this method directly identifies the image and then registers the point cloud data based on the information identified from the image, reducing manual costs, improving the processing efficiency of point cloud data, and also improving the accuracy and efficiency of point cloud data registration.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0043] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments, taken in conjunction with the accompanying drawings. In the drawings:

[0044] Figure 1 A flowchart is shown for a point cloud data registration method according to an embodiment of the present disclosure.

[0045] Figure 2 A schematic diagram illustrating the high-precision map production process according to an embodiment of the present disclosure is shown.

[0046] Figure 3 A structural block diagram of a point cloud data registration apparatus according to an embodiment of the present disclosure is shown.

[0047] Figure 4 This is a schematic diagram of the structure of an electronic device suitable for implementing a point cloud data registration method according to an embodiment of the present disclosure. Detailed Implementation

[0048] In the following, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement them. Furthermore, for clarity, portions unrelated to the description of the exemplary embodiments have been omitted from the drawings.

[0049] In this disclosure, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, figures, steps, behaviors, components, parts or combinations thereof disclosed in this specification, and do not preclude the possibility of the presence or addition of one or more other features, figures, steps, behaviors, components, parts or combinations thereof.

[0050] It should also be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0051] The details of the embodiments of this disclosure are described in detail below through specific examples.

[0052] Figure 1 A flowchart illustrating a point cloud data registration method according to an embodiment of the present disclosure is shown. Figure 1 As shown, the point cloud data registration method includes the following steps:

[0053] In step S101, at least two sets of point cloud data to be registered are acquired; the at least two sets of point cloud data are acquired for the same acquisition area, and at least three calibration balls are fixedly placed in the acquisition area during the acquisition process;

[0054] In step S102, the point cloud data is converted into image data;

[0055] In step S103, the image region corresponding to the calibration ball is identified from the image data;

[0056] In step S104, based on the relative positional relationship of the at least three calibration spheres in the image data, the calibration spheres identified in the image data corresponding to the at least two point cloud data are matched;

[0057] In step S105, the at least two sets of point cloud data are registered based on the matching results.

[0058] In this embodiment, multiple point cloud data sets from the same acquisition area collected at different times or from different perspectives can be registered first, and then the registered point cloud data sets can be stitched together to obtain relatively complete and multi-directional visual data of the acquisition area. This visual data can be applied to scenarios such as map building. However, the processing efficiency of existing point cloud data severely restricts the generation process of corresponding products such as high-precision maps. In existing technologies, this process usually requires a large amount of manual point selection and inspection work, which results in high labor costs and affects the production efficiency of high-precision maps, making it difficult to produce large-scale map data. To this end, this disclosure proposes a point cloud data registration method based on a calibration sphere.

[0059] In this embodiment of the disclosure, at least two point cloud data sets to be registered can be collected from the same acquisition area, and during the acquisition process, multiple calibration balls can be set in the acquisition area. The positions of the multiple calibration balls are fixed, that is, when acquiring any point cloud data set, the placement positions of the multiple calibration balls are fixed.

[0060] In some embodiments, at least three calibration balls may be placed in the acquisition area.

[0061] In some embodiments, each point cloud data set to be configured is processed separately and then converted into image data. In some embodiments, the image data can be an intensity image. In some embodiments, the method for converting point cloud data into image data can refer to existing technologies, and is not specifically limited here.

[0062] In some embodiments, any two point cloud data sets to be registered can be point cloud data collected at different times from the same viewpoint for the same acquisition area, or point cloud data collected at the same time from different viewpoints for the same acquisition area. When collecting point cloud data for the same acquisition area from different viewpoints, the point cloud data acquisition device, such as a laser source, can be set at different locations in the non-acquisition area.

[0063] After converting point cloud data into image data, the image region corresponding to the calibration sphere can be identified from the image data. In other words, the image region where the calibration sphere is located can be identified from the image.

[0064] After identifying all or part of the image regions containing the calibration spheres from the image data, the relative positional relationships between the calibration spheres in the image data can be determined. Since the different point cloud data corresponding to different image data all come from the same acquisition area, and the positions of the calibration spheres placed in that acquisition area remain fixed, the relative positional relationships between the calibration spheres in the same image data can be determined first. Then, based on these relative positional relationships, the positional correspondence between the image regions of the same calibration sphere in different image data can be matched. In other words, the image regions of the same calibration sphere in different image data can be determined.

[0065] Multiple point cloud datasets can be registered based on the positional correspondence of image regions in different image data using the same calibration sphere. In some embodiments, registering multiple point cloud datasets can be understood as transforming them to the same coordinate system. This can be achieved by calculating the coordinate transformation parameters between the respective coordinate systems of the multiple point cloud datasets.

[0066] This disclosure proposes a point cloud data registration method based on target spheres. The method first generates image data from point cloud data; then, it automatically detects target spheres on the image data; next, it matches the detected target spheres to obtain corresponding target spheres, i.e., the positional correspondence of the same target sphere in different image data; finally, it registers multiple point cloud data sets based on the corresponding target spheres. This method does not directly process massive amounts of point cloud data, but rather converts the point cloud data into image data and processes it from an image perspective, avoiding the processing of large amounts of point cloud data. Furthermore, unlike traditional point cloud data registration algorithms, this method does not require the use of normals, curvature, or other computational features to filter corresponding point pairs. For massive point cloud data, normal calculations consume significant resources, affecting data processing performance, and the registration results are also constrained by the quality of the normals. Therefore, this method directly identifies the image and then registers the point cloud data based on the information identified from the image, reducing manual costs, improving the processing efficiency of point cloud data, and also improving the accuracy and efficiency of point cloud data registration.

[0067] In an optional implementation of this embodiment, step S103, namely the step of identifying the image region corresponding to the calibration ball from the image data, further includes the following steps:

[0068] The spherical candidate region corresponding to the calibration sphere is identified by detecting circular regions from the image data;

[0069] Image regions corresponding to the calibration sphere are obtained by filtering from the spherical candidate regions based on the attribute information of the calibration sphere.

[0070] In this optional implementation, circles can be detected from image data, and the image region enclosed by the detected circles can be determined as spherical candidate regions corresponding to the calibration sphere. In some embodiments, circles can be detected from image data using the Hough transform. The Hough transform can detect the center and radius of the circle.

[0071] In some embodiments, a histogram of the image data can be obtained first, which can display the grayscale distribution of the image data. Then, the image is binarized to obtain a clear black and white effect, while also reducing the amount of data and highlighting the target of interest, such as a calibration ball. Next, an erosion and dilation process is performed on the image data with the clear black and white effect to eliminate small objects and smooth the boundaries of larger objects. Finally, a Hough transform is used to find circular regions. This process can be repeated multiple times to achieve better results.

[0072] Since many circles can be obtained when detecting circles in image data, the image regions containing these circles can be considered as spherical candidate regions for the image region corresponding to the calibration sphere. However, some of these spherical candidate regions may not actually correspond to the calibration sphere. Therefore, these non-corresponding image regions need to be removed, and finally, the image regions corresponding to the calibration sphere are selected.

[0073] During the screening process, spherical candidate regions that do not meet the attributes of the calibration sphere can be eliminated based on the attribute information of the calibration sphere, while spherical candidate regions that do meet the attributes of the calibration sphere can be retained as image regions of the calibration sphere. In some embodiments, the attribute information of the calibration sphere may include, but is not limited to, the placement position of the calibration sphere, the color of the calibration sphere, the distance information of the calibration sphere relative to the acquisition point (i.e., the laser source), and / or the size information of the calibration sphere.

[0074] In an optional implementation of this embodiment, the step of filtering the image region corresponding to the calibration sphere from the spherical candidate regions based on the attribute information of the calibration sphere further includes the following steps:

[0075] Based on the color of the calibration ball and / or the distance threshold of the calibration ball relative to the point cloud data acquisition point, spherical candidate regions that are not corresponding to the calibration ball will be eliminated;

[0076] Based on the radius of the calibration sphere, the image region corresponding to the calibration sphere is selected from the remaining spherical candidate regions.

[0077] In this optional implementation, the color and radius of the calibration sphere are readily available. Typically, the calibration sphere is white, and its radius is usually one or more fixed values ​​in the industry, making it relatively easy to determine. Therefore, embodiments of this disclosure can perform a first-step screening based on the spherical candidate regions detected from image data according to the color of the calibration sphere, retaining spherical candidate regions with matching colors and discarding those with mismatched colors.

[0078] Furthermore, a distance threshold between the center of the calibration sphere and the acquisition point (i.e., the laser source) can be preset based on experience. Spherical candidate regions that match the distance threshold are retained, while those that do not are discarded. For example, when acquiring point cloud data for a certain area, the calibration sphere is typically placed approximately 50 meters from the acquisition point. Therefore, a distance threshold range of approximately 50 meters can be set. If the center of a spherical candidate region detected from the image data is outside this distance threshold range, the candidate region can be discarded; otherwise, it can be retained.

[0079] Subsequently, based on the radius of the calibration sphere, spherical candidate regions that do not meet the radius characteristics of the calibration sphere can be eliminated. For example, if the calibration spheres used in the acquisition process are all of equal size, and if there are spherical candidate regions with multiple radii in the image data, those with radii that do not meet the requirements can be eliminated. In some embodiments, radius can be used for final elimination because most spherical candidate regions that are definitely not calibration spheres can be eliminated by color, distance, etc., and most of the remaining spherical candidate regions may correspond to the calibration sphere. If there are individual spherical candidate regions with radii that are not equal to other spherical candidate regions, these individual spherical candidate regions can be eliminated. In this implementation, only the color and coordinate information of the calibration sphere are needed to determine the center position of the calibration sphere from the image data, without requiring other additional information, which reduces the complexity of the algorithm.

[0080] In some embodiments, the radius of the calibration sphere can also be information about the known actual radius size, and spherical candidate regions that do not conform to the actual radius can be directly removed from the image data based on this actual radius.

[0081] In an optional implementation of this embodiment, step S104, which is the step of matching the calibration spheres identified in the image data corresponding to the at least two point cloud data sets based on the relative positional relationship of the at least three calibration spheres in the image data, further includes the following steps:

[0082] Determine a triangle formed by the centers of any three calibration spheres in the image data;

[0083] Based on the geometric information of the triangle, target triangle pairs that match are selected from the image data corresponding to the at least two point cloud data sets;

[0084] Determine the one-to-one correspondence between the vertices of the target triangle pair;

[0085] Based on the one-to-one correspondence, the positional correspondence of the same calibration ball in the image data corresponding to the at least two sets of point cloud data is determined.

[0086] In this optional implementation, as described above, three or more calibration spheres are typically placed in the acquisition area. The image region corresponding to the calibration sphere identified from the image is a circular region with a known center and radius. To register multiple point cloud data sets, the triangles formed by the centers of any three calibration spheres in the image regions corresponding to the multiple images of the point cloud data sets can be matched.

[0087] In other words, one can first determine any triangle formed by the center of the image region corresponding to the calibration sphere, and then, based on the geometric information of the triangle, filter out target triangle pairs formed by matching triangles between any two images. In some embodiments, the geometric information of the triangle may include, but is not limited to, the perimeter, area, and angle of the triangle.

[0088] To register any two sets of point cloud data, any three calibration spheres from the image data corresponding to a single point cloud data set can be arranged into a triangle, with the vertices of the triangle representing the coordinates of the centers of the calibration spheres. Triangles formed by the same three calibration spheres in two sets of image data are congruent triangles, sharing the same side lengths, interior angles, perimeter, and area. Therefore, in this embodiment, this feature can be used to match triangles formed by the same three calibration spheres in different image data sets. In other words, the target triangle pair for matching includes two triangles formed by the same three calibration spheres in any two sets of image data sets.

[0089] After matching and obtaining target triangle pairs consisting of the same three calibration spheres, a one-to-one correspondence between the vertices of these triangle pairs can be determined. Based on this correspondence, the positional correspondence between image regions corresponding to the same calibration sphere in any two image data can be determined, thus enabling the determination of the positional correspondence of the same calibration sphere in different images. This embodiment utilizes the topological invariance of calibration spheres with the same name in different coordinate systems to perform matching of calibration spheres with the same name between different images.

[0090] In an optional implementation of this embodiment, the step of filtering out matching target triangle pairs from the image data corresponding to the at least two point cloud data sets based on the geometric information of the triangles further includes the following steps:

[0091] Calculate the average side length of the triangle, and calculate the side length error of the triangle based on the average side length;

[0092] Identify candidate triangle pairs with equal side length errors in the image data corresponding to the at least two point cloud data sets;

[0093] The target triangle pair is selected from the candidate triangle pairs based on the geometric information of the two triangles in the candidate triangle pair.

[0094] In this optional implementation, two triangles in a target triangle pair formed by three identical calibration spheres from any two image data sets are congruent triangles. To improve efficiency, this embodiment first calculates the side length error of each triangle. The side length error can be calculated based on the difference between the average side length of each triangle and the individual side lengths.

[0095] The following describes one method for calculating side length error.

[0096] Assuming the three sides of a triangle are a, b, and c, the average side length avg of the triangle is calculated as follows:

[0097] avg = (a + b + c) / 3;

[0098] The side length error σ is calculated as follows:

[0099]

[0100] Considering the rounding errors in computer storage and various errors generated during the calculation process, a small threshold can be set to identify any two triangles in any two images whose side length errors differ within this threshold as candidate triangle pairs. Solving for the side length errors can eliminate triangles that cannot be the target triangle pair, improving computational efficiency. Furthermore, this method only requires determining whether the side length errors are equal (or whether the difference is within the threshold range), without involving additional thresholds, making it simple and universally applicable.

[0101] Next, the geometric information of the two triangles in the candidate triangle pair is used to filter out the target triangle pair. For example, the target triangle pair is filtered by comparing whether the perimeters of the two triangles in the target triangle pair are equal, whether the three corresponding sides are equal, whether the areas of the triangles are equal, and / or whether the corresponding interior angles of the triangles are equal.

[0102] In an optional implementation of this embodiment, step S105, namely the step of registering the at least two sets of point cloud data based on the matching result, further includes the following steps:

[0103] Based on the positional correspondence of the same calibration sphere, the coordinate transformation parameters between the coordinate systems of the at least two point cloud data sets are determined.

[0104] In this optional implementation, as described above, the positional correspondence between the same calibration spheres of any two image data sets has been determined. This positional correspondence can include the position information of the center of the same calibration sphere in any image data set. Based on this positional correspondence, the coordinate transformation parameters between any two image data sets can be calculated, and thus the coordinate transformation parameters between the point cloud data corresponding to any two image data sets can be determined.

[0105] In some embodiments, coordinate transformation parameters may include, but are not limited to, translation parameters, rotation parameters, and scaling parameters.

[0106] The following describes a method for calculating coordinate transformation parameters.

[0107] Given two sets of point pairs, each set containing the coordinates of the center of a common calibration sphere in one of two image datasets, corresponding point pairs in the two sets represent the coordinates of the centers of the common calibration spheres in the two image datasets. The coordinate system transformation parameters can be solved using the following formula:

[0108]

[0109] Where R is the rotation parameter, t is the translation parameter, and c is the scaling parameter. i and y i , where n represents the coordinates of the center of the same calibrated sphere in different images, n is the total number of point pairs, i.e., the coordinates of the center of the sphere, and e is the error.

[0110] In the solution process, the coordinates of the sphere centers in the two sets of point pairs can be substituted into the above formula to find the value that makes e 2 The coordinate system transformation parameters R, t, and c are defined in the case where (R, t, c) is minimized.

[0111] It should be noted that the above example of registering two image datasets illustrates the process of solving for coordinate transformation parameters. The same method can be used to solve for coordinate transformation parameters between two point cloud datasets. This is because, given the one-to-one correspondence between the centers of the calibration spheres in the image data, the correspondence between each point in the point cloud data and the image data is known. Therefore, based on the one-to-one correspondence between the centers of the spheres in the image data and the correspondence between each point in the image data and the point cloud data, the correspondence between the centers of the two identical calibration spheres in the two point cloud datasets can be obtained. Furthermore, based on the correspondence between the centers of the two identical calibration spheres in the two point cloud datasets and the aforementioned formula, the coordinate transformation parameters between the point cloud datasets can be calculated.

[0112] As can be seen from the above description, the embodiments of this disclosure start from the perspective of images and use image detection technology to detect calibration balls in images. Since it does not require processing a large amount of point cloud data, it can improve the running efficiency of the algorithm and thus simplify the difficulty of the problem.

[0113] In some embodiments, this disclosure reads point cloud data from a storage device using memory mapping, avoiding the inefficiency of point-by-point reading in traditional technologies.

[0114] Furthermore, this embodiment only requires knowledge of the 3D coordinates and color information of each point cloud data point, without any other additional information. Moreover, the matching process for the calibration spheres only needs the center coordinates of each sphere; any three center points form a triangle for matching. The matching principle is whether the side length errors are equal or within a threshold range, without involving additional thresholds, thus making it more universal.

[0115] According to one embodiment of this disclosure, a map production method is also proposed, which uses the above-mentioned point cloud data registration method to register the collected point cloud data and produces a map based on the registered point cloud data.

[0116] In this embodiment, when creating a map, multiple point cloud data sets from different perspectives can be collected for the same collection area. During the point cloud data collection process, at least three calibration spheres can be set within the area. After the collected point cloud data sets are processed using the point cloud data registration method described above, coordinate transformation parameters between each pair of point cloud data sets can be obtained. Based on these coordinate transformation parameters, the multiple point cloud data sets can be stitched together, and then map data for the area can be created based on the stitched point cloud data.

[0117] Figure 2 A schematic diagram illustrating a high-precision map production process according to an embodiment of this disclosure is shown. Figure 2As shown, when creating a high-precision map for a specific road segment, the area can be divided into smaller local acquisition zones. Each local acquisition zone uses multiple perspectives, such as three perspectives, to collect point cloud data. Multiple laser sources for point cloud data collection can be placed around each local acquisition zone. Multiple calibration spheres, such as three, can also be placed within the local acquisition zone, all within the field of view of the laser sources. The perspectives of the laser sources are designed to completely cover all locations within the local area. Each laser source can collect one or more sets of point cloud data. This point cloud data is uploaded to a server, which performs registration using the aforementioned point cloud data registration method. The registered point cloud data can then be stitched together to obtain a comprehensive point cloud dataset for the local area. Maps can be created based on this point cloud data. The created maps can be distributed to high-precision map users, such as autonomous vehicles; they can also be uploaded to navigation servers so that navigation servers can guide autonomous vehicles to navigate within the local area.

[0118] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein.

[0119] Figure 3 A structural block diagram of a point cloud data registration apparatus according to an embodiment of the present disclosure is shown. This apparatus can be implemented as part or all of an electronic device through software, hardware, or a combination of both. Figure 3 As shown, the point cloud data registration device includes:

[0120] The acquisition module 301 is configured to acquire at least two sets of point cloud data to be registered; the at least two sets of point cloud data are acquired for the same acquisition area, and at least three calibration balls are fixedly placed in the acquisition area during the acquisition process;

[0121] The conversion module 302 is configured to convert the point cloud data into image data;

[0122] The recognition module 303 is configured to identify the image region corresponding to the calibration ball from the image data;

[0123] Matching module 304 is configured to match the calibration spheres identified in the image data corresponding to the at least two point cloud data sets based on the relative positional relationship of the at least three calibration spheres in the image data;

[0124] The registration module 305 is configured to register the at least two sets of point cloud data based on the matching result.

[0125] In this embodiment, multiple point cloud data sets from the same acquisition area collected at different times or from different perspectives need to be registered. The registered point cloud data sets can then be stitched together. However, the processing efficiency of existing point cloud data technologies severely restricts the generation process of corresponding products such as high-precision maps. In existing technologies, this process typically requires extensive manual point selection and inspection, often incurring significant labor costs and severely impacting the production efficiency of high-precision maps, making large-scale map data production difficult. Therefore, this disclosure proposes a point cloud data registration device based on a calibration sphere.

[0126] In this embodiment of the disclosure, the at least two point cloud data sets to be registered can be collected from the same acquisition area, and during the acquisition process, multiple calibration spheres can be set in the acquisition area, with the positions of the multiple calibration spheres remaining fixed. That is, when acquiring any point cloud data set, the placement positions of the multiple calibration spheres are fixed.

[0127] In some embodiments, at least three calibration balls may be placed in the acquisition area.

[0128] In some embodiments, each piece of point cloud data to be configured is processed and then converted into image data. In some embodiments, the image data is intensity data. In some embodiments, the apparatus for converting point cloud data into image data can be found in existing technologies, and will not be described in detail here.

[0129] In some embodiments, any two point cloud data sets to be registered can be point cloud data collected at different times from the same viewpoint for the same acquisition area, or point cloud data collected at the same time from different viewpoints for the same acquisition area. When collecting point cloud data for the same acquisition area from different viewpoints, the point cloud data acquisition device, such as a laser source, can be set at different locations in the non-acquisition area.

[0130] After converting point cloud data into image data, the image region corresponding to the calibration sphere can be identified from the image data. In other words, the image region where the calibration sphere is located can be identified from the image.

[0131] After identifying all or part of the image regions containing the calibration spheres from the image data, the relative positional relationships between the calibration spheres in the image data can be determined. Since different point cloud data corresponding to different image data are all collected from the same acquisition area, and the positions of the calibration spheres placed in that acquisition area remain fixed, the relative positional relationships between the calibration spheres in the same image data can be determined first. Then, based on these relative positional relationships, the positional correspondence between the image regions of the same calibration sphere in different image data can be matched. In other words, the image regions of the same calibration sphere in different image data can be determined.

[0132] Multiple point cloud datasets can be registered using image regions in different image data sets based on the same calibration sphere. In some embodiments, registering multiple point cloud datasets can be understood as transforming them to the same coordinate system. This can be achieved by calculating the coordinate transformation parameters between the respective coordinate systems of the multiple point cloud datasets.

[0133] This disclosure proposes a point cloud data registration device based on a target sphere. The device first generates image data from the point cloud data; then, it automatically detects a target sphere on the image data; next, it matches the detected target spheres to obtain corresponding target spheres, i.e., the positional correspondence of the same target sphere in different image data; finally, it registers multiple point cloud datasets based on the corresponding target spheres. This device does not directly process massive amounts of point cloud data, but rather converts the point cloud data into image data and processes it from an image perspective, avoiding the processing of large amounts of point cloud data. Furthermore, unlike traditional point cloud data registration algorithms, this disclosure does not require the use of normals, curvature, or other computational features to filter corresponding point pairs. For massive point cloud data, normal calculations consume significant resources, affecting data processing performance, and the registration results are also constrained by the quality of the normals. Therefore, the present disclosure directly identifies the image and then registers the point cloud data based on the information identified from the image, which can reduce labor costs, improve the processing efficiency of point cloud data, and also improve the accuracy and efficiency of point cloud data registration.

[0134] In an optional implementation of this embodiment, the identification module includes:

[0135] The recognition submodule is configured to identify the spherical candidate region corresponding to the calibration sphere by detecting circular regions from the image data;

[0136] The first filtering submodule is configured to filter image regions corresponding to the calibration sphere from the spherical candidate regions based on the attribute information of the calibration sphere.

[0137] In this alternative implementation, circles can be detected from image data, and the image region enclosed by the detected circles can be determined as spherical candidate regions. In some embodiments, circles can be detected from image data using the Hough transform. The Hough transform can detect the center and radius of the circle.

[0138] In some embodiments, a histogram of the image data can be obtained first, which can display the grayscale distribution of the image data. Then, the image is binarized to obtain a clear black and white effect, while also reducing the amount of data and highlighting the target of interest, such as a calibration ball. Next, an erosion and dilation process is performed on the image data with the clear black and white effect to eliminate small objects and smooth the boundaries of larger objects. Finally, a Hough transform is used to find circular regions. This process can be repeated multiple times to achieve better results.

[0139] Since many circles can be obtained when detecting circles in image data, the image regions containing these circles can be considered as spherical candidate regions for the image region corresponding to the calibration sphere. However, some of these spherical candidate regions may not actually correspond to the calibration sphere. Therefore, these non-corresponding image regions need to be removed, and finally, the image regions corresponding to the calibration sphere are selected.

[0140] During the screening process, spherical candidate regions that do not meet the attributes of the calibration sphere can be eliminated based on the attribute information of the calibration sphere, while spherical candidate regions that do meet the attributes of the calibration sphere can be retained as image regions of the calibration sphere. In some embodiments, the attribute information of the calibration sphere may include, but is not limited to, the placement position of the calibration sphere, the color of the calibration sphere, the distance information of the calibration sphere relative to the acquisition point (i.e., the laser source), and / or the size information of the calibration sphere.

[0141] In an optional implementation of this embodiment, the first filtering submodule includes:

[0142] The elimination submodule is configured to eliminate spherical candidate regions that are not corresponding to the calibration ball based on the color of the calibration ball and / or a distance threshold of the calibration ball relative to the acquisition point of the point cloud data.

[0143] The second filtering submodule is configured to filter out image regions corresponding to the calibration sphere from the remaining spherical candidate regions based on the radius of the calibration sphere.

[0144] In this optional implementation, the color and radius of the calibration sphere are readily available. Typically, the calibration sphere is white, and its radius is usually one or more fixed values ​​in the industry, making it relatively easy to determine. Therefore, embodiments of this disclosure can perform a first-step screening based on the spherical candidate regions detected from image data according to the color of the calibration sphere, retaining spherical candidate regions with matching colors and discarding those with mismatched colors.

[0145] Furthermore, a distance threshold between the center of the calibration sphere and the acquisition point (i.e., the laser source) can be preset based on experience. Spherical candidate regions that match the distance threshold are retained, while those that do not are discarded. For example, when acquiring point cloud data for a certain area, the calibration sphere is typically placed approximately 50 meters from the acquisition point. Therefore, a distance threshold range of approximately 50 meters can be set. If the center of a spherical candidate region detected from the image data is outside this distance threshold range, the candidate region can be discarded; otherwise, it can be retained.

[0146] Subsequently, based on the radius of the calibration sphere, spherical candidate regions that do not meet the radius characteristics of the calibration sphere can be eliminated. For example, if the calibration spheres used in the acquisition process are all of equal size, and if there are spherical candidate regions with multiple radii in the image data, those with radii that do not meet the requirements can be eliminated. In some embodiments, radius can be used for final elimination because most spherical candidate regions that are not labels can be eliminated by color, distance, etc., and most of the remaining spherical candidate regions may correspond to the calibration sphere. If there are individual spherical candidate regions with radii that are not equal to other spherical candidate regions, these individual spherical candidate regions can be eliminated. In this implementation, only the color and coordinate information of the calibration sphere are needed to determine the center position of the calibration sphere from the image data, without requiring other additional information, which reduces the complexity of the algorithm.

[0147] In some embodiments, the radius of the calibration sphere can also be information about the known actual radius size, and spherical candidate regions that do not conform to the actual radius can be directly removed from the image data based on this actual radius.

[0148] In an optional implementation of this embodiment, the matching module includes:

[0149] The first determining submodule is configured to determine a triangle formed by the centers of any three calibration spheres in the image data as vertices;

[0150] The third filtering submodule is configured to filter out matching target triangle pairs in the image data corresponding to the at least two point cloud data based on the geometric information of the triangle;

[0151] The second determining submodule is configured to determine the one-to-one correspondence between the vertices of the target triangle pair;

[0152] The third determining submodule is configured to determine the positional correspondence of the same calibration ball in the image data corresponding to the at least two sets of point cloud data based on the one-to-one correspondence.

[0153] In this optional implementation, as described above, three or more calibration spheres are typically placed in the acquisition area. The image region corresponding to the calibration sphere identified from the image is a circular region with a known center and radius. To register multiple point cloud data sets, the triangles formed by the centers of any three calibration spheres in the image regions corresponding to the multiple images of the point cloud data sets can be matched.

[0154] In other words, one can first determine any triangle formed by the center of the image region corresponding to the calibration sphere, and then, based on the geometric information of the triangle, filter out target triangle pairs formed by matching triangles between any two images. In some embodiments, the geometric information of the triangle may include, but is not limited to, the perimeter, area, and angle of the triangle.

[0155] To register any two sets of point cloud data, any three calibration spheres from the image data corresponding to a single point cloud data set can be arranged into a triangle, with the vertices of the triangle representing the coordinates of the centers of the calibration spheres. Triangles formed by the same three calibration spheres in two sets of image data are congruent triangles, and their areas are also the same. Therefore, in this embodiment of the disclosure, triangles formed by the same three calibration spheres in different image data sets can be matched based on this feature. That is, the target triangle pair for matching includes two triangles formed by the same three calibration spheres in any two sets of image data sets.

[0156] After matching the target triangle pairs obtained from the three identical calibration spheres, a one-to-one correspondence between the vertices of the target triangle pair can be determined. Based on this one-to-one correspondence, the image region corresponding to the same calibration sphere in any two image data can be determined, thereby determining the positional correspondence of the same calibration sphere in different images. This embodiment utilizes the topological invariance of calibration spheres with the same name in different coordinate systems to perform matching of calibration spheres with the same name between different images.

[0157] In an optional implementation of this embodiment, the third filtering submodule includes:

[0158] The calculation submodule is configured to calculate the average side length of the triangle and calculate the side length error of the triangle based on the average side length;

[0159] The fourth determining submodule is configured to determine candidate triangle pairs with equal side length errors in the image data corresponding to the at least two sets of point cloud data;

[0160] The fourth filtering submodule is configured to filter the target triangle pair from the candidate triangle pair based on the geometric information of the two triangles in the candidate triangle pair.

[0161] In this optional implementation, two triangles in a target triangle pair formed by three identical calibration spheres from any two image data sets are congruent triangles. To improve efficiency, this embodiment first calculates the side length error of each triangle. The side length error can be calculated based on the average side length of each triangle.

[0162] The following describes one method for calculating side length error.

[0163] Assuming the three sides of a triangle are a, b, and c, the average side length avg of the triangle is calculated as follows:

[0164] avg = (a + b + c) / 3;

[0165] The side length error σ is calculated as follows:

[0166]

[0167] Considering the rounding errors in computer storage and various errors generated during the calculation process, a small threshold can be set to identify any two triangles in any two images whose side length errors differ within this threshold as candidate triangle pairs. Solving for the side length errors can eliminate triangles that cannot be the target triangle pair, improving computational efficiency. Furthermore, this method only requires determining whether the side length errors are equal (or whether the difference is within the threshold range), without involving an additional threshold, making it a relatively universal approach.

[0168] Next, the geometric information of the two triangles in the candidate triangle pair is used to filter out the target triangle pair. For example, the target triangle pair is filtered by comparing whether the perimeters of the two triangles in the target triangle pair are equal, whether the three corresponding sides are equal, whether the areas of the triangles are equal, and / or whether the corresponding interior angles of the triangles are equal.

[0169] In an optional implementation of this embodiment, the registration module includes:

[0170] The fifth determining submodule is configured to determine the coordinate transformation parameters between the coordinate systems of the at least two sets of point cloud data based on the positional correspondence of the same calibration sphere.

[0171] In this optional implementation, as described above, the positional correspondence between the same calibration spheres of any two image data sets has been determined. This positional correspondence can include the position information of the center of the same calibration sphere in any image data set. Based on this positional correspondence, the coordinate transformation parameters between any two image data sets can be calculated, and thus the coordinate transformation parameters between the point cloud data corresponding to any two image data sets can be determined.

[0172] In some embodiments, coordinate transformation parameters may include, but are not limited to, translation parameters, rotation parameters, and scaling parameters.

[0173] The following describes a method for calculating coordinate transformation parameters.

[0174] Given two sets of point pairs, each set containing the coordinates of the center of a common calibration sphere in one of two image datasets, corresponding point pairs in the two sets represent the coordinates of the centers of the common calibration spheres in the two image datasets. The coordinate system transformation parameters can be solved using the following formula:

[0175]

[0176] Where R is the rotation parameter, t is the translation parameter, and c is the scaling parameter. i and y i , where n represents the coordinates of the center of the same calibrated sphere in different images, n is the total number of point pairs, i.e., the coordinates of the center of the sphere, and e is the error.

[0177] In the solution process, the coordinates of the sphere centers in the two sets of point pairs can be substituted into the above formula to find the value that makes e 2 The coordinate system transformation parameters R, t, and c are defined in the case where (R, t, c) is minimized.

[0178] It should be noted that the above example of registering two image datasets illustrates the process of solving for coordinate transformation parameters. The same method can be used to solve for coordinate transformation parameters between two point cloud datasets. This is because, given the one-to-one correspondence between the centers of the calibration spheres in the image data, the correspondence between each point in the point cloud data and the image data is known. Therefore, based on the one-to-one correspondence between the centers of the spheres in the image data and the correspondence between each point in the image data and the point cloud data, the correspondence between the centers of the two identical calibration spheres in the two point cloud datasets can be obtained. Furthermore, based on the correspondence between the centers of the two identical calibration spheres in the two point cloud datasets and the aforementioned formula, the coordinate transformation parameters between the point cloud datasets can be calculated.

[0179] As can be seen from the above description, the embodiments of this disclosure start from the perspective of images and use image detection technology to detect calibration balls in images. Since it does not require processing a large amount of point cloud data, it can improve the running efficiency of the algorithm and thus simplify the difficulty of the problem.

[0180] In some embodiments, this disclosure reads point cloud data from a storage device using memory mapping, avoiding the inefficiency of point-by-point reading in traditional technologies.

[0181] Furthermore, this embodiment only requires knowledge of the 3D coordinates and color information of each point cloud data point, without any other additional information. Moreover, the matching process for the calibration spheres only needs the coordinates of the center of each sphere; any three center points form a triangle for matching. The matching principle is whether the side length errors are equal or within a threshold range, without involving additional thresholds, thus making it more universal.

[0182] Figure 4 This is a schematic diagram of the structure of an electronic device suitable for implementing a point cloud data registration method according to an embodiment of the present disclosure.

[0183] like Figure 4 As shown, the electronic device 400 includes a processing unit 401, which can be implemented as a CPU, GPU, FPGA, NPU, or other processing unit. The processing unit 401 can execute various processes according to any of the methods described above in this disclosure, based on a program stored in the read-only memory (ROM) 402 or a program loaded from the storage portion 408 into the random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the electronic device 400. The processing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0184] The following components are connected to I / O interface 405: an input section 406 including a keyboard, mouse, etc.; an output section 407 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to I / O interface 405 as needed. A removable medium 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0185] In particular, according to embodiments of this disclosure, any of the methods described above in the embodiments of this disclosure can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing any of the methods in the embodiments of this disclosure. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411.

[0186] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0187] The units or modules described in the embodiments of this disclosure can be implemented in software or hardware. The described units or modules can also be located in a processor, and the names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0188] In another aspect, this disclosure also provides a computer-readable storage medium, which may be a computer-readable storage medium included in the apparatus described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into a device. The computer-readable storage medium stores one or more programs that are used by one or more processors to perform the methods described in this disclosure.

[0189] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A point cloud data registration method, wherein, include: Obtain at least two sets of point cloud data to be registered; The at least two point cloud data sets were collected from the same collection area, and at least three calibration balls were fixedly placed in the collection area during the collection process; Convert the point cloud data into image data; The spherical candidate region corresponding to the calibration sphere is identified by detecting circular regions from the image data; Based on the attribute information of the calibration sphere, an image region corresponding to the calibration sphere is obtained from the spherical candidate region; Based on the relative positional relationship of the at least three calibration spheres in the image data, the calibration spheres identified in the image data corresponding to the at least two point cloud data are matched; The at least two sets of point cloud data are registered based on the matching results.

2. The method according to claim 1, wherein, Based on the attribute information of the calibration sphere, image regions corresponding to the calibration sphere are obtained from the spherical candidate regions, including: Based on the color of the calibration ball and / or the distance threshold of the calibration ball relative to the point cloud data acquisition point, spherical candidate regions that are not corresponding to the calibration ball will be eliminated; Based on the radius of the calibration sphere, image regions corresponding to the calibration sphere are selected from the remaining spherical candidate regions.

3. The method according to claim 1 or 2, wherein, Based on the relative positional relationship of the at least three calibration spheres in the image data, matching the calibration spheres identified in the image data corresponding to the at least two point cloud data sets includes: Determine a triangle formed by the centers of any three calibration spheres in the image data; Based on the geometric information of the triangle, target triangle pairs that match are selected from the image data corresponding to the at least two point cloud data sets; Determine the one-to-one correspondence between the vertices of the target triangle pair; Based on the one-to-one correspondence, the positional correspondence of the same calibration ball in the image data corresponding to the at least two sets of point cloud data is determined.

4. The method according to claim 3, wherein, Based on the geometric information of the triangle, matching target triangle pairs are selected from the image data corresponding to the at least two point cloud data sets, including: Calculate the average side length of the triangle, and calculate the side length error of the triangle based on the average side length; Identify candidate triangle pairs with equal side length errors in the image data corresponding to the at least two point cloud data sets; The target triangle pair is selected from the candidate triangle pairs based on the geometric information of the two triangles in the candidate triangle pair.

5. The method according to any one of claims 1-2 and 4, wherein, Based on the matching results, the at least two sets of point cloud data are registered, including: The coordinate transformation parameters between the coordinate systems of the at least two point cloud data sets are determined based on the positional correspondence of the same calibration sphere.

6. A map production method, wherein the method uses the method described in any one of claims 1-5 to register collected point cloud data, and produces a map based on the registered point cloud data.

7. A point cloud data registration device, wherein, include: The acquisition module is configured to acquire at least two sets of point cloud data to be registered; The at least two point cloud data sets were collected from the same collection area, and at least three calibration balls were fixedly placed in the collection area during the collection process; The conversion module is configured to convert the point cloud data into image data; The recognition module is configured to identify the spherical candidate region corresponding to the calibration sphere by detecting circular regions from the image data; Based on the attribute information of the calibration sphere, an image region corresponding to the calibration sphere is obtained from the spherical candidate region; The matching module is configured to match the calibration spheres identified in the image data corresponding to the at least two point cloud data sets based on the relative positional relationship of the at least three calibration spheres in the image data; The registration module is configured to register the at least two sets of point cloud data based on the matching results.

8. An electronic device, wherein, The method includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method of any one of claims 1-6.

9. A computer program product comprising computer instructions, wherein, When executed by a processor, the computer instructions implement the method described in any one of claims 1-6.

Images