Three-dimensional scene modeling method and system based on laser point cloud scanning and joint descriptor

Abstract

The invention discloses a three-dimensional scene modeling method and system based on laser point cloud scanning and a joint descriptor, and the method comprises the steps: carrying out the map construction through employing a laser radar, and obtaining a constructed global map; a sub-map is obtained from the constructed global map, key points are extracted from the sub-map, the extracted key points are coded into a joint descriptor, and the joint descriptor comprises a triangular descriptor and three binary descriptors corresponding to three vertexes forming a triangle; taking the sub-map obtained by query as a query sub-map, obtaining candidate joint descriptors of all candidate sub-maps according to the query sub-map, performing binary similarity check and triangle geometric verification on the candidate joint descriptors to judge whether the candidate sub-map is an effective loopback detection result, and if the candidate sub-map is an effective loopback detection result, judging whether the candidate sub-map is an effective loopback detection result. If yes, loopback detection is executed, and the global map is updated. According to the method, mapping loss is not liable to occur, and the large-scene mapping stability of large-batch laser radar point cloud data can be improved.

Description

Technical Field

[0001] This invention belongs to the field of 3D modeling technology. It relates to a 3D scene modeling method and system based on laser point cloud scanning and joint descriptors. Background Technology

[0002] Loop closure detection is one of the core technologies in 3D scene modeling. Descriptor-based loop closure detection transforms complex raw point cloud data into more discriminative and robust feature representations (descriptors), and then identifies whether the same location has been revisited by comparing these descriptors. The system estimates the system state and constructs a 3D scene map in real time using sensors (such as LiDAR, cameras, and IMUs). However, sensor noise (such as odometry drift and visual feature matching errors) leads to accumulated mapping errors. After long-term operation, the mapping and localization results gradually deviate from the true values. This invention aims to correct this error. Summary of the Invention

[0003] The purpose of this invention is to provide a 3D scene modeling method and system based on laser point cloud scanning and joint descriptors, which is less prone to mapping loss and can improve the stability of large-scale scene mapping based on large batches of LiDAR point cloud data.

[0004] The technical solution to achieve the purpose of this invention is as follows: A 3D scene modeling method based on laser point cloud scanning and joint descriptors includes the following steps: A global map is constructed using LiDAR. Obtain a sub-map from the constructed global map, extract key points from the sub-map, the key points being the pixels with the highest intensity in a local area, and encode the extracted key points into a binary and triangle joint descriptor, the joint descriptor including a triangle descriptor and three binary descriptors corresponding to the three vertices forming the triangle; The submap obtained from the query is used as the query submap. Based on the query submap, candidate joint descriptors of all candidate submaps are obtained. Binary similarity checks and geometric verification of triangles are performed on the candidate joint descriptors to determine whether the candidate submap is a valid loop closure detection result. If it is a valid loop closure detection result, loop closure detection is performed and the global map is updated.

[0005] In the preferred technical solution, the following is included before using LiDAR for map building: The LiDAR scanning data is reconstructed based on the timestamps of the camera images, so that the frequency of the camera image data is consistent with the frequency of the LiDAR data.

[0006] In the preferred technical solution, extracting key points from the sub-map includes: Perform planar detection on the submap using region growing to obtain a list of planes; After obtaining the list of planes, a reference plane is generated; After obtaining the reference plane, Point cloud is projected onto each reference plane to create A highly encoded image; Keypoints are determined by identifying the pixels with the highest intensity within local regions of a highly coded image.

[0007] In the preferred technical solution, generating the reference plane includes: Select an initial planar voxel And check adjacent planar voxels Obtain the initial planar voxels and adjacent planar voxels center point and and normal vector and If the following conditions are met:

[0008] in, and There are two preset thresholds, and max() is used to find the maximum value. It is the inverse cosine function; Then adjacent planar voxels With the initial planar voxel Merge, after merging, the new plane center point and normal vector Calculated using the following formula:

[0009] in, and Representing adjacent planar voxels respectively With the initial planar voxel The number of points, and Representing adjacent planar voxels respectively With the initial planar voxel covariance, It's a new plane. covariance; Through the Perform eigenvalue decomposition to calculate the normal vector of the merging plane. ; The merging process is carried out in the form of region growth until there are no planes in the adjacent voxels.

[0010] In the preferred technical solution, after encoding the extracted key points into a joint descriptor, the following is also included: The extracted feature points are managed using a hash table. A hash function is constructed to calculate the hash key from the quantized triangle side lengths. A fixed resolution is used to map continuous side lengths to discrete values, and the quantized side lengths are calculated, as shown in the following formula:

[0011] in, It calculates the hash value of list L. It is the execution layer function that calculates the hash value. These are the original side lengths of the three sides of the triangle. It is the quantized side length. It is the scaling factor. is the number of hash buckets, and Mod() is the modulo function.

[0012] In the preferred technical solution, the candidate joint descriptors for obtaining all candidate submaps based on the query submap include: Extract the combined binary and triangular descriptor from the query submap and calculate the hash key:

[0013] in, Represents two joint descriptors and Overall similarity Calculate the similarity between two binary feature blocks. and Representing two joint descriptors respectively and The k-th binary feature block This represents the local similarity of the k-th binary feature block. The & represents the number of 1s in the binary feature block; & is the bitwise AND operation. Query the joint descriptor database, find the corresponding container in the hash table, and obtain the candidate joint descriptors for all candidate submaps.

[0014] In the preferred technical solution, geometric verification is performed by calculating the planar overlap between the query submap and the candidate submap, including: To retrieve the list of planes of the query submap The planar list of candidate submaps is as follows ; Using the center point of the candidate submap plane Build a Tree, for each query submap plane center point , For a set of planes in a single query submap, a coarse transformation is first used to obtain the rigid body transformation matrix. Conversion Then Search for the nearest point in the tree The overlap between two planes is determined by checking the difference between their planar distance and normal vector. The constraints for the planar distance and normal vector are as follows:

[0015] in, and These represent the unit normal vectors in the query submap plane and the candidate submap plane, respectively. Indicates querying the coordinate system of the sub-map To candidate submap coordinate system transformation, and These represent querying points from the submap plane list and the candidate submap plane list, respectively. and These represent the distance and angle thresholds, respectively. Indicates querying the coordinate system of the sub-map To candidate submap coordinate system The rotating part; If a pair of planes satisfies the constraints of planar distance and normal vector, then the pair of planes are considered to overlap.

[0016] In the preferred technical solution, determining whether a candidate submap is a valid loop closure detection result includes: After examining all planes of the query submap, calculate the query submap. and candidate submap Percentage of planar overlap between As shown in the following formula:

[0017] For each candidate sub-map, determine whether the calculated percentage of planar overlap exceeds a threshold. If the number of loop closures exceeds a certain threshold, then the loop closure detection for that candidate submap is valid.

[0018] This invention also discloses a 3D scene modeling system based on laser point cloud scanning and joint descriptors, comprising: The global map building module uses LiDAR to build a global map. The joint descriptor encoding module obtains a sub-map from the constructed global map, extracts key points from the sub-map, the key points being the pixels with the highest intensity in a local area, and encodes the extracted key points into a joint descriptor, the joint descriptor including a triangle descriptor and three binary descriptors corresponding to the three vertices forming the triangle; The loop closure detection module uses the sub-map obtained from the query as the query sub-map, obtains candidate joint descriptors for all candidate sub-maps based on the query sub-map, and performs binary similarity checks and geometric verification of triangles on the candidate joint descriptors to determine whether the candidate sub-map is a valid loop closure detection result. If it is a valid loop closure detection result, loop closure detection is performed and the global map is updated.

[0019] The present invention also discloses a computer storage medium storing a computer program thereon, wherein when the computer executes the computer program, it implements the three-dimensional scene modeling method based on laser point cloud scanning and joint descriptors described above.

[0020] The present invention also discloses an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program stored in the memory. When the computer program is executed, it implements the three-dimensional scene modeling method based on laser point cloud scanning and joint descriptors described in any of the above claims.

[0021] Compared with the prior art, the significant advantages of this invention are: (1) The accuracy of mapping is better; (2) Mapping loss is less likely to occur when modeling large scenes; (3) The stability of large-scale mapping of large batches of point cloud data is improved. This invention has good practical application value and engineering significance in improving the accuracy of large-scale laser point cloud mapping. Attached Figure Description

[0022] Figure 1 Here is a flowchart of a 3D scene modeling method based on laser point cloud scanning and joint descriptors; Figure 2 A schematic diagram illustrating the reconstruction of LiDAR and camera data; Figure 3 A rendering of a traditional 3D scene model based on laser point clouds; Figure 4 This is a modeling effect diagram of the method of the present invention; Figure 5 This is a batch 3D modeling effect diagram of the method of the present invention. Detailed Implementation

[0023] The principle of this invention is to construct a binary and triangle joint descriptor, perform binary similarity checks and triangle geometric verification to determine whether the candidate sub-map is a valid loop closure detection result. This can perform effective loop closure detection, reduce mapping loss when modeling large scenes, and improve the stability of large-scale mapping of large batches of LiDAR point cloud data.

[0024] Example: like Figure 1 As shown, a 3D scene modeling method based on laser point cloud scanning and joint descriptors includes the following steps: A global map is constructed using LiDAR. Obtain a sub-map from the constructed global map, extract key points from the sub-map, the key points being the pixels with the highest intensity in a local area, and encode the extracted key points into a binary and triangle joint descriptor, the joint descriptor including a triangle descriptor and three binary descriptors corresponding to the three vertices forming the triangle; The submap obtained from the query is used as the query submap. Based on the query submap, candidate joint descriptors of all candidate submaps are obtained. Binary similarity checks and geometric verification of triangles are performed on the candidate joint descriptors to determine whether the candidate submap is a valid loop closure detection result. If it is a valid loop closure detection result, loop closure detection is performed and the global map is updated.

[0025] In a preferred embodiment, the method further includes the following steps prior to map building using LiDAR: The LiDAR scanning data is reconstructed based on the timestamps of the camera images, so that the frequency of the camera image data is consistent with the frequency of the LiDAR data.

[0026] In a preferred embodiment, extracting key points from a submap includes: Perform planar detection on the submap using region growing to obtain a list of planes; After obtaining the list of planes, a reference plane is generated; After obtaining the reference plane, Point cloud is projected onto each reference plane to create A highly encoded image; Keypoints are determined by identifying the pixels with the highest intensity within local regions of a highly coded image.

[0027] It should be noted that other methods can also be used to extract feature points, and this is not a limitation here.

[0028] In a preferred embodiment, generating the reference plane includes: Select an initial planar voxel And check adjacent planar voxels Obtain the initial planar voxels and adjacent planar voxels center point and and normal vector and If the following conditions are met:

[0029] in, and There are two preset thresholds, and max() is used to find the maximum value. It is the inverse cosine function; Then adjacent planar voxels With the initial planar voxel Merge, after merging, the new plane center point and normal vector Calculated using the following formula:

[0030] in, and Representing adjacent planar voxels respectively With the initial planar voxel The number of points, and Representing adjacent planar voxels respectively With the initial planar voxel covariance, It's a new plane. covariance; Through the Perform eigenvalue decomposition to calculate the normal vector of the merging plane. ; The merging process is carried out in the form of region growth until there are no planes in the adjacent voxels.

[0031] In a preferred embodiment, after encoding the extracted key points into a binary and triangle joint descriptor, the method further includes: The extracted feature points are managed using a hash table. A hash function is constructed to calculate the hash key from the quantized triangle side lengths. A fixed resolution is used to map continuous side lengths to discrete values, and the quantized side lengths are calculated, as shown in the following formula:

[0032] in, It calculates the hash value of list L. It is the execution layer function that calculates the hash value. These are the original side lengths of the three sides of the triangle. It is the quantized side length. It is the scaling factor. is the number of hash buckets, and Mod() is the modulo function.

[0033] In a preferred embodiment, obtaining candidate joint descriptors for all candidate submaps based on the query submap includes: Extract the combined binary and triangular descriptor from the query submap and calculate the hash key:

[0034] in, Represents two joint descriptors and Overall similarity Calculate the similarity between two binary feature blocks. and Representing two joint descriptors respectively and The k-th binary feature block This represents the local similarity of the k-th binary feature block. The & represents the number of 1s in the binary feature block; & is the bitwise AND operation. Query the joint descriptor database, find the corresponding container in the hash table, and obtain the candidate joint descriptors for all candidate submaps.

[0035] In a preferred embodiment, geometric verification is performed by calculating the planar overlap between the query submap and the candidate submap, including: To retrieve the list of planes of the query submap The planar list of candidate submaps is as follows ; Using the center point of the candidate submap plane Build a Tree, for each query submap plane center point , For a set of planes in a single query submap, a coarse transformation is first used to obtain the rigid body transformation matrix. Conversion Then Search for the nearest point in the tree The overlap between two planes is determined by checking the difference between their planar distance and normal vector. The constraints for the planar distance and normal vector are as follows:

[0036] in, and These represent the unit normal vectors in the query submap plane and the candidate submap plane, respectively. Indicates querying the coordinate system of the sub-map To candidate submap coordinate system transformation, and These represent querying points from the submap plane list and the candidate submap plane list, respectively. and These represent the distance and angle thresholds, respectively. Indicates querying the coordinate system of the sub-map To candidate submap coordinate system The rotating part; If a pair of planes satisfies the constraints of planar distance and normal vector, then the pair of planes are considered to overlap.

[0037] In a preferred embodiment, determining whether a candidate submap is a valid loop closure detection result includes: After examining all planes of the query submap, calculate the query submap. and candidate submap Percentage of planar overlap between As shown in the following formula:

[0038] For each candidate sub-map, determine whether the calculated percentage of planar overlap exceeds a threshold. If the number of loop closures exceeds a certain threshold, then the loop closure detection for that candidate submap is valid.

[0039] In another embodiment, a computer storage medium stores a computer program thereon, and when the computer executes the computer program, it implements the three-dimensional scene modeling method based on laser point cloud scanning and joint descriptors as described above.

[0040] In another embodiment, an electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor runs the computer program stored in the memory, wherein the computer program, when executed, implements the three-dimensional scene modeling method based on laser point cloud scanning and joint descriptors as described above.

[0041] In another embodiment, a 3D scene modeling system based on laser point cloud scanning and joint descriptors includes: The global map building module uses LiDAR to build a global map. The joint descriptor encoding module obtains a sub-map from the constructed global map, extracts key points from the sub-map, the key points being the pixels with the highest intensity in a local area, and encodes the extracted key points into a joint descriptor, the joint descriptor including a triangle descriptor and three binary descriptors corresponding to the three vertices forming the triangle; The loop closure detection module uses the sub-map obtained from the query as the query sub-map, obtains candidate joint descriptors for all candidate sub-maps based on the query sub-map, and performs binary similarity checks and geometric verification of triangles on the candidate joint descriptors to determine whether the candidate sub-map is a valid loop closure detection result. If it is a valid loop closure detection result, loop closure detection is performed and the global map is updated.

[0042] The following example illustrates the workflow of a 3D scene modeling system based on laser point cloud scanning and joint descriptors, including the following steps: Step 1: Reconstruct the batch data acquired by the LiDAR and camera. Based on the timestamps corresponding to the batch data acquired by the camera, reconstruct the timestamps of the batch data acquired by the LiDAR to ensure that the frequency of the batch data acquired by the camera is consistent with the frequency of the batch data acquired by the LiDAR. During the reconstruction process, IMU data backpropagation is used for distortion correction.

[0043] Step 2: Map construction is performed using LiDAR / Inertial Odometry (IMU). The specific construction method can adopt existing methods, which will not be elaborated here. The state is estimated by using the forward propagation of IMU data. The estimated pose information is input into the iterative error Kalman filter to further optimize the pose information and output the global map and pose information.

[0044] Step 3: Obtain all sub-maps from the constructed global map, extract key points from each sub-map, and encode them into binary and triangle joint descriptors. Store the encoded joint descriptors in the database for management.

[0045] Step 4: Query all candidate descriptors from the database, perform binary similarity checks and triangle geometric verification to determine whether the candidate submap is a valid loop closure detection result.

[0046] Step 5: If the loop closure detection result is valid, perform loop closure detection to eliminate the accumulated error in Step 2 and output a high-precision global map and pose information.

[0047] like Figure 2 As shown, step 1 specifically includes: Step 1.1: Scan with one frame from the lidar For example, the scan begins ,finally A total of 6 camera images were captured during this period, taken at [times not specified]. , , , , and At any moment, but only when The camera image at any given moment contains a single frame of LiDAR point cloud.

[0048] Step 1.2: Based on the temporal characteristics of continuous scanning by lidar, arrive The batch point cloud collected during this period is defined as a reconstruction scan. This scan corresponds to Camera images captured at any given time. The reconstruction scan can be derived similarly. , ,……, These scans correspond to , ,……, The reconstructed radar scan frequency of the camera image at any given moment is kept consistent with the frequency of the input camera image.

[0049] Step 3 specifically includes: Step 3.1: Key point extraction is achieved through planar detection, then a reference plane is generated, height encoding is performed on this plane, and key points are extracted on each height-encoded image. Finally, a binary and triangle joint descriptor is constructed.

[0050] Step 3.2: Manage the extracted key points using a hash table. Construct a simple and efficient hash function to calculate the hash key from the quantized triangle side lengths. Specifically, to quantize the triangle, a fixed resolution is used, and the quantized side lengths are calculated, as shown in Equation (1.1): (1.1) in, It calculates the hash value of list L. It is the execution layer function that calculates the hash value. These are the original side lengths of the three sides of the triangle. It is the quantized side length (processed with a fixed resolution to map continuous side lengths to discrete values). It is the scaling factor. This is the number of hash buckets.

[0051] Step 3.1 includes the following specific implementation steps: Step 3.1.1: Given a point cloud sub-map, first perform planar detection using region growing. Divide the entire point cloud into sub-maps of size [size missing]. (For example The voxels are denoted by a set of points. Then, the point covariance matrix is ​​calculated for each voxel. As shown in equation (1.2): (1.2) in, It is the centroid (mean point) of the point cloud within the voxel. It is the i-th three-dimensional coordinate within the voxel. This represents the total number of points contained within a voxel. Planar features can be identified by calculating the eigenvalues ​​and eigenvectors of the covariance matrix.

[0052] The specific method involves determining the planarity of a point cloud by the relative magnitudes of its eigenvalues. If a eigenvalue is significantly smaller than the other two, the points within that voxel are considered to form a plane. This process is performed across the entire point cloud submap, identifying all possible planar regions.

[0053] Step 3.1.2: After obtaining the list of planes, continue generating reference planes. This involves merging adjacent planes to form a larger plane. Plane merging begins by selecting an initial plane voxel and progressively checks planes in adjacent voxels. If planes in adjacent voxels have similar normal vectors and their distance is close to zero, they are merged with the initial plane. If the initial plane voxel... and adjacent planar voxels With a center point and and normal vector and The merging criteria are as shown in equation (1.3): (1.3) in, and These are two preset thresholds. If these conditions are met, then adjacent planar voxels will be... With the initial planar voxel Merge. After merging, the new plane... center point and normal vector It can be calculated using equation (1.4): (1.4) in, and Representing adjacent planar voxels respectively With the initial planar voxel The number of points, and Representing adjacent planar voxels respectively With the initial planar voxel The center point, and Representing adjacent planar voxels respectively With the initial planar voxel covariance, It's a new plane. The covariance.

[0054] In addition, through the By performing eigenvalue decomposition, the normal vector of the merging plane can be calculated. This merging process continues in the form of region growth until there are no more planes in the adjacent voxels.

[0055] Step 3.1.3: After obtaining the reference plane, Point cloud is projected onto each reference plane to create A highly encoded image, each pixel region is... Square meters. Select. A trade-off needs to be struck between computational efficiency and the ability to capture sufficient detail in highly encoded images. To encode height information, the maximum height of a pixel on each plane is selected. and divide it into Layers, each with a fixed resolution. For each pixel, calculate a... A binary string composed of bits If the corresponding layer contains any point within that height range, this bit is set to 1; otherwise, it is set to 0. The pixel intensity is obtained by summing all m-bit values, and this intensity, along with a binary string, is stored in each pixel.

[0056] Step 3.1.4: When you have After generating highly encoded images, key points are extracted from each image by identifying local features. Keypoints are determined by the pixels with the highest intensity within a region. These local maxima represent densely populated areas of points, thus preserving the original... The most information in the point cloud. To suppress the number of keypoints, we set a threshold on the local maximum intensity. Only pixels with the highest local intensity exceeding this threshold are selected as keypoints. Once a keypoint is identified in the highly coded image, its location within the submap is determined. Coordinates. First, by mapping all points above the pixel used for height encoding. The coordinates are averaged to determine the point positions on the reference plane. Then, using these in-plane positions, the complete keypoints are calculated based on the plane parameters. Location.

[0057] Step 3.1.5: The binary and triangular joint descriptor combines the advantages of binary and triangular descriptors, consisting of a triangular descriptor and three binary descriptors corresponding to the three key points (i.e., vertices) that form the triangle. This combination allows the triangular descriptor to provide pose invariance and the correspondence between vertices, while the binary descriptors provide the local point cloud distribution, thereby further improving matching accuracy.

[0058] Step 4 specifically includes: Step 4.1: The sub-maps obtained by sequentially querying the sub-map database built from the previous global map are used as query sub-maps. For the query sub-map, the binary and triangle joint descriptors are extracted according to the method in step 3.1, their hash keys are calculated (as shown in equation (1.5)), the descriptor database is queried, and the corresponding container is found in the hash table.

[0059] (1.5)

[0060] in, Represents two joint descriptors and The overall similarity score is calculated, and the final output range is between [0,1]. The larger the value, the higher the similarity. Calculate the similarity between two binary feature blocks. and Representing two joint descriptors respectively and The k-th binary feature block This represents the local similarity of the k-th binary feature block. This represents the number of 1s in the binary feature block. & The & operator is used for bitwise AND operations on binary feature blocks.

[0061] Next, for all candidate descriptors in the container Perform binary descriptor similarity checks to eliminate possible outliers. For a candidate descriptor... If it is a query descriptor For a true match to occur, the vertices of the triangle should have similar binary descriptors. Furthermore, since the side lengths are stored in unique ascending order, checking the similarity of the binary descriptors helps verify the validity of the match.

[0062] Step 4.2: To enhance robustness, the RANSAC algorithm is used to find a transformation that maximizes the number of correctly matching vertices. This transformation is called the coarse transformation and is denoted as: . Let be the rigid body transformation matrix. For rotation matrix, This is a translation vector. When a coarse transformation is applied to the triangle vertices in the query submap, represented in the submap's local coordinate system, they are directly aligned with the candidate submap.

[0063] Step 4.3: Obtain the rigid body transformation matrix using a coarse transformation. The planar overlap between the query submap and candidate submaps is calculated for geometric verification. The list of planes for the query submap is obtained as follows: These planes have been extracted, and the list of planes for the candidate submap is as follows: Using the center point of the candidate submap plane Build a Tree. Then, for each query sub-map plane center point , For a single query submap's set of planes, a coarse transformation is first applied. Conversion Then Search for the nearest point in the tree The overlap of two planes is determined by checking the difference between their planar distance and normal vector, as shown in equation (1.6): (1.6) in, and These represent the unit normal vectors in the query submap plane and the candidate submap plane, respectively. Indicates querying the coordinate system of the sub-map To candidate submap coordinate system transformation, and These represent querying points from the submap plane list and the candidate submap plane list, respectively. and These represent the distance and angle thresholds, respectively. Indicates querying the coordinate system of the sub-map To candidate submap coordinate system The rotation part. If a pair of planes satisfies the plane distance and normal vector constraints in equation (1.6), then the pair of planes are considered to overlap.

[0064] After examining all planes of the query submap, calculate the query submap. and candidate submap Percentage of planar overlap between As shown in equation (1.7): (1.7) For each candidate submap, determine whether the calculated percentage of planar overlap exceeds a certain threshold. If the number of loop closures exceeds the threshold for a candidate submap, it is considered valid. If multiple candidate submaps pass geometric verification, the overlap percentage is selected. The highest value is selected as the best candidate. By performing geometric verification, the chance of detecting erroneous loop closures is significantly reduced, thereby improving the accuracy of the system.

[0065] Step 5 includes: Step 5.1: After passing the geometric verification, the loop is determined to be a true loop closure. The loop closure constraint is added to the factor map for optimization. This can be achieved using existing methods, which will not be elaborated here. The cumulative error of the historical trajectory is corrected, and the global map of the 3D scene is updated.

[0066] from Figure 2 As can be seen, the radar timestamps are reconstructed according to the frequency of the camera timestamps, and then the reconstructed radar data stream is segmented. This allows the reconstructed radar data timestamps to be aligned with the camera data timestamps, achieving a synchronization effect.

[0067] Based on the above scheme, in order to verify the actual effect of the method of the present invention, comparative experiments were conducted using data collected by lidar, such as... Figure 3 and Figure 4 As shown, the specific analysis is as follows: from Figure 3 As can be seen, traditional 3D scene modeling methods based on laser point clouds can only construct 3D models of relatively small scenes, and are prone to errors when constructing large scenes, such as... Figure 3 The area within the red circle is visible.

[0068] from Figure 4 As can be seen, when using the method of this invention for batch modeling, the mapping effect is better than that of traditional mapping algorithms, and it can achieve batch 3D reconstruction of large scenes, such as... Figure 5 As shown.

[0069] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A 3D scene modeling method based on laser point cloud scanning and joint descriptors, characterized in that, Includes the following steps: A global map is constructed using LiDAR. Obtain a sub-map from the constructed global map, extract key points from the sub-map, the key points being the pixels with the highest intensity in a local area, and encode the extracted key points into a joint descriptor, the joint descriptor including a triangle descriptor and three binary descriptors corresponding to the three vertices forming the triangle; The submap obtained from the query is used as the query submap. Based on the query submap, candidate joint descriptors of all candidate submaps are obtained. Binary similarity checks and geometric verification of triangles are performed on the candidate joint descriptors to determine whether the candidate submap is a valid loop closure detection result. If it is a valid loop closure detection result, loop closure detection is performed and the global map is updated.

2. The 3D scene modeling method based on laser point cloud scanning and joint descriptors according to claim 1, characterized in that, Before using LiDAR for map building, the following also needs to be included: The LiDAR scanning data is reconstructed based on the timestamps of the camera images, so that the frequency of the camera image data is consistent with the frequency of the LiDAR data.

3. The 3D scene modeling method based on laser point cloud scanning and joint descriptors according to claim 1, characterized in that, Extracting key points from a submap includes: Perform planar detection on the submap using region growing to obtain a list of planes; After obtaining the list of planes, a reference plane is generated; After obtaining the reference plane, Point cloud is projected onto each reference plane to create A highly encoded image; Keypoints are determined by identifying the pixels with the highest intensity within local regions of a highly coded image.

4. The 3D scene modeling method based on laser point cloud scanning and joint descriptors according to claim 3, characterized in that, Generating the reference plane includes: Select an initial planar voxel And check adjacent planar voxels Obtain the initial planar voxels and adjacent planar voxels center point and and normal vector and If the following conditions are met: in, and There are two preset thresholds, and max() is used to find the maximum value. It is the inverse cosine function; Then adjacent planar voxels With the initial planar voxel Merge, after merging, the new plane center point and normal vector Calculated using the following formula: in, and Representing adjacent planar voxels respectively With the initial planar voxel The number of points, and Representing adjacent planar voxels respectively With the initial planar voxel covariance, It's a new plane. covariance; Through the Perform eigenvalue decomposition to calculate the normal vector of the merging plane. ; The merging process is carried out in the form of region growth until there are no planes in the adjacent voxels.

5. The 3D scene modeling method based on laser point cloud scanning and joint descriptors according to claim 1, characterized in that, After encoding the extracted keypoints into joint descriptors, the following steps are also included: The extracted feature points are managed using a hash table. A hash function is constructed to calculate the hash key from the quantized triangle side lengths. A fixed resolution is used to map continuous side lengths to discrete values, and the quantized side lengths are calculated, as shown in the following formula: in, It calculates the hash value of list L. It is the execution layer function that calculates the hash value. These are the original side lengths of the three sides of the triangle. It is the quantized side length. It is the scaling factor. is the number of hash buckets, and Mod() is the modulo function.

6. The 3D scene modeling method based on laser point cloud scanning and joint descriptors according to claim 5, characterized in that, Based on the query submap, the candidate joint descriptors of all candidate submaps are obtained, including: Extract the combined binary and triangular descriptor from the query submap and calculate the hash key: in, Represents two joint descriptors and Overall similarity Calculate the similarity between two binary feature blocks. and Representing two joint descriptors respectively and The k-th binary feature block This represents the local similarity of the k-th binary feature block. The & represents the number of 1s in the binary feature block; & is the bitwise AND operation. Query the joint descriptor database, find the corresponding container in the hash table, and obtain the candidate joint descriptors for all candidate submaps.

7. The 3D scene modeling method based on laser point cloud scanning and joint descriptors according to claim 1, characterized in that, Geometric verification is performed by calculating the planar overlap between the query submap and the candidate submap, including: To retrieve the list of planes of the query submap The planar list of candidate submaps is as follows ; Using the center point of the candidate submap plane Build a Tree, for each query submap plane center point , For a single query submap, a set of planes; use a coarse transformation to obtain the rigid body transformation matrix. Conversion ,exist Search for the nearest point in the tree The overlap between two planes is determined by checking the difference between their planar distance and normal vector. The constraints for the planar distance and normal vector are as follows: in, and These represent the unit normal vectors in the query submap plane and the candidate submap plane, respectively. Indicates querying the coordinate system of the sub-map To candidate submap coordinate system transformation, and These represent querying points from the submap plane list and the candidate submap plane list, respectively. and These represent the distance and angle thresholds, respectively. Indicates querying the coordinate system of the sub-map To candidate submap coordinate system The rotating part; If a pair of planes satisfies the constraints of planar distance and normal vector, then the pair of planes are considered to overlap.

8. The 3D scene modeling method based on laser point cloud scanning and joint descriptors according to claim 7, characterized in that, Determining whether a candidate submap is a valid loop closure detection result includes: After examining all planes of the query submap, calculate the query submap. and candidate submap Percentage of planar overlap between As shown in the following formula: For each candidate sub-map, determine whether the calculated percentage of planar overlap exceeds a threshold. If the number of loop closures exceeds a certain threshold, then the loop closure detection for that candidate submap is valid.

9. A 3D scene modeling system based on laser point cloud scanning and joint descriptors, used to execute the method as described in any one of claims 1-8, characterized in that, include: The global map building module uses LiDAR to build a global map. The joint descriptor encoding module obtains a sub-map from the constructed global map, extracts key points from the sub-map, the key points being the pixels with the highest intensity in a local area, and encodes the extracted key points into a joint descriptor, the joint descriptor including a triangle descriptor and three binary descriptors corresponding to the three vertices forming the triangle; The loop closure detection module uses the sub-map obtained from the query as the query sub-map, obtains candidate joint descriptors for all candidate sub-maps based on the query sub-map, and performs binary similarity checks and geometric verification of triangles on the candidate joint descriptors to determine whether the candidate sub-map is a valid loop closure detection result. If it is a valid loop closure detection result, loop closure detection is performed and the global map is updated.

10. A computer storage medium having a computer program stored thereon, characterized in that, When the computer executes the computer program, it implements the three-dimensional scene modeling method based on laser point cloud scanning and joint descriptors as described in any one of claims 1-8.

11. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor runs the computer program stored in the memory. When the computer program is executed, it implements the three-dimensional scene modeling method based on laser point cloud scanning and joint descriptors as described in any one of claims 1-8.

Images