A High-Precision Mapping and Identification Method for Substations Using Handheld LiDAR

Abstract

This invention discloses a high-precision mapping and identification method for substations using a handheld LiDAR. It utilizes a handheld rotating LiDAR scanner for high-precision mapping of the surrounding environment and the substation. The handheld rotating LiDAR device enables rapid and accurate 3D scanning of the substation, generating detailed terrain and structural data. In particular, it accurately identifies specific substation equipment by combining image recognition and point cloud analysis. The point cloud generates the morphological structure of the substation scene, and details are generated from RGBD images. The morphological structure and details are then fused to obtain a detailed morphological structure model of the substation scene. Finally, analysis of the initial substation scene model can detect detailed defects in the substation, thus enabling better measurement and maintenance. Analysis of the mapped substation scene model allows for the analysis of the state changes of various equipment in the substation, such as corrosion, cracks, deformation, or damage, which is helpful for the planning, construction, and maintenance of the substation.

Description

Technical Field

[0001] This invention relates to the field of surveying and mapping, and in particular to a high-precision surveying and identification method for substations using a handheld lidar. Background Technology

[0002] Mapping substations is crucial for identifying the condition of substation equipment, such as corrosion, cracks, deformation, and damage. Traditional mapping methods in modern substation monitoring and maintenance often rely on manual inspections and static image acquisition. However, static images, simply by capturing images of the substation scene, cannot reflect the actual condition of the equipment, lacking depth and spatial location information, thus failing to accurately identify the deformation of equipment within the substation environment.

[0003] Therefore, it is necessary to extract the depth information from the images and utilize handheld devices for dual acquisition of images (specifically depth images) and laser data to model, identify, and detect anomalies on the surface of power equipment. A significant challenge lies in accurately registering laser point cloud data and image data (specifically RGBD images) during model building. This involves transforming data acquired from different sources, at different times, or from different sensors into the same coordinate system. In substation equipment inspection, the registration of laser point cloud data and RGBD image data is crucial. The main purpose of this process is to accurately correlate these two types of data for subsequent 3D modeling and anomaly detection.

[0004] In summary, the ability to register and model RGBD images and laser point cloud data using handheld devices has a significant impact on the assessment and detection of device anomalies. Summary of the Invention

[0005] The purpose of this invention is to provide a high-precision mapping and identification method for substations using a handheld lidar, which solves the aforementioned technical problems pointed out in the prior art.

[0006] This invention provides a high-precision mapping and identification method for substations using a handheld lidar, comprising the following steps:

[0007] The handheld LiDAR device is used to acquire RGB image data of the substation; simultaneously, the handheld LiDAR device is used to acquire LiDAR point cloud data of the substation.

[0008] An RGBD image is obtained by calculating the RGB image data using the camera parameters of the image acquisition device;

[0009] Image registration is performed based on the RGBD image and the laser point cloud data to obtain a substation scene model;

[0010] The substation equipment status is obtained by analyzing the substation scenario model.

[0011] Preferably, the step of performing image registration based on the RGBD image and the laser point cloud data to obtain the substation scene model includes the following steps:

[0012] Extract image feature points from the RGBD image, and simultaneously extract point cloud feature points from the laser point cloud data;

[0013] A registered feature point group is obtained by registering the image feature points with the point cloud feature points.

[0014] Based on the registration feature point group, the RGBD image and the laser point cloud data are registered to obtain the substation scene model.

[0015] Preferably, the step of registering the image feature points with the point cloud feature points to obtain a registered feature point group includes the following steps:

[0016] An image feature descriptor is obtained based on the image feature points; a point cloud feature descriptor is obtained based on the point cloud feature points; and a first registration point pair is obtained by searching the KD tree constructed from the point cloud feature descriptors based on the image feature descriptors.

[0017] The initial transformation matrix is ​​obtained by estimating the first registration point group through RANSAC coarse registration processing;

[0018] Based on the initial transformation matrix, iterative processing and analysis are performed through ICP fine registration to obtain the registration feature point group.

[0019] Preferably, the step of searching and constructing the first registration point pair based on the image feature descriptor in the KD tree constructed from the point cloud feature descriptor specifically includes:

[0020] The image feature descriptor is used to search for K nearest neighbors in the KD tree to obtain K first registration point cloud feature points corresponding to each image feature point;

[0021] A first registration point pair is constructed based on the image feature points and the k first registration point cloud feature points.

[0022] Preferably, each element in the initial transformation matrix represents a second registration point group; the second registration point group represents a combination of image feature points and point cloud feature points with higher accuracy obtained by re-registering the first registration point group.

[0023] Preferably, the iterative processing and analysis based on the initial transformation matrix through ICP fine registration to obtain the registration feature point group includes the following steps:

[0024] Initialize the iteration conditions; the iteration conditions include an iteration counter, a maximum number of iterations, and a minimum error threshold; the number of iterations of the counter is initially set to 0;

[0025] Iterative processing operation: Based on the initial transformation matrix, for each of the image feature points S i The point cloud S' to be registered is obtained through calculation and processing. i ;

[0026] Increment the iteration count of the iteration counter by 1 to obtain the current iteration count, and simultaneously, based on the point cloud S' to be registered... i With the point cloud S' to be registered i The shortest distance laser point cloud data t j The registration error E is calculated.

[0027] Based on the registration error E and the minimum error threshold, a group of registration feature points to be determined is iteratively output.

[0028] The registration degree is calculated based on the set of registration feature points to be determined; the target registration feature point set is then output based on the registration degree and a preset registration degree threshold.

[0029] Preferably, the step of judging and iteratively outputting the registration feature point group to be determined based on the registration error E and the minimum error threshold specifically includes the following steps:

[0030] Determine whether the registration error E is less than or equal to the minimum error threshold. If so, output the image feature point S. i With the laser point cloud data t j The system constructs a group of feature points to be registered; if not, it determines whether the current iteration number is greater than or equal to the maximum iteration number; if so, it outputs the image feature point S. i With the laser point cloud data t j If not, then based on the registration error E and the laser point cloud data t, form a set of registration feature points to be determined; otherwise, use the registration error E and the laser point cloud data t. j and the image feature points S i A new initial transformation matrix is ​​obtained through analysis and calculation. Based on the new initial transformation matrix, the above iterative processing operation is returned until the output is the group of registration feature points to be determined.

[0031] Preferably, the step of calculating the registration degree based on the group of registration feature points to be determined, and iteratively outputting the target registration feature point group based on the registration degree and a preset registration degree threshold, specifically includes the following steps:

[0032] The registration degree is calculated based on the group of registration feature points to be determined; it is then determined whether the registration degree is greater than or equal to a preset registration degree threshold. If so, the group of registration feature points to be determined is output as the target registration feature point group; otherwise, a new initial transformation matrix is ​​calculated using the image feature points and point cloud feature points corresponding to the group of registration feature points to be determined, and the above iterative processing operation is returned until the registration degree is greater than or equal to the preset registration threshold, and the target registration feature point group is output.

[0033] Preferably, the point cloud S' to be registered i The calculation method is as follows:

[0034] s′ i =R0s i +t0;

[0035] In the formula, R0 is the rotation matrix in the initial transformation matrix; t0 is the translation vector in the initial transformation matrix;

[0036] The registration error E is calculated as follows:

[0037] E=(1 / m)*Σ(min‖s' j -t j || 2 );

[0038] In the formula, m is the number of image feature points; min‖s' i -t j ‖ represents the point cloud S' to be registered. i With the point cloud S' to be registered i The Euclidean distance between the shortest distance laser point cloud data tj.

[0039] Preferably, the step of calculating the registration degree based on the set of registration feature points to be determined includes the following steps:

[0040] Based on multiple laser point cloud data tj from the multiple groups of feature points to be registered and the point cloud S' to be registered. i Multiple registration blocks are obtained through clustering;

[0041] The spatial autocorrelation Δ between the q-th registration block and each of the other registration blocks p is calculated. q,p ;

[0042] Based on the spatial autocorrelation Δ q,p The registration degree is obtained by calculating the number of registration blocks w.

[0043] The registration degree is calculated as follows:

[0044]

[0045] Where, γ w Δ represents the registration degree of the w-th registration block; q,p The expression represents the spatial autocorrelation between the q-th registration block and all other registration blocks p except the q-th registration block; w represents the total number of registration blocks. To calculate the spatial autocorrelation between each pair of these registration blocks, the total number of spatial autocorrelation to be calculated is 0.5ww-1; []* represents a regular expression, [] represents a character class, matching any single character within the square brackets; * is a quantifier, indicating that the character class can appear zero or more times.

[0046] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0047] Analysis of the high-precision substation mapping and identification method using handheld lidar provided by this invention reveals that, in practical applications, a handheld rotating lidar scanner is used to perform high-precision mapping of the surrounding environment and substation. Through this handheld rotating lidar device, a rapid and accurate 3D scan of the substation can be performed, generating detailed terrain and structural data. In particular, image recognition and point cloud analytics are used to accurately identify specific substation equipment. The point cloud generates the morphological structure of the substation scene, and RGBD images containing RGB and depth information are generated from RGB image data and camera parameters to create details. Then, the morphological structure and details are fused to obtain a detailed morphological structure model of the substation scene. Finally, analysis of the initial substation scene model can detect detailed defects in the substation, thus enabling better measurement and maintenance. Analysis of the mapped substation scene model allows for the analysis of the state changes of various equipment in the substation, such as corrosion, cracks, deformation, or damage, which is helpful for the planning, construction, and maintenance of the substation. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the main process of a high-precision substation mapping and identification method using handheld lidar;

[0049] Figure 2 This is a schematic diagram of a point cloud simulation of a substation scene in a high-precision mapping and identification method for substations using a handheld lidar.

[0050] Figure 3 This is a schematic diagram illustrating the operation steps for registering a substation scene model in a high-precision substation mapping and identification method using a handheld lidar.

[0051] Figure 4 A schematic diagram of the KD tree in a high-precision substation mapping and identification method using a handheld lidar;

[0052] Figure 5 This is a schematic diagram of the operation process for obtaining a registration feature point group through iterative processing in a high-precision substation mapping and identification method using a handheld lidar. Detailed Implementation

[0053] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings.

[0055] like Figure 1 As shown in the figure, this invention provides a high-precision mapping and identification method for substations using a handheld lidar, comprising the following steps:

[0056] Step S10: Acquire RGB image data of the substation using the image acquisition device mounted on the handheld LiDAR device; simultaneously acquire LiDAR point cloud data of the substation using the LiDAR device mounted on the handheld LiDAR device.

[0057] Step S20: Calculate the RGBD image based on the RGB image data using the camera parameters of the image acquisition device;

[0058] It should be noted that in the above embodiments of this application, the RGBD image is a combination of RGB and RGBD images, containing four channels: red, green, blue, and depth. For the depth channel, each pixel value represents the distance from the corresponding point in the scene to the camera. The RGBD image can be viewed as an organized point cloud, where each pixel can be converted into a point in 3D space. However, due to issues such as the image acquisition device on the handheld LiDAR device and the acquisition angle of the LiDAR device, the RGBD image cannot be directly converted into LiDAR point cloud data. Instead, it needs to be registered with the LiDAR point cloud data to achieve a complete match between the RGB information of objects in the substation scene and the LiDAR point cloud data. This improves the accuracy of the obtained substation scene model and, consequently, the accuracy of the subsequent analysis of the substation equipment status. Through the calculations in the above embodiments of this application, richer visual and spatial information can be provided, thereby improving the efficiency of generating the substation scene model.

[0059] Step S30: Perform image registration based on the RGBD image and the laser point cloud data to obtain a substation scene model;

[0060] It should be noted that the above-mentioned substation scene model is a scene model that reflects the details and form of the substation. In the above-mentioned embodiment of this application, the morphological structure of the substation scene is generated from point cloud, the details are generated from RGBD image, and then the morphological structure and details are fused to obtain the detailed morphological structure model of the substation scene (i.e., the above-mentioned substation scene model), and the scene is detailed and complete.

[0061] Step S40: Analyze the substation scenario model to obtain the equipment status of the substation;

[0062] It should be noted that the above analysis specifically refers to using the high-resolution visual information of the model to detect anomalies on the surface of equipment, such as corrosion and cracks; by comparing the actual 3D shape of the equipment with the standard shape, deformation or damage can be identified; and the relative positions and distances between equipment can be checked to ensure compliance with safety standards.

[0063] The embodiments described above utilize a handheld rotating lidar scanner for high-precision mapping of the surrounding environment and substations. Through this handheld rotating lidar device, a rapid and accurate three-dimensional scan of the substation can be performed (e.g., Figure 2 As shown in the figure, it generates detailed terrain and structural data, especially by using image recognition and point cloud to accurately identify specific substation equipment; the method designed can improve the efficiency and accuracy of surveying, especially in complex substation environments, and can provide more accurate measurement results, which is helpful for the planning, construction and maintenance of substations; by analyzing the current surveyed substation scene model, it is possible to analyze the state changes of various equipment in the substation, such as corrosion, cracks, deformation or damage of the equipment;

[0064] The above-described embodiment of this application generates the morphological structure of a substation scene from point clouds, generates details from RGBD images, and then fuses the morphological structure with the details to obtain a detailed morphological structure model of the substation scene (i.e., the above-described substation scene model). Finally, by analyzing the initial substation scene model, detailed defects in the substation can be detected, thereby enabling better measurement and maintenance.

[0065] Specifically, such as Figure 3 As shown, in step S30, image registration is performed based on the RGBD image and the laser point cloud data to obtain the substation scene model, including the following operation steps:

[0066] Step S31: Extract image feature points from the RGBD image and simultaneously extract point cloud feature points from the laser point cloud data;

[0067] Step S32: Register the image feature points with the point cloud feature points to obtain a registered feature point group;

[0068] Step S33: Based on the registration feature point group, register the RGBD image and the laser point cloud data to obtain the substation scene model.

[0069] It should be noted that after obtaining the registration feature point group as described above, the substation scene model can be obtained by triangulation using the RGBD image pixel coordinates corresponding to the registration feature point group and the laser point cloud data coordinates. Specifically, after establishing the registration relationship between the image feature points and the point cloud feature points, the RGBD image pixel coordinates of the RGBD image corresponding to the image feature points and the laser point cloud data coordinates of the laser point cloud data corresponding to the point cloud feature points are also established. Furthermore, through the registration relationship (that is, each RGBD image pixel coordinate is registered to obtain the corresponding laser point cloud data coordinates), the RGBD image pixel coordinates and laser point cloud data coordinates corresponding to the registration results are mapped to a unified coordinate system (usually the coordinate system of the laser point cloud data) to achieve image registration and obtain the substation scene model.

[0070] The embodiments described above first extract image feature points from the RGBD image and simultaneously extract point cloud feature points from the laser point cloud data. By extracting feature points, the complexity of data processing can be reduced, thereby reducing the number of points that need to be registered and improving registration efficiency. Furthermore, registration is performed based on the extracted image feature points and point cloud feature points to obtain a registration feature point group, enabling data from different sources (RGBD image and laser point cloud) to be aligned for subsequent processing. Finally, the coordinates of each RGBD image pixel are mapped to the coordinates of the laser point cloud data using a triangulation method, thereby achieving data unification. This not only enhances the spatial relationship of the data but also improves the accuracy and reliability of the final generated scene model.

[0071] Specifically, in step S32, a registered feature point group is obtained by registering the image feature points with the point cloud feature points, including the following steps:

[0072] Step S321: Analyze the image feature points to obtain an image feature descriptor; analyze the point cloud feature points to obtain a point cloud feature descriptor; search the KD tree constructed from the point cloud feature descriptor to obtain a first registration point pair;

[0073] Specifically, the above embodiments of this application construct a KD tree based on the point cloud feature descriptor; use the image feature descriptor to search for K nearest neighbors in the KD tree to obtain K first registration point cloud feature points corresponding to each image feature point; and construct a first registration point pair based on the image feature points and the k first registration point cloud feature points.

[0074] It should be noted that the above embodiment of this application is a coarse registration operation (first registration operation), which aims to filter out fuzzy matching options and retain some more reliable image feature points corresponding to K point cloud feature points;

[0075] The above point cloud feature descriptor is obtained by analyzing the feature points of the point cloud using Fast Point Feature Histogram (FPFH), a three-dimensional feature descriptor based on point clouds, mainly used for point cloud feature extraction and registration, and often used for coarse registration of point clouds.

[0076] For example, such as Figure 4 As shown: The following simplified 3D point cloud feature descriptors currently exist: A(2,3,7), B(4,7,9), C(5,4,5), D(7,2,11), E(9,6,2), F(1,8,8), etc. Figure 4 In the diagram, the first layer is divided along the x-axis (7 is the dividing point); the second layer is divided along the y-axis; the third layer is divided along the z-axis; then the process loops back to the x-axis, and so on.

[0077] The aforementioned image feature descriptors are generated by processing image feature points using algorithms such as SIFT or SURF;

[0078] The KD tree in the above embodiment of this application is a KD tree of point cloud feature descriptors constructed by the FLANN library. Then, based on each image feature descriptor, the Lower's algorithm (the Lower's algorithm is used to further filter matching points to obtain excellent matching points, which is the so-called "refining" and is generally used to further obtain excellent matching points) is used to search for K first registration point cloud feature points, filtering out fuzzy matches and retaining only the most reliable matching pairs; this provides a good starting point for subsequent fine registration and greatly reduces the search space;

[0079] For example: Given an image feature descriptor p1, its two nearest neighbors found in the point cloud feature descriptor are q1 and q2, with distances of d1 and d2 respectively;

[0080] If d1 / d2 < 0.8 (the threshold is usually between 0.7 and 0.8), then (p1, q1) is considered a good matching pair;

[0081] Finally, we obtain an initial set of matching pairs, such as {(p1, q1), (p2, q2), ..., (pn, qn)}, which provides a good starting point for subsequent steps.

[0082] Step S322: Estimate the initial transformation matrix by performing RANSAC coarse registration on the first registration point group; each element in the initial transformation matrix represents the second registration point group; the second registration point group represents the combination of image feature points and point cloud feature points with higher accuracy obtained after re-registration of the first registration point group (re-registration is RANSAC coarse registration, i.e., the second registration operation).

[0083] It should be noted that the RANSAC coarse registration in the above embodiments of this application is processed by random sampling and consistency check, which can find reliable transformations in the presence of a large number of outliers (erroneous matches). It can effectively remove erroneous correspondences in the initial match (i.e., the first registration point group), providing a good initial estimate for the subsequent ICP algorithm, and greatly improving the convergence speed and accuracy of ICP.

[0084] The aforementioned initial transformation matrix is ​​used in the registration process of the substation scene model. Since this embodiment of the application processes data in 3D space, the initial transformation matrix estimated by RANSAC coarse registration is a 4×4 homogeneous transformation matrix, which describes rigid body transformations (rotation and translation) in 3D space. Through the initial transformation matrix, the 3D information in the RGBD image can be accurately mapped to the point cloud coordinate system, providing a good initial estimate for the subsequent ICP fine registration, which can improve the efficiency and accuracy of registration.

[0085] The above initial transformation matrix can be expressed as:

[0086] Where R is a 3×3 rotation matrix, describing the rotation transformation from the RGBD image coordinate system to the point cloud coordinate system; t is a 3×1 translation vector, describing the translation from the origin of the RGBD image coordinate system to the origin of the point cloud coordinate system; 0 is a 1×3 zero vector; 1 is a scalar 1; the above 0 and 1 ensure that the matrix can be used for homogeneous coordinate transformations;

[0087] Specifically, the estimation in the above-described embodiment of this application refers to converting the first registration point pair into an array representation, and then randomly selecting 3 pairs of matching points (minimum set); further, calculating the rigid body transformation defined by these 3 pairs of points; then applying the transformation to all image feature points and calculating the number of interior points; finally repeating the above RANSAC iteration operation N times (N is usually 1000-10000), selecting the transformation with the most interior points, and obtaining the initial transformation matrix;

[0088] For example: If there are 100 pairs of first registration points, after RANSAC iterations 1000 times, the optimal transformation T is found. Under this transformation, the error of 80 pairs of matches is less than the threshold (e.g., 1 cm). These 80 pairs are considered to be interior points. Then, a higher quality matching pair (or second registration point pair) is obtained from these 80 interior points corresponding to an initial transformation matrix T that is more reliable than the many first registration points.

[0089] Step S323: Based on the initial transformation matrix, perform iterative processing and analysis through ICP fine registration to obtain the registration feature point group;

[0090] It should be noted that the above-mentioned ICP fine registration is a point cloud fine registration algorithm (i.e., the third registration operation), which can finely align two or more point cloud datasets to facilitate subsequent 3D reconstruction, topology analysis and other operations.

[0091] The above-described embodiments of this application employ three registration steps. First, image feature descriptors and point cloud feature descriptors are extracted from image feature points and point cloud feature points, respectively. Then, a first registration, i.e., coarse registration, is performed based on a KD tree constructed from the point cloud feature descriptors. This aims to filter out unreliable fuzzy matches and retain relatively reliable matching pairs, providing a good starting point for subsequent fine registration. At the same time, it effectively reduces the complexity of data processing. By matching excellent feature point pairs, the search space in subsequent registration processes is reduced, thereby improving the efficiency and accuracy of registration.

[0092] The first registration point group is further processed by the RANSAC algorithm, and random sampling and consistency checks are performed to estimate the initial transformation matrix. The generated initial transformation matrix (4×4 homogeneous transformation matrix) can accurately map the 3D information in the RGBD image to the point cloud coordinate system, which significantly improves the convergence speed and accuracy of subsequent fine registration algorithms (such as ICP).

[0093] Finally, based on the initial transformation matrix, the ICP algorithm is used for iterative processing to finely align the registration feature points, further improving the registration accuracy.

[0094] In the specific implementation process of the above-described embodiments of this application, the technicians discovered that during the ICP fine registration process, since the RGBD image and the laser point cloud data may contain multiple identical transformers simultaneously, directly using the traditional IPC fine registration algorithm for registration operation will result in a registration error between the RGBD image and the laser point cloud data of a certain transformer, thus causing an error in the overall registration of the RGBD image and the laser point cloud data. Therefore, in the specific execution of the ICP fine registration operation, multiple iterative judgment and analysis are required to ensure that the RGBD image and the laser point cloud data of each transformer are correctly registered, thereby ensuring the accurate configuration of the final overall RGBD image and the laser point cloud data. For details, please refer to steps S3231-S3235 in the following embodiments.

[0095] Specifically, such as Figure 5 As shown, in step S323, based on the initial transformation matrix, iterative processing and analysis are performed through ICP fine registration to obtain the registration feature point group, including the following operation steps:

[0096] Step S3231: Initialize iteration conditions; the iteration conditions include an iteration counter, a maximum number of iterations, and a minimum error threshold; the number of iterations of the counter is initially set to 0;

[0097] Step S3232: Iterative processing operation: Based on the initial transformation matrix, process each of the image feature points S i The point cloud S' to be registered is obtained through calculation and processing. i ;

[0098] The point cloud S' to be registered i The calculation method is as follows:

[0099] s′ i =R0s i +t0;

[0100] In the formula, R0 is the rotation matrix in the initial transformation matrix; t0 is the translation vector in the initial transformation matrix;

[0101] The embodiments described above use an initial transformation matrix to rotate and translate the pixels of the RGBD image corresponding to the image feature points, thereby obtaining the point cloud S' to be registered in the laser point cloud coordinate system. i ;

[0102] Step S3233: Increment the iteration count of the iteration counter by 1 to obtain the current iteration count, and simultaneously, based on the point cloud S' to be registered... i With the point cloud S' to be registered i The shortest distance laser point cloud data t j The registration error E is calculated.

[0103] The registration error E is calculated as follows:

[0104] E=(1 / m)*Σ(min‖s' i -t j || 2 );

[0105] In the formula, m is the number of image feature points; min‖s' i -t j ‖ represents the point cloud S' to be registered. i With the point cloud S' to be registered i The shortest distance laser point cloud data t j The Euclidean distance between them;

[0106] It should be noted that in the above embodiments of this application, the registration error reflects the average distance difference between two point clouds, and a smaller error indicates a better registration effect;

[0107] Step S3234: Based on the registration error E and the minimum error threshold, make a judgment and iteratively output the registration feature point group to be determined;

[0108] Specifically, judging and iteratively outputting the registration feature point group to be determined based on the registration error E and the minimum error threshold means: judging whether the registration error E is less than or equal to the minimum error threshold; if so, then outputting the image feature point S. i With the laser point cloud data t j The system constructs a group of feature points to be registered; if not, it determines whether the current iteration number is greater than or equal to the maximum iteration number; if so, it outputs the image feature point S. i With the laser point cloud data t j If not, then based on the registration error E and the laser point cloud data t, form a set of registration feature points to be determined; otherwise, use the registration error E and the laser point cloud data t. j and the image feature points S i A new initial transformation matrix is ​​obtained through analysis and calculation. Based on the new initial transformation matrix, the above operation (i.e. the above iterative processing operation) is returned until the set of registration feature points to be determined is output.

[0109] Step S3235: Calculate the registration degree based on the group of registration feature points to be determined; perform iterative judgment and output the target registration feature point group based on the registration degree and the preset registration degree threshold;

[0110] Specifically, the iterative output of the target registration feature point group based on the registration degree and the preset registration degree threshold refers to: determining whether the registration degree is greater than or equal to the preset registration degree threshold; if so, outputting the registration feature point group to be determined as the target registration feature point group; if not, calculating a new initial transformation matrix using the image feature points and point cloud feature points corresponding to the registration feature point group to be determined (i.e., calculating a new initial transformation matrix according to step S3235 above, that is, using the laser point cloud data t corresponding to the registration feature point group to be determined). j and image feature points S i The calculation and analysis are performed to obtain a new initial transformation matrix, and the above operation (i.e. the above iterative processing operation) is returned until the registration degree is greater than or equal to the preset registration degree threshold, and the target registration feature point group is output.

[0111] It should be noted that, in the above embodiments of this application, the image feature points are first rotated and translated using an initial transformation matrix to obtain the point cloud data to be registered, ensuring that the two are compared and matched in the same coordinate system. Furthermore, by calculating the registration error E between the point cloud to be registered and the laser point cloud, the current registration effect can be quantified, helping to determine whether to continue iterating or end the registration process. In addition, by continuously evaluating the error and the number of iterations, the registration strategy can be flexibly adjusted to ensure that the best registration effect is achieved within a reasonable time.

[0112] Finally, the registration accuracy is calculated to verify the above-mentioned registration feature point group to be determined, thereby ensuring that the analysis and acquisition of the target registration feature point group are more accurate.

[0113] Specifically, in step S3235, calculating the registration degree based on the group of registration feature points to be determined includes the following steps:

[0114] Step S32351: Based on multiple laser point cloud data t from the multiple groups of registration feature points to be determined j and the point cloud S' to be registered i Multiple registration blocks are obtained through clustering;

[0115] Step S32352: Calculate the spatial autocorrelation Δ between the q-th registration block and each of the other registration blocks p. q,p ;

[0116] It should be noted that the above spatial autocorrelation Δ q,p This measures the spatial similarity of a registration block to other registration blocks, where high spatial autocorrelation (i.e., spatial autocorrelation Δ) is a key characteristic. q,pA spatial autocorrelation value greater than the spatial autocorrelation threshold used to measure registration quality indicates that the registration block is similar in spatial distribution to other blocks, which may mean that the registration quality is good; while a low spatial autocorrelation value (i.e., spatial autocorrelation Δ) indicates that the registration block is similar to other blocks in spatial distribution, which may mean that the registration quality is good; while a low spatial autocorrelation value (i.e., spatial autocorrelation Δ) indicates that the registration block is similar in spatial distribution to other blocks, which may mean that the registration quality is good. q,p A value less than or equal to the spatial autocorrelation threshold used to measure the quality of registration indicates that the registration block differs significantly from other blocks, which may suggest a problem with registration in that region.

[0117] Step S32353: Based on the spatial autocorrelation Δ q,p The registration degree is obtained by calculating the number of registration blocks w.

[0118] The registration degree is calculated as follows:

[0119]

[0120] Where, γ w Δ represents the registration degree of the w-th registration block; q,p The expression represents the spatial autocorrelation between the q-th registration block and all other registration blocks p except the q-th registration block; w represents the total number of registration blocks. To calculate the spatial autocorrelation between each pair of these registration blocks, the total number of spatial autocorrelation pairs to be calculated is 0.5ww-1; []* represents a regular expression, [] represents a character class, matching any single character within the square brackets; * is a quantifier, indicating that the character class can appear zero or more times;

[0121] It should be noted that the above-described embodiments of this application reduce the complexity of subsequent calculations by grouping a large amount of point cloud data through clustering. Furthermore, the calculation of spatial autocorrelation provides rich information for each registration block, enabling the analysis of the structural characteristics of different regions in a quantitative manner, thereby better understanding the spatial distribution of point cloud data. Through the feedback of registration accuracy, the scheme can be dynamically adjusted to further optimize the point cloud registration process and thus improve the final registration accuracy.

[0122] The registration error E and the laser point cloud data t are based on j and the image feature points S i The analysis and calculation are performed to obtain a new initial transformation matrix, specifically including the following steps:

[0123] Step S32341: Based on the laser point cloud data t j With the image feature point S i The corresponding point cloud S' to be registered i The combined point pairs are constructed;

[0124] Step S32342: Calculate the centroid μ of the laser point cloud data. t With the point cloud S' to be registered i The center of mass μs ; For the laser point cloud data t mentioned above j Perform a centroid removal operation to obtain the zero-mean laser point cloud y. i Simultaneously, the point cloud S' to be registered... i Perform centroid removal to obtain the zero-mean point cloud to be registered, x. i ′;

[0125] The zero-mean laser point cloud y i The calculation method for ′ is as follows:

[0126] y i ′=t j -μ t ; where μ t The centroid of the laser point cloud data;

[0127] The zero-mean laser point cloud y i The calculation method for ′ is as follows:

[0128] x i ′=s i ′-μ s ; where μ s For the point cloud S' to be registered i The center of mass;

[0129] Step S32343: Based on the zero-mean laser point cloud y i ′ and the zero-mean point cloud to be registered x i Calculate and obtain the covariance matrix H;

[0130] The covariance matrix H is calculated as follows:

[0131]

[0132] Step S32344: Perform singular value decomposition on the covariance matrix H to obtain the orthogonal matrix U and the transpose V of the orthogonal matrix. T ;

[0133] Step S32345: Based on the orthogonal matrix U and the transpose V of the orthogonal matrix T The new rotation matrix R' is calculated.

[0134] The new rotation matrix R' is calculated as follows:

[0135] R = VU T ;

[0136] Step S32346: Based on the centroid μ of the laser point cloud data t With the point cloud S' to be registered i The center of mass μ sThe new rotation matrix R' is used to calculate the new translation vector t';

[0137] The new translation vector t' is calculated as follows:

[0138] t = μ t -R'μ s ;

[0139] Step S32347: Construct a new initial transformation matrix based on the new rotation matrix R' and the new translation vector t'.

[0140] It should be noted that the above-described embodiments of this application effectively integrate laser point cloud and image feature point data by constructing combined point pairs, thereby improving the utilization rate of information during the registration process. Furthermore, the centroid removal operation eliminates the influence of translation, allowing subsequent calculations to focus more on the relative positions of the point clouds, improving the accuracy of registration. At the same time, the calculation of the covariance matrix provides a quantitative description of the relationship between the point clouds, providing important information for solving the rotation matrix. In a further operation, through singular value decomposition and the calculation of the rotation matrix, the scheme can accurately capture the spatial relationship between the point clouds, thereby generating a new translation vector and ensuring the precise alignment of the laser point cloud and the point cloud to be registered.

[0141] The above-described embodiments of this application transform the complex point cloud registration problem into a series of easily processed steps using linear algebra, which can effectively improve computational efficiency and stability.

[0142] In summary, the present invention presents a high-precision substation mapping and identification method using a handheld LiDAR scanner. This method utilizes a handheld rotating LiDAR scanner for high-precision mapping of the surrounding environment and the substation. The handheld rotating LiDAR device enables rapid and accurate 3D scanning of the substation, generating detailed terrain and structural data. In particular, it accurately identifies specific substation equipment by combining image recognition and point cloud analysis. The point cloud generates the morphological structure of the substation scene, and details are generated from RGBD images. The morphological structure and details are then fused to obtain a detailed morphological structure model of the substation scene. Finally, analysis of the initial substation scene model can detect detailed defects in the substation, thus enabling better measurement and maintenance. Furthermore, analysis of the mapped substation scene model allows for the analysis of the state changes of various equipment within the substation, such as corrosion, cracks, deformation, or damage, which is helpful for the planning, construction, and maintenance of the substation.

[0143] In the specific operation, three registration steps are taken. First, image feature descriptors and point cloud feature descriptors are extracted from image feature points and point cloud feature points, respectively. Then, the first registration, i.e. coarse registration, is performed based on the KD tree constructed from the point cloud feature descriptors. This aims to filter out unreliable fuzzy matches and retain more reliable matching pairs, providing a good starting point for subsequent fine registration. At the same time, it effectively reduces the complexity of data processing. By matching excellent feature point pairs, the search space in the subsequent registration process is reduced, improving the efficiency and accuracy of registration.

[0144] The first registration point group is further processed by the RANSAC algorithm, and random sampling and consistency checks are performed to estimate the initial transformation matrix. The generated initial transformation matrix (4×4 homogeneous transformation matrix) can accurately map the 3D information in the RGBD image to the point cloud coordinate system, which significantly improves the convergence speed and accuracy of subsequent fine registration algorithms (such as ICP).

[0145] Finally, based on the initial transformation matrix, the ICP algorithm is used for iterative processing to finely align the registration feature points, further improving the registration accuracy.

[0146] In the specific iterative process of executing the ICP algorithm, the image feature points are first rotated and translated using the initial transformation matrix to obtain the point cloud data to be registered, ensuring that the two are compared and matched in the same coordinate system. Furthermore, by calculating the registration error E between the point cloud to be registered and the laser point cloud, the current registration effect can be quantified, helping to determine whether to continue iterating or end the registration process. In addition, by continuously evaluating the error and the number of iterations, the registration strategy is flexibly adjusted to ensure that the best registration effect is achieved within a reasonable time.

[0147] Finally, the registration process is optimized by incorporating spatial autocorrelation calculations to determine the registration accuracy, thereby verifying the aforementioned group of registration feature points to be determined and ensuring more accurate analysis and acquisition of the target registration feature point group.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; those skilled in the art can modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-precision mapping and identification method for substations using a handheld lidar, characterized in that, The following steps are included: The handheld LiDAR device is used to acquire RGB image data of the substation; simultaneously, the handheld LiDAR device is used to acquire LiDAR point cloud data of the substation. An RGBD image is obtained by calculating the RGB image data using the camera parameters of the image acquisition device; Image registration is performed based on the RGBD image and the laser point cloud data to obtain a substation scene model; Based on the substation scenario model, the equipment status of the substation is obtained through analysis. The step of performing image registration based on the RGBD image and the laser point cloud data to obtain the substation scene model includes the following steps: Extract image feature points from the RGBD image, and simultaneously extract point cloud feature points from the laser point cloud data; A registered feature point group is obtained by registering the image feature points with the point cloud feature points. Based on the registration feature point group, the RGBD image and the laser point cloud data are registered to obtain a substation scene model; The step of registering the image feature points with the point cloud feature points to obtain a registered feature point group includes the following steps: Image feature descriptors are obtained based on the image feature points; point cloud feature descriptors are obtained based on the point cloud feature points; a first registration point pair is obtained by searching the KD tree constructed from the point cloud feature descriptors based on the image feature descriptors; and then a first registration point group is obtained. The initial transformation matrix is ​​obtained by estimating the first registration point group through RANSAC coarse registration processing; Based on the initial transformation matrix, iterative processing and analysis are performed through ICP fine registration to obtain a registration feature point group. The step of iteratively processing and analyzing the initial transformation matrix through ICP fine registration to obtain the registration feature point group includes the following steps: Initialize the iteration conditions; the iteration conditions include an iteration counter, a maximum number of iterations, and a minimum error threshold; the number of iterations of the counter is initially set to 0; Iterative processing operation: Based on the initial transformation matrix, for each of the image feature points The point cloud to be registered is obtained through calculation and processing. ; Increment the iteration count of the iteration counter by 1 to obtain the current iteration count, and simultaneously, based on the point cloud to be registered... With the point cloud to be matched The registration error E is calculated using the laser point cloud data tj with the shortest distance. Based on the registration error E and the minimum error threshold, a group of registration feature points to be determined is iteratively output. The registration degree is calculated based on the set of registration feature points to be determined; the target registration feature point set is then iteratively output based on the registration degree and a preset registration degree threshold. The step of calculating the registration degree based on the set of registration feature points to be determined includes the following steps: Based on multiple laser point cloud data tj from the multiple groups of feature points to be registered and the point cloud to be registered Multiple registration blocks are obtained through clustering; The calculation yields the q-th registration block and all other registration blocks except the q-th registration block. Spatial autocorrelation ; Based on the spatial autocorrelation The registration degree is obtained by calculating the number of registration blocks w. The registration degree is calculated as follows: ； in, Indicates the first Registration accuracy of each registration block; This represents the q-th registration block and all other registration blocks except the q-th registration block. Spatial autocorrelation; The total number of registration blocks is indicated; []* represents a regular expression, [] represents a character class, matching any single character within the square brackets; * is a quantifier, indicating that the character class appears zero or more times.

2. The high-precision mapping and identification method for substations using a handheld lidar as described in claim 1, characterized in that, The process of searching and constructing the first registration point pair based on the image feature descriptor in the KD tree constructed from the point cloud feature descriptor specifically includes: The image feature descriptor is used to search for K nearest neighbors in the KD tree to obtain K first registration point cloud feature points corresponding to each image feature point; A first registration point pair is constructed based on the image feature points and the K first registration point cloud feature points.

3. The high-precision mapping and identification method for substations using a handheld lidar as described in claim 2, characterized in that, Each element in the initial transformation matrix represents a second registration point group; the second registration point group represents a combination of image feature points and point cloud feature points with higher accuracy obtained after re-registration of the first registration point group.

4. The high-precision mapping and identification method for substations using a handheld lidar as described in claim 3, characterized in that, The step of judging and iteratively outputting the registration feature point group to be determined based on the registration error E and the minimum error threshold specifically includes the following steps: Determine whether the registration error E is less than or equal to the minimum error threshold; if so, output the image feature points. With the laser point cloud data t j The system constructs a group of feature points to be registered; if not, it determines whether the current iteration number is greater than or equal to the maximum iteration number; if so, it outputs the image feature points. The laser point cloud data tj forms a group of registration feature points to be determined; otherwise, the registration error E is used in conjunction with the laser point cloud data tj and the image feature points. A new initial transformation matrix is ​​obtained through analysis and calculation. Based on the new initial transformation matrix, the above iterative processing operation is returned until the set of registration feature points to be determined is output.

5. A high-precision mapping and identification method for substations using a handheld lidar as described in claim 4, characterized in that, The process of calculating the registration degree based on the set of registration feature points to be determined, and then iteratively outputting the target registration feature point set based on the registration degree and a preset registration degree threshold, specifically includes the following steps: The registration degree is calculated based on the group of registration feature points to be determined; it is determined whether the registration degree is greater than or equal to a preset registration degree threshold. If so, the group of registration feature points to be determined is output as the target registration feature point group. If not, a new initial transformation matrix is ​​calculated using the image feature points and point cloud feature points corresponding to the group of registration feature points to be determined, and the above iterative processing operation is returned until the registration degree is greater than or equal to the preset registration degree threshold, and the target registration feature point group is output.

6. A high-precision mapping and identification method for substations using a handheld lidar as described in claim 5, characterized in that, The point cloud to be registered The calculation method is as follows: ； In the formula, This is the rotation matrix in the initial transformation matrix; The translation vector in the initial transformation matrix; The registration error E is calculated as follows: E = (1 / m) * Σ(min‖ - tj‖²); In the formula, m is the number of image feature points; min‖ - tj‖ indicates the point cloud to be registered With the point cloud to be matched The Euclidean distance between the shortest distance laser point cloud data tj.

Images