Transformer substation high-precision surveying and mapping identification method of handheld laser radar

Abstract

The invention discloses a transformer substation high-precision surveying and mapping identification method based on a handheld laser radar, and the method comprises the steps: carrying out the high-precision surveying and mapping of a surrounding environment and a transformer substation through a handheld rotary laser radar scanner, carrying out the quick and precise three-dimensional scanning of the transformer substation through handheld rotary laser radar equipment, generating detailed terrain and structure data, and carrying out the high-precision surveying and mapping of the transformer substation. The method comprises the following steps: jointly and accurately identifying specific substation equipment by using image identification and point cloud, generating a morphological structure of a substation scene through the point cloud, generating details through RGBD images, fusing the morphological structure and the details to obtain a detailed morphological structure model of the substation scene, and finally analyzing through an initial substation scene model to obtain a detailed morphological structure model of the substation scene. Detail defects in the transformer substation can be detected, so that measurement and maintenance can be better carried out; by analyzing the surveying and mapping substation scene model, the state change, such as corrosion, crack, deformation or damage of the equipment, of each piece of equipment in the substation can be analyzed, and planning construction and maintenance work of the substation are facilitated.

Description

Technical Field

[0001] The present invention relates to the field of surveying and mapping, and in particular to a high-precision surveying and identifying method for a substation using a handheld laser radar. Background Art

[0002] The surveying and mapping of substations is crucial to identifying the status of substation equipment, such as corrosion, cracks, deformation and damage of substation equipment. In the monitoring and maintenance of modern substations, traditional surveying and mapping methods often rely on manual inspections and static image acquisition. It is further found that static images can only capture images of substation scenes and cannot reflect the actual status of substation equipment. They lack depth information and spatial location information, and thus cannot accurately identify the deformation status of equipment in substation scenes.

[0003] Therefore, it is necessary to extract the depth information of the image, use handheld devices to perform image (specifically depth image) and laser dual acquisition to achieve modeling and recognition to detect abnormalities on the surface of power equipment; in the process of building the model, it is very difficult to accurately realize the registration of laser point cloud data and image data (specifically RGBD image); the process of converting data obtained from different sources, at different times or from different sensors to the same coordinate system. In substation equipment detection, it mainly involves the registration of laser point cloud data and RGBD image data. The main purpose of this process is to accurately match the two types of data for subsequent 3D modeling and abnormality detection.

[0004] In summary, how to realize the registration and modeling of RGBD images and laser point cloud data on handheld devices will ultimately have an important impact on the evaluation and detection of abnormal device status. Summary of the invention

[0005] The purpose of the present invention is to provide a high-precision substation mapping and identification method using a handheld laser radar, which solves the above-mentioned technical problems pointed out in the prior art.

[0006] The present invention provides a high-precision mapping and identification method for a substation using a handheld laser radar, comprising the following steps:

[0007] The RGB image data of the substation is acquired through the image acquisition device carried by the handheld laser radar device; at the same time, the laser point cloud data of the substation is acquired through the laser radar device carried by the handheld laser radar device;

[0008] Based on the RGB image data, a RGBD image is obtained by calculating the camera parameters of the image acquisition device;

[0009] Performing image registration based on the RGBD image and the laser point cloud data to obtain a substation scene model;

[0010] An analysis is performed based on the substation scenario model to obtain the equipment status of the substation.

[0011] Preferably, performing image registration based on the RGBD image and the laser point cloud data to obtain a substation scene model comprises the following steps:

[0012] Extracting image feature points from the RGBD image and simultaneously extracting point cloud feature points from the laser point cloud data;

[0013] Acquire a registration feature point group based on registering the image feature points with the point cloud feature points;

[0014] The RGBD image and the laser point cloud data are registered based on the registration feature point group to obtain a substation scene model.

[0015] Preferably, the obtaining a registration feature point group based on the registration of the image feature points and the point cloud feature points comprises the following steps:

[0016] An image feature descriptor is obtained based on the image feature point analysis; a point cloud feature descriptor is obtained based on the point cloud feature point analysis; a first registration point pair is obtained by searching and constructing a KD tree constructed based on the image feature descriptor;

[0017] The first registration point group is estimated by RANSAC coarse registration processing to obtain an initial transformation matrix;

[0018] Based on the initial transformation matrix, an iterative processing analysis is performed through ICP fine registration to obtain a registration feature point group.

[0019] Preferably, the step of searching and constructing a first registration point pair in a KD tree constructed based on the image feature descriptor in the point cloud feature descriptor specifically includes:

[0020] Using the image feature descriptor to search for K nearest neighbors in the KD tree, to obtain K first registration point cloud feature points corresponding to each of the image feature points;

[0021] A first registration point pair is constructed based on the image feature point and the k first registration point cloud feature points.

[0022] Preferably, each element in the initial transformation matrix represents a second registration point group; the second registration point group represents a combination of image feature points and point cloud feature points with higher accuracy obtained by screening the first registration point group through re-registration.

[0023] Preferably, the iterative processing and analysis based on the initial transformation matrix through ICP fine registration to obtain the registration feature point group includes the following steps:

[0024] Initialize iteration conditions; the iteration conditions include an iteration counter, a maximum number of iterations, and a minimum error threshold; the number of iterations of the counter is initially 0;

[0025] Iterative processing operation: Based on the initial transformation matrix, each of the image feature points S i Perform calculations to obtain the point cloud S' to be registered i ;

[0026] The iteration number of the iteration counter is increased by 1 to obtain the current iteration number, and based on the point cloud to be registered S' i and the point cloud S' to be registered i Laser point cloud data with the shortest distance t j Calculate and obtain the registration error E;

[0027] Based on the registration error E and the minimum error threshold, a judgment is made and a group of registration feature points to be determined is iteratively outputted;

[0028] A registration degree is calculated based on the to-be-determined registration feature point group; and a target registration feature point group is iteratively output based on the registration degree and a preset registration degree threshold.

[0029] Preferably, the step of judging based on the registration error E and the minimum error threshold and iteratively outputting the to-be-determined registration feature point group specifically includes the following steps:

[0030] Determine whether the registration error E is less than or equal to the minimum error threshold, if so, output the image feature point S i With the laser point cloud data j Constitute a group of registration feature points to be determined; if not, determine whether the current number of iterations is greater than or equal to the maximum number of iterations; if so, output the image feature point S i With the laser point cloud data j If not, then based on the registration error E and the laser point cloud data t j And the image feature point S i Perform analysis and calculation to obtain a new initial transformation matrix, and return to the above iterative processing operation based on the new initial transformation matrix until a group of registration feature points to be determined is output;

[0031] Preferably, the step of calculating the registration degree based on the to-be-determined registration feature point group; and iteratively outputting the target registration feature point group based on the registration degree and a preset registration degree threshold value specifically includes the following steps:

[0032] A registration degree is calculated based on the to-be-determined registration feature point group; whether the registration degree is greater than or equal to a preset registration degree threshold is determined, and if so, the to-be-determined registration feature point group is output as a target registration feature point group; if not, a new initial transformation matrix is ​​calculated using the image feature points and point cloud feature points corresponding to the to-be-determined registration feature point group, and the above-mentioned iterative processing operation is returned until the registration degree is greater than or equal to the preset registration degree threshold, and the target registration feature point group is output.

[0033] Preferably, the point cloud to be registered S' i The calculation method is:

[0034] s′ i =R0s i +t0;

[0035] In the formula, R0 is the rotation matrix in the initial transformation matrix; t0 is the translation vector in the initial transformation matrix;

[0036] The registration error E is calculated as follows:

[0037] E=(1 / m)*Σ(min‖s' j -t j ‖ 2 );

[0038] Where m is the number of image feature points; min‖s' i -t j ‖ represents the point cloud S' to be registered i And the point cloud S' to be registered i The Euclidean distance between the laser point cloud data tj with the shortest distance.

[0039] Preferably, the step of calculating the registration degree based on the to-be-determined registration feature point group comprises the following steps:

[0040] Based on the plurality of laser point cloud data tj in the plurality of registration feature point groups to be determined and the point cloud S' to be registered i Multiple registration blocks are obtained through clustering;

[0041] The spatial autocorrelation Δ between the qth registration block and each registration block p except the qth registration block is calculated. q,p ;

[0042] Based on the spatial autocorrelation Δ q,p And the number of the registration blocks w is calculated to obtain the registration degree;

[0043] The registration degree is calculated as follows:

[0044]

[0045] Among them, γ w represents the registration degree of the wth registration block; Δ q,p represents the spatial autocorrelation between the qth registration block and each registration block p except the qth registration block; w represents the total number of registration blocks, and the total number of spatial autocorrelations that need to be calculated to calculate the spatial autocorrelations of these registration blocks is 0.5ww-1; []* represents a regular expression, [] represents a character class, matching any single character in the square brackets; * is a quantifier, indicating that the character class can appear zero or more times.

[0046] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0047] From the analysis of the above-mentioned high-precision mapping and identification method of substation using a handheld laser radar provided by the present invention, it can be known that in specific applications, a handheld rotating laser radar scanner is used to perform high-precision mapping of the surrounding environment and the substation. Through the handheld rotating laser radar device, the substation can be quickly and accurately scanned in three dimensions to generate detailed terrain and structural data, especially by using image recognition and point cloud to accurately identify specific substation equipment, and the morphological structure of the substation scene is generated through the point cloud. The RGBD image containing RGB information and depth information is generated by the RGB image data and camera parameters to generate details, and then the morphological structure is fused with the details to obtain a detailed morphological structure model of the substation scene. Finally, the initial substation scene model is analyzed to detect detailed defects in the substation, so that measurement and maintenance can be better performed. By analyzing the surveyed substation scene model, the status changes of each equipment in the substation can be analyzed, such as corrosion, cracks, deformation or damage of the equipment, which is helpful for the planning, construction and maintenance of the substation. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the main process of a high-precision mapping and identification method for a substation using a handheld laser radar;

[0049] Figure 2 A schematic diagram of a substation scene point cloud simulation in a high-precision substation mapping and identification method using a handheld laser radar;

[0050] Figure 3 A schematic diagram of the operation steps of registering a substation scene model in a high-precision mapping and identification method of a substation using a handheld laser radar;

[0051] Figure 4 A schematic diagram of the KD tree in a high-precision mapping and identification method for a substation using a handheld laser radar;

[0052] Figure 5 The present invention is a schematic diagram of the operation flow of iterative processing to obtain a registration feature point group in a high-precision mapping and identification method for a substation using a handheld laser radar. DETAILED DESCRIPTION

[0053] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] The present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0055] like Figure 1 As shown, the embodiment of the present invention provides a high-precision mapping and identification method for a substation using a handheld laser radar, comprising the following operating steps:

[0056] Step S10: acquiring RGB image data of the substation through the image acquisition device carried by the handheld laser radar device; and acquiring laser point cloud data of the substation through the laser radar device carried by the handheld laser radar device;

[0057] Step S20: Calculating an RGBD image based on the RGB image data using the camera parameters of the image acquisition device;

[0058] It should be noted that in the above-mentioned embodiment of the present application, the above-mentioned RGBD image is a combination of an RGB image and an RGBD image, which contains 4 channels: red, green, blue and depth; wherein, for the depth channel, each pixel value represents the distance from the corresponding point in the scene to the camera, and the RGBD image can be regarded as an organized point cloud, and each pixel can be converted into a point in 3D space. However, due to problems such as the image acquisition device on the handheld laser radar device and the acquisition angle of the laser radar device, the above-mentioned RGBD image cannot be directly converted into laser point cloud data, but needs to be registered with the laser point cloud data through the RGBD image, so as to achieve a complete match between the RGB information of the objects in the substation scene and the laser point cloud data, thereby improving the accuracy of the obtained substation scene model, thereby improving the accuracy of the subsequent analysis of the equipment status of the substation; through the calculation of the above-mentioned embodiment of the present application, richer visual and spatial information can be provided, thereby improving the efficiency of the subsequent generation of the substation scene model.

[0059] Step S30: performing image registration based on the RGBD image and the laser point cloud data to obtain a substation scene model;

[0060] It should be noted that the above-mentioned substation scene model is a scene model that reflects the details and morphology of the substation. The above-mentioned embodiment of the present application generates the morphological structure of the substation scene from the point cloud, generates the details from the RGBD image, and then fuses the morphological structure with the details to obtain the detailed morphological structure model of the substation scene (i.e., the above-mentioned substation scene model), and the scene is fine and complete.

[0061] Step S40: Analyze based on the substation scenario model to obtain the equipment status of the substation;

[0062] It should be noted that the above analysis specifically refers to the use of high-resolution visual information from the model to detect anomalies on the surface of the equipment, such as corrosion and cracks; by comparing the actual 3D shape of the equipment with the standard shape, deformation or damage can be identified; the relative position and distance between the equipment can be checked to ensure compliance with safety standards;

[0063] The above-mentioned embodiment of the present application uses a handheld rotating laser radar scanner to perform high-precision mapping of the surrounding environment and the substation. Through the handheld rotating laser radar device, the substation can be quickly and accurately scanned in three dimensions (such as Figure 2 As shown in the figure, detailed terrain and structure data are generated, especially by using image recognition and point cloud to accurately identify specific substation equipment; the designed method can improve the efficiency and accuracy of surveying and mapping, especially in complex substation environments, and can provide more accurate measurement results, which is helpful for the planning, construction and maintenance of substations; by analyzing the substation scene model currently surveyed, the status changes of various equipment in the substation can be analyzed, such as corrosion, cracks, deformation or damage of the equipment;

[0064] The above-mentioned embodiment of the present application generates the morphological structure of the substation scene from the point cloud, generates the details from the RGBD image, and then fuses the morphological structure with the details to obtain a detailed morphological structure model of the substation scene (i.e., the above-mentioned substation scene model). Finally, the initial substation scene model is used for analysis to detect detail defects in the substation, thereby enabling better measurement and maintenance.

[0065] Specifically, Figure 3 As shown, in step S30, image registration is performed based on the RGBD image and the laser point cloud data to obtain a substation scene model, including the following operation steps:

[0066] Step S31: extracting image feature points from the RGBD image, and simultaneously extracting point cloud feature points from the laser point cloud data;

[0067] Step S32: performing registration based on the image feature points and the point cloud feature points to obtain a registration feature point group;

[0068] Step S33: based on the registration feature point group, the RGBD image and the laser point cloud data are registered to obtain a substation scene model.

[0069] It should be noted that after obtaining the registration feature point group as described above, the RGBD image pixel point coordinates corresponding to the registration feature point group and the laser point cloud data coordinates can be registered by triangulation to obtain the substation scene model; specifically, after the image feature points and the point cloud feature points establish a registration relationship, the RGBD image pixel point coordinates of the RGBD image corresponding to the image feature points and the laser point cloud data coordinates of the laser point cloud data corresponding to the point cloud feature points also establish a registration relationship; further, through the registration relationship (that is, each RGBD image pixel point coordinate is registered to obtain the corresponding laser point cloud data coordinate), the RGBD image pixel point coordinates corresponding to the registration result and the laser point cloud data coordinates are mapped to a unified coordinate system (usually the coordinate system of the laser point cloud data) to achieve image registration and obtain the substation scene model;

[0070] The above-mentioned embodiment of the present application first extracts image feature points in the RGBD image, and simultaneously extracts point cloud feature points in the laser point cloud data. By extracting the feature points, the complexity of data processing can be reduced, thereby reducing the number of points that need to be aligned and improving the efficiency of alignment; further, alignment is performed based on the extracted image feature points and point cloud feature points to obtain a group of aligned feature points, so that data from different sources (RGBD image and laser point cloud) can be aligned for subsequent processing; finally, through the triangulation method, the coordinates of each RGBD image pixel point are corresponded to the coordinates of the laser point cloud data, thereby achieving data unification, which can not only enhance the spatial relationship of the data, but also improve the accuracy and reliability of the finally generated scene model.

[0071] Specifically, in step S32, registering the image feature points with the point cloud feature points to obtain a registration feature point group includes the following steps:

[0072] Step S321: obtaining an image feature descriptor based on the image feature point analysis; obtaining a point cloud feature descriptor based on the point cloud feature point analysis; and obtaining a first registration point pair based on the image feature descriptor by searching and constructing a KD tree constructed by the point cloud feature descriptor.

[0073] Specifically, the above-mentioned embodiment of the present application constructs a KD tree based on the point cloud feature descriptor; uses the image feature descriptor to search for K nearest neighbors in the KD tree to obtain K first registration point cloud feature points corresponding to each of the image feature points; and constructs a first registration point pair based on the image feature point and the k first registration point cloud feature points;

[0074] It should be noted that the above embodiment of the present application is a coarse registration operation (first registration operation), which is intended to filter the fuzzy matching options and retain K point cloud feature points corresponding to some more reliable image feature points;

[0075] The above point cloud feature descriptor is obtained by analyzing the point cloud feature points using the FPFH fast point feature histogram (a three-dimensional feature descriptor based on point cloud, mainly used for point cloud feature extraction and registration, and often used for coarse registration of point cloud);

[0076] For example, Figure 4 As shown: There are currently the following simplified 3D point cloud feature descriptors: A(2, 3, 7), B(4, 7, 9), C(5, 4, 5), D(7, 2, 11), E(9, 6, 2), F(1, 8, 8), such as Figure 4 In the example, the first layer is divided along the x-axis (7 is the dividing point); the second layer is divided along the y-axis; the third layer is divided along the z-axis; then the cycle returns to the x-axis, and so on;

[0077] The above-mentioned image feature descriptors are generated by processing image feature points using algorithms such as SIFT or SURF;

[0078] The KD tree in the above-mentioned embodiment of the present application is a KD tree of the point cloud feature descriptor constructed by the FLANN library, and then based on each image feature descriptor, the Lower's algorithm is used (the Lower's algorithm is to further screen the matching points to obtain excellent matching points, which is the so-called "refining the coarse", and the Lower's algorithm is generally used to further obtain excellent matching points) to search for K first registration point cloud feature points, filter out fuzzy matches, and retain only the most reliable matching pairs; this provides a good starting point for subsequent fine registration and greatly reduces the search space;

[0079] For example: there is currently an image feature descriptor p1, and its two nearest neighbors found in the point cloud feature descriptor are q1 and q2, with distances d1 and d2 respectively;

[0080] If d1 / d2<0.8 (the threshold is usually between 0.7-0.8), (p1, q1) is considered a good matching pair;

[0081] Finally, a set of initial matching pairs such as {(p1, q1), (p2, q2), ..., (pn, qn)} is obtained, which provides a good starting point for subsequent steps.

[0082] Step S322: the first registration point group is estimated by RANSAC rough registration processing to obtain an initial transformation matrix; each element in the initial transformation matrix represents a second registration point group; the second registration point group represents a combination of image feature points and point cloud feature points with higher accuracy obtained by screening the first registration point group after re-registration (re-registration is RANSAC rough registration, i.e., the second registration operation);

[0083] It should be noted that the RANSAC coarse registration in the above-mentioned embodiment of the present application is to find a reliable transformation in the presence of a large number of outliers (wrong matches) through random sampling and consistency check processing, which can effectively remove the wrong correspondences in the initial match (i.e., the first registration point group), and provide a good initial estimate for the subsequent ICP algorithm, greatly improving the convergence speed and accuracy of ICP;

[0084] The above initial transformation matrix is ​​in the registration process of the substation scene model. Since the embodiment of the present application processes data in 3D space, the initial transformation matrix estimated by RANSAC coarse registration is a 4×4 homogeneous transformation matrix, which describes the rigid body transformation (rotation and translation) in 3D space; through the initial transformation matrix, the 3D information in the RGBD image can be accurately mapped to the point cloud coordinate system, which provides a good initial estimate for the subsequent ICP fine registration and improves the efficiency and accuracy of the registration;

[0085] The above initial transformation matrix can be expressed as:

[0086] Among them, R is a 3×3 rotation matrix, describing the rotation transformation from the RGBD image coordinate system to the point cloud coordinate system; t is a 3×1 translation vector, describing the translation from the origin of the RGBD image coordinate system to the origin of the point cloud coordinate system; 0 is a 1×3 zero vector; 1 is a scalar 1; the above 0 and 1 ensure that the matrix can be used for homogeneous coordinate transformation;

[0087] Specifically, the estimation in the above embodiment of the present application refers to converting the above first registration point pair into an array expression, and then randomly selecting 3 pairs of matching points (minimum set) therefrom; further, calculating the rigid body transformation defined by the 3 pairs of points; then applying the transformation to all image feature points, and calculating the number of inliers; finally repeating the above RANSAC iteration operation N times (N is usually 1000-10000), selecting the transformation with the most inliers, and obtaining the initial transformation matrix;

[0088] For example: there are currently 100 pairs of first registration point pairs. After RANSAC iterates 1000 times, it finds the best transformation T. Under this transformation, 80 pairs of matches have errors less than a threshold (such as 1 cm). These 80 pairs are considered to be inliers. Then, a higher quality matching pair (a higher quality matching pair or a second registration point pair) corresponding to an initial transformation matrix T that is more reliable than the numerous first registration point pairs is obtained from these 80 pairs of inliers.

[0089] Step S323: performing iterative processing and analysis through ICP fine registration based on the initial transformation matrix to obtain a registration feature point group;

[0090] It should be noted that the above ICP fine registration is a point cloud fine registration algorithm (i.e., the third registration operation), which can finely align two or more point cloud data sets for subsequent 3D reconstruction, topology analysis and other operations;

[0091] The above-mentioned embodiment of the present application performs three registrations, first extracting image feature descriptors and point cloud feature descriptors from image feature points and point cloud feature points respectively, and then performing the first registration based on the KD tree constructed based on the point cloud feature descriptors, i.e., a coarse registration operation, which aims to filter out unreliable fuzzy matches and retain relatively reliable matching pairs, thereby providing a good starting point for subsequent fine registration. At the same time, it effectively reduces the complexity of data processing, reduces the search space in the subsequent registration process by matching excellent feature point pairs, and improves the efficiency and accuracy of registration.

[0092] The first registration point group is further processed by the RANSAC algorithm, random sampling and consistency checking are performed to estimate the initial transformation matrix. The generated initial transformation matrix (4×4 homogeneous transformation matrix) can accurately map the 3D information in the RGBD image to the point cloud coordinate system, significantly improving the convergence speed and accuracy of subsequent fine registration algorithms (such as ICP).

[0093] Finally, based on the initial transformation matrix, the ICP algorithm is iteratively processed to finely align the registration feature points, further improving the accuracy of the registration.

[0094] During the specific implementation of the above-mentioned embodiment of the present application, the technicians found that in the ICP fine registration process, since the RGBD image and the laser point cloud data may contain multiple identical transformers at the same time, at this time, directly using the traditional IPC fine registration algorithm for registration operations will result in the RGBD image of a certain transformer and the laser point cloud data. The registration error will cause errors in the overall RGBD image and the laser point cloud data; therefore, in the specific execution of the ICP fine registration operation, it is necessary to perform multiple iterative judgment and analysis to ensure that the RGBD image of each transformer is correctly registered with the laser point cloud data, so that the final overall RGBD image and the laser point cloud data are accurately configured. For details, see steps S3231-S3235 in the subsequent implementation method.

[0095] Specifically, Figure 5 As shown, in step S323, an iterative processing analysis is performed based on the initial transformation matrix through ICP fine registration to obtain a registration feature point group, including the following operation steps:

[0096] Step S3231: Initialize iteration conditions; the iteration conditions include an iteration counter, a maximum number of iterations, and a minimum error threshold; the number of iterations of the counter is initially 0;

[0097] Step S3232: Iterative processing operation: based on the initial transformation matrix, each of the image feature points S i Perform calculations to obtain the point cloud S' to be registered i ;

[0098] The point cloud S' to be registered i The calculation method is:

[0099] s′ i =R0s i +t0;

[0100] In the formula, R0 is the rotation matrix in the initial transformation matrix; t0 is the translation vector in the initial transformation matrix;

[0101] The above embodiment of the present application rotates and translates the pixel points of the RGBD image corresponding to the image feature points through the initial transformation matrix, thereby obtaining the point cloud S' to be registered corresponding to the image feature points in the laser point cloud coordinate system. i ;

[0102] Step S3233: Add 1 to the iteration number of the iteration counter to obtain the current iteration number, and based on the point cloud S' to be registered i and the point cloud S' to be registered i Laser point cloud data with the shortest distance j Calculate and obtain the registration error E;

[0103] The registration error E is calculated as follows:

[0104] E=(1 / m)*Σ(min‖s' i -t j ‖ 2 );

[0105] Where m is the number of image feature points; min‖s' i -t j ‖ represents the point cloud S' to be registered i And the point cloud S' to be registered i Laser point cloud data with the shortest distance t j The Euclidean distance between

[0106] It should be noted that in the above-mentioned embodiment of the present application, the registration error reflects the average distance difference between the two point clouds, and a smaller error indicates a better registration effect;

[0107] Step S3234: judging based on the registration error E and the minimum error threshold, and iteratively outputting a group of registration feature points to be determined;

[0108] Specifically, judging based on the registration error E and the minimum error threshold and iteratively outputting the to-be-determined registration feature point group refers to: judging whether the registration error E is less than or equal to the minimum error threshold, and if so, outputting the image feature point S i With the laser point cloud data j Constitute a group of registration feature points to be determined; if not, determine whether the current number of iterations is greater than or equal to the maximum number of iterations; if so, output the image feature point S i With the laser point cloud data j If not, then based on the registration error E and the laser point cloud data t j And the image feature point S i Perform analysis and calculation to obtain a new initial transformation matrix, and return to the above operation (i.e., the above iterative processing operation) based on the new initial transformation matrix until a group of registration feature points to be determined is output;

[0109] Step S3235: calculating the registration degree based on the to-be-determined registration feature point group; iteratively judging and outputting the target registration feature point group based on the registration degree and a preset registration degree threshold;

[0110] Specifically, iteratively outputting the target registration feature point group based on the determination of the registration degree and the preset registration degree threshold value refers to: determining whether the registration degree is greater than or equal to the preset registration degree threshold value, and if so, outputting the to-be-determined registration feature point group as the target registration feature point group; if not, calculating a new initial transformation matrix using the image feature points and point cloud feature points corresponding to the to-be-determined registration feature point group (i.e., calculating a new initial transformation matrix according to the above step S3235, that is, using the laser point cloud data t corresponding to the to-be-determined registration feature point group j And the image feature point S i Perform calculation and analysis to obtain a new initial transformation matrix), and return to the above operation (ie, the above iterative processing operation) until the registration degree is greater than or equal to a preset registration degree threshold, and output the target registration feature point group.

[0111] It should be noted that the above-mentioned embodiment of the present application firstly uses the initial transformation matrix to rotate and translate the image feature points to obtain the point cloud data to be registered, ensures that the two are compared and matched in the same coordinate system, and further calculates the registration error E between the point cloud to be registered and the laser point cloud, so as to quantify the effect of the current registration and help judge whether it is necessary to continue iterating or end the registration process; further, by continuously evaluating the error and the number of iterations, the registration strategy is flexibly adjusted to ensure that the best registration effect is achieved within a reasonable time;

[0112] Finally, the above-mentioned registration feature point group to be determined is verified by calculating the registration degree, thereby ensuring that the analysis and acquisition of the target registration feature point group is more accurate.

[0113] Specifically, in step S3235, calculating the registration degree based on the to-be-determined registration feature point group includes the following operation steps:

[0114] Step S32351: Based on the plurality of laser point cloud data in the plurality of registration feature point groups to be determined, j And the point cloud to be registered S' i Multiple registration blocks are obtained through clustering;

[0115] Step S32352: Calculate the spatial autocorrelation Δ between the qth registration block and each registration block p except the qth registration block q,p ;

[0116] It should be noted that the above spatial autocorrelation Δ q,p It measures the spatial similarity of a registration block to other registration blocks, where high spatial autocorrelation (i.e. spatial autocorrelation Δ q,pGreater than the spatial autocorrelation threshold used to measure the registration quality) indicates that the registration block is similar to other blocks in spatial distribution, which may mean good registration quality; while low spatial autocorrelation (i.e. spatial autocorrelation Δ q,p Less than or equal to the spatial autocorrelation threshold used to measure the quality of registration) indicates that the registered block is very different from other blocks, which may suggest that there is a problem with the registration in this area.

[0117] Step S32353: Based on the spatial autocorrelation Δ q,p And the number of the registration blocks w is calculated to obtain the registration degree;

[0118] The registration degree is calculated as:

[0119]

[0120] Among them, γ w represents the registration degree of the wth registration block; Δ q,p represents the spatial autocorrelation between the qth registration block and each registration block p except the qth registration block; w represents the total number of registration blocks. To calculate the spatial autocorrelation of these registration blocks, the total number of spatial autocorrelations that need to be calculated is 0.5ww-1; []* represents a regular expression, [] represents a character class, matching any single character in the square brackets; * is a quantifier, indicating that the character class can appear zero or more times;

[0121] It should be noted that the above-mentioned embodiment of the present application reduces the complexity of subsequent calculations by grouping a large amount of point cloud data through clustering. Furthermore, the calculation of spatial autocorrelation provides rich information for each registration block, making it possible to analyze the structural characteristics of different regions in a quantitative manner, thereby better understanding the spatial distribution of point cloud data, and through the feedback of the registration degree, the scheme can realize dynamic adjustment and further optimize the point cloud registration process, thereby improving the final registration accuracy.

[0122] The method based on the registration error E and the laser point cloud data t j And the image feature point S i Perform analysis and calculation to obtain a new initial transformation matrix, which specifically includes the following steps:

[0123] Step S32341: Based on the laser point cloud data t j With the image feature point S i The corresponding point cloud to be registered S' i Construct a combined point pair;

[0124] Step S32342: Calculate the centroid μ of the laser point cloud data t And the point cloud S' to be registered i The center of mass μs ; For the laser point cloud data t j Perform the centroid removal operation to obtain the zero-mean laser point cloud y i '; At the same time, the point cloud to be registered S' i Perform the centroid removal operation to obtain the zero-mean point cloud x to be registered i ′;

[0125] The zero-mean laser point cloud y i ' is calculated as:

[0126] y i ′=t j -μ t ; where μ t is the centroid of the laser point cloud data;

[0127] The zero-mean laser point cloud y i ' is calculated as:

[0128] x i ′=s i ′-μ s ; where μ s is the point cloud S' to be registered i The centroid of

[0129] Step S32343: Based on the zero-mean laser point cloud y i ′ and the zero-mean point cloud x to be registered i 'Calculate and obtain the covariance matrix H;

[0130] The covariance matrix H is calculated as follows:

[0131]

[0132] Step S32344: Perform singular value decomposition on the covariance matrix H to obtain an orthogonal matrix U and a transpose V of the orthogonal matrix T ;

[0133] Step S32345: Based on the orthogonal matrix U and the transpose V of the orthogonal matrix T Calculate the new rotation matrix R';

[0134] The new rotation matrix R' is calculated as follows:

[0135] R=VU T ;

[0136] Step S32346: Based on the centroid μ of the laser point cloud data t And the point cloud S' to be registered i The center of mass μ sAnd the new rotation matrix R' is calculated to obtain a new translation vector t';

[0137] The new translation vector t' is calculated as follows:

[0138] t=μ t -R'μ s ;

[0139] Step S32347: construct a new initial transformation matrix based on the new rotation matrix R' and the new translation vector t'.

[0140] It should be noted that the above-mentioned embodiment of the present application effectively integrates the laser point cloud and the image feature point data by constructing a combined point pair, thereby improving the utilization rate of information in the registration process; further, the centroid removal operation eliminates the influence of translation, so that subsequent calculations can be more focused on the relative position of the point cloud, thereby improving the accuracy of registration. At the same time, the calculation of the covariance matrix provides a quantitative description of the relationship between the point clouds, providing important information for solving the rotation matrix; in further operation, through the calculation of singular value decomposition and rotation matrix, the scheme can accurately capture the spatial relationship between the point clouds, thereby generating a new translation vector, ensuring the precise alignment of the laser point cloud with the point cloud to be registered;

[0141] The above-mentioned embodiments of the present application transform the complex point cloud registration problem into a series of easy-to-handle steps through the method of linear algebra, which can effectively improve the computing efficiency and stability.

[0142] In summary, the present invention proposes a high-precision mapping and identification method for a substation using a handheld laser radar. By using a handheld rotating laser radar scanner to perform high-precision mapping of the surrounding environment and the substation, the handheld rotating laser radar device can perform a fast and accurate three-dimensional scan of the substation to generate detailed terrain and structural data, especially by using image recognition and point cloud to accurately identify specific substation equipment, generate the morphological structure of the substation scene through the point cloud, generate details from the RGBD image, and then fuse the morphological structure with the details to obtain a detailed morphological structure model of the substation scene. Finally, the initial substation scene model is analyzed to detect detail defects in the substation, so that measurement and maintenance can be better performed. By analyzing the surveyed substation scene model, the status changes of each device in the substation can be analyzed, such as corrosion, cracks, deformation or damage of the equipment, which is helpful for the planning, construction and maintenance of the substation.

[0143] In the specific operation process, through three registrations, firstly, the image feature descriptors and point cloud feature descriptors are extracted from the image feature points and point cloud feature points respectively, and then the first registration is performed based on the KD tree constructed based on the point cloud feature descriptors, that is, the coarse registration operation, which aims to filter out unreliable fuzzy matches and retain more reliable matching pairs, providing a good starting point for subsequent fine registration. At the same time, it effectively reduces the complexity of data processing, reduces the search space in the subsequent registration process by matching excellent feature point pairs, and improves the efficiency and accuracy of registration;

[0144] The first registration point group is further processed by the RANSAC algorithm, random sampling and consistency checking are performed to estimate the initial transformation matrix. The generated initial transformation matrix (4×4 homogeneous transformation matrix) can accurately map the 3D information in the RGBD image to the point cloud coordinate system, significantly improving the convergence speed and accuracy of subsequent fine registration algorithms (such as ICP).

[0145] Finally, based on the initial transformation matrix, the ICP algorithm is used for iterative processing to finely align the registration feature points, further improving the registration accuracy;

[0146] In the specific iterative process of the ICP algorithm, the image feature points are first rotated and translated using the initial transformation matrix to obtain the point cloud data to be registered, ensuring that the two are compared and matched in the same coordinate system. The registration error E between the point cloud to be registered and the laser point cloud is further calculated to quantify the effect of the current registration and help determine whether it is necessary to continue iterating or end the registration process. Furthermore, by continuously evaluating the error and the number of iterations, the registration strategy is flexibly adjusted to ensure that the best registration effect is achieved within a reasonable time.

[0147] Finally, the registration degree is calculated by adding the calculation of spatial autocorrelation to optimize the point cloud registration process, and the above-mentioned registration feature point group to be determined is verified, thereby ensuring that the analysis and acquisition of the target registration feature point group is more accurate.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. A person skilled in the art may modify the technical solutions described in the above embodiments, or replace part or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-precision mapping and identification method for substations using a handheld laser radar, characterized in that: The steps are as follows: The RGB image data of the substation is acquired through the image acquisition device carried by the handheld laser radar device; at the same time, the laser point cloud data of the substation is acquired through the laser radar device carried by the handheld laser radar device; Based on the RGB image data, a RGBD image is obtained by calculating the camera parameters of the image acquisition device; Performing image registration based on the RGBD image and the laser point cloud data to obtain a substation scene model; An analysis is performed based on the substation scenario model to obtain the equipment status of the substation.

2. According to claim 1, a high-precision mapping and identification method for substations using a handheld laser radar is characterized in that: The step of performing image registration based on the RGBD image and the laser point cloud data to obtain a substation scene model includes the following steps: Extracting image feature points from the RGBD image and simultaneously extracting point cloud feature points from the laser point cloud data; Acquire a registration feature point group based on registering the image feature points with the point cloud feature points; The RGBD image and the laser point cloud data are registered based on the registration feature point group to obtain a substation scene model.

3. According to claim 2, a high-precision mapping and identification method for substations using a handheld laser radar is characterized in that: The step of obtaining a registration feature point group based on the registration of the image feature points and the point cloud feature points comprises the following steps: An image feature descriptor is obtained based on the image feature point analysis; a point cloud feature descriptor is obtained based on the point cloud feature point analysis; a first registration point pair is obtained by searching and constructing a KD tree constructed based on the image feature descriptor; The first registration point group is estimated by RANSAC coarse registration processing to obtain an initial transformation matrix; Based on the initial transformation matrix, an iterative processing analysis is performed through ICP fine registration to obtain a registration feature point group.

4. According to claim 3, the high-precision mapping and identification method for substations using a handheld laser radar is characterized in that: The step of searching and constructing a first registration point pair in a KD tree constructed based on the image feature descriptor and the point cloud feature descriptor specifically includes: Using the image feature descriptor to search for K nearest neighbors in the KD tree, to obtain K first registration point cloud feature points corresponding to each of the image feature points; A first registration point pair is constructed based on the image feature point and the k first registration point cloud feature points.

5. According to claim 4, the high-precision mapping and identification method for substations using a handheld laser radar is characterized in that: Each element in the initial transformation matrix represents a second registration point group; the second registration point group represents a combination of image feature points and point cloud feature points with higher accuracy obtained by screening the first registration point group through re-registration.

6. The high-precision mapping and identification method for substations using a handheld laser radar according to claim 5 is characterized in that: The iterative processing and analysis based on the initial transformation matrix through ICP fine registration to obtain a registration feature point group includes the following steps: Initialize iteration conditions; the iteration conditions include an iteration counter, a maximum number of iterations, and a minimum error threshold; the number of iterations of the counter is initially 0; Iterative processing operation: Based on the initial transformation matrix, each of the image feature points S i Perform calculations to obtain the point cloud S' to be registered i ; The iteration number of the iteration counter is increased by 1 to obtain the current iteration number, and based on the point cloud to be registered S' i and the point cloud S' to be registered i Laser point cloud data with the shortest distance t j Calculate and obtain the registration error E; Based on the registration error E and the minimum error threshold, a judgment is made and a group of registration feature points to be determined is iteratively outputted; Calculating the registration degree based on the to-be-determined registration feature point group; A target registration feature point group is iteratively output based on the registration degree and a preset registration degree threshold.

7. The high-precision mapping and identification method of a substation using a handheld laser radar according to claim 6 is characterized in that: The step of judging based on the registration error E and the minimum error threshold and iteratively outputting the to-be-determined registration feature point group specifically includes the following steps: Determine whether the registration error E is less than or equal to the minimum error threshold, if so, output the image feature point S i With the laser point cloud data j Constitute a group of registration feature points to be determined; if not, determine whether the current number of iterations is greater than or equal to the maximum number of iterations; if so, output the image feature point S i With the laser point cloud data j If not, then based on the registration error E and the laser point cloud data t j And the image feature point S i Perform analysis and calculation to obtain a new initial transformation matrix, and return to the above iterative processing operation based on the new initial transformation matrix until a group of registration feature points to be determined is output.

8. The high-precision mapping and identification method of a substation using a handheld laser radar according to claim 7 is characterized in that: The step of calculating the registration degree based on the to-be-determined registration feature point group; and iteratively judging and outputting the target registration feature point group based on the registration degree and a preset registration degree threshold specifically includes the following steps: Calculating the registration degree based on the to-be-determined registration feature point group; determining whether the registration degree is greater than or equal to a preset registration degree threshold, and if so, outputting the to-be-determined registration feature point group as a target registration feature point group; If not, a new initial transformation matrix is ​​calculated using the image feature points and point cloud feature points corresponding to the to-be-determined registration feature point group, and the above iterative processing operation is returned until the registration degree is greater than or equal to the preset registration degree threshold, and the target registration feature point group is output.

9. The high-precision mapping and identification method for substations using a handheld laser radar according to claim 8 is characterized in that: The point cloud to be registered S' i The calculation method is: s′ i =R0s i +t0; In the formula, R0 is the rotation matrix in the initial transformation matrix; t0 is the translation vector in the initial transformation matrix; The registration error E is calculated as follows: E=(1 / m)*Σ(min‖s' i -t j ‖ 2 ): In the formula, m is the number of image feature points; min‖s' i -t j ‖ represents the point cloud S' to be registered i And the point cloud S' to be registered i Laser point cloud data with the shortest distance t j The Euclidean distance between .

10. The high-precision mapping and identification method for substations using a handheld laser radar according to claim 9 is characterized in that: The step of calculating the registration degree based on the to-be-determined registration feature point group comprises the following steps: Based on the plurality of laser point cloud data t in the plurality of registration feature point groups to be determined j And the point cloud to be registered S' i Multiple registration blocks are obtained through clustering; The spatial autocorrelation Δ between the qth registration block and each registration block p except the qth registration block is calculated. q,p ; Based on the spatial autocorrelation Δ q,p And the number of the registration blocks w is calculated to obtain the registration degree; The registration degree is calculated as follows: Among them, γ w represents the registration degree of the wth registration block; Δ q,p represents the spatial autocorrelation between the qth registration block and each registration block p except the qth registration block; w represents the total number of registration blocks; []* represents a regular expression, [] represents a character class, matching any single character in the square brackets; * is a quantifier, indicating that the character class can appear zero or more times.

Images