Multi-sensor fusion SLAM method for substation scene modeling

Abstract

The invention relates to a multi-sensor fusion SLAM (Simultaneous Localization and Mapping) method for substation scene modeling, which comprises the following steps of: firstly, segmenting angular points and pixel blocks detected by multiple sensors by adopting an improved RANSAC (Random Sample Consensus) algorithm, and correlating the angular points and the pixel blocks; then, carrying out joint multi-target pose estimation on the associated angular points and pixel blocks; then, a point cloud map of the substation scene is established based on a plane scanning algorithm, and finally, closed-loop optimization and improvement are performed on the point cloud map. According to the invention, the improved RANSAC algorithm is adopted, so that the defects of large data volume and non-uniform distribution of the point cloud data of the transformer substation can be effectively reduced; by adopting the improved Mahalanobis distance calculation method, the success rate and the accuracy of correlating the angular points and the pixel blocks can be greatly improved. And the combined multi-target pose estimation is combined with a logic utility function, so that the prior information allowing combination of behaviors of different descriptor types is realized. The problems of noise residue, high equipment segmentation difficulty, low model matching efficiency and the like in substation modeling are solved.

Description

Technical Field

[0001] The present invention relates to the field of sensor technology, and in particular to a multi-sensor fusion SLAM method for substation scene modeling. Background Art

[0002] The development of smart grids can make the power system more visual and refined, which is more conducive to the health monitoring of the power grid and the subsequent operation and maintenance management. As the hub of the power system, the substation not only has the function of distributing power supply, but also can control the voltage rise and fall. Therefore, once its state is abnormal, it will cause great economic losses and social impact, so it is very necessary to maintain its stable operation. Substations are different from other civil buildings. They not only have a large number of intricately distributed power cables and complex loads, but also cover a large number of various types of electrical equipment, which brings great challenges to the maintenance of substations. Therefore, it is very meaningful to seek an effective and fast substation operation and maintenance method.

[0003] At present, the development of three-dimensional digital technology is very consistent with the maintenance needs of new substations. It is an effective technology. The digital three-dimensional model of the substation can realize the visualization of the substation. Accurately and effectively reconstructing the three-dimensional digital expression of the substation from the real world is the key to substation visualization. In recent years, the country's use of electricity has increased, and the scale of large-capacity and high-voltage substations has also increased, thereby ensuring the stable output and distribution of electricity. However, it has brought about increasingly complex and arduous operation and maintenance work. Therefore, it is very necessary to research and develop intelligent detection and automatic operation and maintenance technology, and at the same time expand the scale of professional training for relevant operators. Therefore, the maintenance of substations needs to be improved from both technical and personnel aspects. On the one hand, in order to realize the digitization and intelligence of the power system, accurate and real three-dimensional visualization data of the substation is required as important information. Therefore, accurately and effectively reconstructing the three-dimensional digital expression of the substation from the real world is the key to substation visualization. On the other hand, when training substation operation and maintenance staff, it is impossible to perform live operation and create accidents to train operation and maintenance operations, resulting in a lack of actual operation that affects the training effect. If there is no intuitive on-site operation auxiliary system, the expansion, maintenance and vehicle entry of the substation must be accompanied by on-site survey and measured data and other preparations, which will increase the difficulty of the operator's work and prolong the maintenance time. In order to ensure the safety of technical personnel and reduce the corresponding economic losses, in recent years, how to build a substation 3D simulation model with real information, excellent model pictures and high adaptability has attracted people's attention. At present, the mainstream substation 3D modeling methods are mainly divided into the following categories: modeling method based on virtual reality modeling language, which has high technical threshold, long modeling process and cannot directly observe the model; modeling method based on model geometric features, although compared with virtual reality, the efficiency is improved, but the modeling authenticity is not high, mainly relying on manual model building, and this modeling method requires high labor costs; SLAM modeling method based on ground laser radar, this method is the most popular method, which has the characteristics of high precision and strong intuition, and is very suitable for the needs and applications of 3D simulation substation model construction.

[0004] At present, researchers have conducted various studies on substation scenario modeling methods and proposed various technologies to achieve scenario modeling for substation scenarios.

[0005] For example, Sun Rong and others from the Electric Power Research Institute of Jiangsu Electric Power Co., Ltd. of State Grid applied for a substation modeling method based on three-dimensional scanning on June 2, 2020, which has been authorized. The patent application number is CN202010489195.6. The invention relates to the field of 3D modeling, in particular, a substation modeling method based on three-dimensional scanning. The process is as follows: after the point cloud data of the substation is obtained by scanning with a three-dimensional scanner, the data is first preprocessed, that is, the point cloud data of the power lines on the ground and equipment of the substation are automatically removed, and then the point cloud is clustered to make the point cloud data of the equipment independent. Then, a point cloud model is established through data splicing, data denoising, data simplification and feature extraction. On this basis, the algorithm of repeated structure detection based on the topological structure of the two-dimensional plane bounding box is adopted to repeat the structural pattern of the power station scene respectively. Finally, the modeling method based on component matching is used to model and fit the minimum repeating unit, and combined with the repeated structural pattern to obtain a substation model. When modeling using this invention method, the processing speed of point cloud data is fast and the accuracy is high, which effectively improves the rapidity and effectiveness of virtual reality modeling of power systems.

[0006] There is also a currently authorized three-dimensional modeling method, medium and system for substations based on multi-source data fusion, which was applied for by Wu Yong and others from Jiangsu Power Transmission and Transformation Co., Ltd. on April 28, 2021. The patent application number is CN202110466492.3. The invention discloses a three-dimensional modeling method, medium and system for substations based on multi-source data fusion, including the following steps: 1) scanning the substation in multiple ways; 2) data preprocessing; 3) obtaining a dense point cloud based on oblique photography images and then performing point cloud thinning to obtain a sparse point cloud; 4) completing the unification of the spatial coordinate system between the laser point cloud and the sparse point cloud; 5) using a local linear embedding model for non-aligned multi-view data fusion to complete the rough registration of the laser point cloud and the sparse point cloud; 6) using an iterative closest point algorithm to achieve accurate registration of the laser point cloud and the sparse point cloud, and to achieve the fusion of the laser point cloud and the sparse point cloud data; 7) generating a real-life three-dimensional model of the substation based on the fused point cloud data. The invention can improve the accuracy of three-dimensional modeling of single-source data and improve the efficiency of three-dimensional modeling.

[0007] Although many experts and scholars have conducted extensive research on substation scene modeling, such as the methods proposed in the two related patents, and there have been great breakthroughs in the research on substation scene modeling, there are also many existing solutions, such as the SLAM method, but most of them have defects. The substation scene is established based on the SLAM method. Although the modeling effect is good and the sense of reality is strong, due to the large number of substation equipment and the complex equipment structure, the formation of substation site cloud data will result in large data volume and uneven distribution. In addition, the traditional substation site cloud data processing method is applied to substation modeling, and there are problems such as residual noise, difficulty in equipment segmentation and low model matching efficiency. Summary of the invention

[0008] In view of the above-mentioned defects of the prior art, the present invention proposes a multi-sensor fusion SLAM method for substation scene modeling. First, the improved RANSAC algorithm is used to segment the corner points and pixel blocks detected by multiple sensors and correlate them; then, the correlated corner points and pixel blocks are jointly estimated for multi-target poses; then, a point cloud map of the substation scene is established based on a plane scanning algorithm; finally, the point cloud map is closed-loop optimized and improved.

[0009] To achieve the above object, the present invention adopts the following technical solution: a multi-sensor fusion SLAM method for substation scene modeling, comprising the following steps:

[0010] The improved RANSAC algorithm is used to segment the corner points and pixel blocks detected by multiple sensors and correlate them with each other;

[0011] The obtained associated corner points and pixel blocks are used to perform joint multi-target pose estimation;

[0012] The point cloud pose is estimated based on the plane scanning to establish the point cloud map of the substation scene;

[0013] The obtained point cloud map is optimized and improved in a closed loop.

[0014] Furthermore, the improved RANSAC algorithm is used to segment the corner points and pixel blocks detected by the multi-sensor, and correlate them, specifically:

[0015] The Rodrigues formula is used to rotate the plane in any direction and align it with the XYZ axes. This rotation is applied to all points on the plane, and the points on the plane are projected as R by removing the z value. 3 →R 2 , so that we can get in, represents the corner points of plane v;

[0016] The improved RANSAC algorithm is used for the segmentation of pixel blocks. Assuming that each sensor point cloud is a point cloud candidate of a cylindrical feature, the center of mass c′ and the radius r′ of the cylinder are roughly approximated. Then, the parameters of the two are recalculated. The rough radius r′ is calculated from the maximum and minimum boundaries of the point cloud facing the sensor view. The specific formula is expressed as Among them, n fl is the vector pointing from the sensor origin to the i-th point cloud cluster of the sensor, proj floor It is the vector formed by projecting the unit vector of the sensor direction onto the plane of the floor. The specific formula is:

[0017] The improved RANSAC algorithm extracts corner points and pixel blocks detected by multiple sensors, taking into account the features previously observed in the sensor In the previous measurement Based on this, the feature is updated as The improved Mahalanobis distance calculation method is used to correlate the corner points and pixel blocks. The specific formula is expressed as follows: where P is the covariance in the same coordinate system as the measurements.

[0018] Furthermore, the obtained associated corner points and pixel blocks are used to perform joint multi-target pose estimation, specifically:

[0019] After the associated corner points and pixel blocks, the utility function is introduced to analyze the current observation value, and the weight of each residual type is modified accordingly as the optimization progresses. This scalar multi-objective optimization process is expressed as follows:

[0020]

[0021] Among them, K is the output of the utility function. When the number of feature matches between corner points and pixel blocks is low, the output K of the utility function is calculated based on the residual of the geometric structure. The specific formula is expressed as:

[0022]

[0023] Among them, l is the pyramid level to be optimized, N g is the number of current inner geometry matches; n p and n g are the counting types of corner point and pixel block features, respectively. and are the variances of the photometric and geometric residuals, respectively, ||·|| is the Huber norm, and the energy of the photometric and geometric residual characteristics is expressed as:

[0024] e p (ε)=(r T Wr) p , e g (ε)=(r T Wr) g

[0025] Among them, r is the stacked residual vector of each feature type, r p represents the photometric residual, r g Represents the geometric reprojection residual; then, the weight matrix is ​​used to suppress the pose estimation error caused by large depth changes. The specific formula is expressed as in, Represents the pose estimation error caused by depth change.

[0026] Furthermore, the point cloud pose estimation obtained is used to establish a point cloud map of the substation scene based on plane scanning, specifically:

[0027] The virtual plane established after pose estimation is used to sample the substation scene, implicitly providing depth assumptions to reconstruct the final substation scene.

[0028] The scene virtual plane is parameterized by the normal vector n in the XYZ axis, and the distance d along the normal vector to the sensor is obtained, combined with the intrinsic sensor pose matrix K between the matching pose estimate and the reference pose. k and the relative position R of the external sensor t , calculate the plane induced homography H, the specific formula can be expressed as:

[0029]

[0030] Among them, K ref is the intrinsic sensor reference pose matrix, t is the time, T is the transposed matrix of the normal vector n, and the plane induced homography formula can be used to match the current point cloud map to the global point cloud map.

[0031] Furthermore, the obtained point cloud map is closed-loop optimized and improved, specifically:

[0032] A loop closure method is adopted to generate a global keyframe descriptor by parsing the newly added keyframe into the current moment map model, which is used to query the keyframe global database for potential matches;

[0033] The candidate objects connected to the latest keyframe in the co-visibility graph are discarded in a discarding manner. If no loop candidate is found, the loop closure thread is terminated. If a loop candidate is detected, 3D-3D mapping point matching is performed between the loop candidate keyframe and the most recently added keyframe, and then random matching is used to correct the substation map model.

[0034] The present invention has the following technical effects:

[0035] 1. The present invention adopts an improved RANSAC algorithm, which can effectively reduce the defects of large data volume and uneven distribution of substation cloud data; adopts an improved Mahalanobis distance calculation method, which takes into account the uncertainty of each measurement while calculating the dimensionless distance, and can greatly improve the success rate and accuracy of correlating corner points with pixel blocks.

[0036] 2. The joint multi-objective pose estimation combined with the logical utility function adopted in the present invention provides a mechanism to guide multi-objective optimization in the multi-objective optimization process, thereby realizing prior information that allows the combination of different descriptor types of behaviors. Since the pixel block-based residuals have smaller convergence characteristics, and when the optimization starts from a relatively distant initialization point, the corner point-based residuals perform better. In this way, the utility function is used to give higher weights to the corner point residuals in the early stage of optimization, and then gradually move the weights to the pixel block-based residuals, thereby establishing more reliable feature matching and improving the accuracy of pose estimation.

[0037] 3. The present invention establishes a point cloud map through a plane scanning algorithm, which can solve the problems of residual noise, difficulty in equipment segmentation and low model matching efficiency in substation modeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A flow chart of a multi-sensor fusion SLAM method for substation scene modeling of the present invention. DETAILED DESCRIPTION

[0039] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings to make the technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0040] In view of the above key technical problems, the present invention designs a multi-sensor fusion SLAM method for substation scene modeling. First, an improved RANSAC algorithm is used to segment the corner points and pixel blocks detected by multiple sensors and correlate them with each other; then, the correlated corner points and pixel blocks are jointly estimated for multi-target poses; then, a point cloud map of the substation scene is established based on a plane scanning algorithm; finally, the point cloud map is closed-loop optimized and improved.

[0041] In view of the above technical problems in the prior art, the present invention provides a multi-sensor fusion SLAM method for substation scene modeling, and the specific steps of the method can be as follows:

[0042] Step 1: Use the improved RANSAC algorithm to segment the corner points and pixel blocks detected by multiple sensors and correlate them

[0043] The first step of establishing the substation scene in the present invention is to find the points and pixel blocks that constitute the main surfaces of the scene and parameterize them. First, the Rodrigues formula is used to rotate the plane in any direction and align it with the XYZ axis. This rotation is applied to all points on the plane, which are later projected into R by removing the z value. 3 →R 2 , so that we can get in, Represents the corner point of plane v. The improved RANSAC algorithm is used for the segmentation of pixel blocks. Assuming that each sensor point cloud is a point cloud candidate of a cylindrical feature, the center of mass c′ and the radius r′ of the cylinder are roughly approximated first, and then the parameters of the two are recalculated. The rough radius r′ is calculated from the maximum and minimum boundaries of the point cloud facing the sensor view. The specific formula can be expressed as Among them, n fl is the vector pointing from the sensor origin to the i-th point cloud cluster of the sensor, proj floor It is the vector formed by projecting the unit vector of the sensor direction onto the plane of the floor. The specific formula can be expressed as The improved RANSAC algorithm has extracted the corner points and pixel blocks detected by multiple sensors, taking into account the features previously observed in the sensor In the previous measurement Based on this, the feature is updated as Therefore, the present invention uses an improved Mahalanobis distance calculation method to correlate corner points and pixel blocks. The specific formula can be expressed as: where P is the covariance in the same coordinate system as the measurements.

[0044] The final result is the associated corner points and pixel blocks. Beneficial effects: This step uses the improved RANSAC algorithm, which can effectively reduce the defects of large data volume and uneven distribution of substation cloud data; the improved Mahalanobis distance calculation method is used to calculate the dimensionless distance while also considering the uncertainty of each measurement, which can greatly improve the success rate and accuracy of the correlation between corner points and pixel blocks.

[0045] Step 2: Joint multi-target pose estimation based on the associated corner points and pixel blocks obtained in step 1

[0046] The present invention calculates the sensor attitude by minimizing the photometric and geometric residuals simultaneously, thereby completing the sensor pose estimation. Since the corner points and pixel blocks after association in step 1 have different but complementary characteristics, a utility function is introduced to analyze the current observation value, and the weight of each residual type is modified accordingly as the optimization progresses. This scalarized multi-objective optimization process can be expressed as Among them, K is the output of the utility function. When the number of feature matches between corner points and pixel blocks is low, the output K of the utility function is calculated based on the residual of the geometric structure. The specific formula can be expressed as Among them, l is the pyramid level to be optimized, N g is the number of current inner geometry matches; np and n g are the counting types of corner point and pixel block features, respectively. and are the variances of the photometric and geometric residuals, respectively, ||·|| is the Huber norm, and the energy of the photometric and geometric residual characteristics is expressed as e p (ε)=(r T Wr) p , e g (ε)=(r T Wr) g , where r is the stacked residual vector for each feature type, r p represents the photometric residual (pixel intensity difference), r g Represents the geometric reprojection residual (pixel distance difference). Next, the weight matrix is ​​used to suppress the pose estimation error caused by large depth changes. The specific formula can be expressed as in, Represents the pose estimation error caused by depth change.

[0047] The final result is the sensor pose estimation after the joint multi-objective algorithm. Beneficial effect: The joint multi-objective pose estimation used in this step is combined with the logical utility function, which provides a mechanism to guide the multi-objective optimization in the multi-objective optimization process, thereby realizing the prior information that allows the combination of different descriptor types. Since the pixel block-based residual has a smaller convergence characteristic, when the optimization starts from a relatively far initialization point, the corner point-based residual performs better. In this way, the utility function is used to give the corner point residual a higher weight in the early stage of optimization, and then gradually move the weight to the pixel block-based residual, thereby establishing a more reliable feature match and improving the accuracy of pose estimation.

[0048] Step 3: The point cloud pose estimation obtained in step 2 is used to establish a point cloud map of the substation scene based on plane scanning.

[0049] The present invention adopts a plane scanning algorithm and uses a virtual plane established after pose estimation to sample the substation scene, implicitly providing a depth hypothesis to reconstruct the final substation scene. The scene virtual plane parameterizes the normal vector n by the direction in the XYZ axis, obtains the distance d along the normal vector to the sensor, and combines the intrinsic sensor pose matrix K between the matching pose estimation and the reference pose k and the relative position R of the external sensor t , calculate the plane induced homography H, the specific formula can be expressed as Among them, K refis the intrinsic sensor reference pose matrix, t is the time, T is the transposed matrix of the normal vector n, and the plane induced homography formula can be used to match the current point cloud map to the global point cloud map.

[0050] The final result is a point cloud map of the substation scene. Beneficial effects: This step uses a plane scanning algorithm to establish a point cloud map, which can solve the problems of residual noise, difficulty in equipment segmentation, and low model matching efficiency in substation modeling.

[0051] Step 4: Closed-loop optimization and improvement of the point cloud map obtained in step 3

[0052] The present invention adopts a loop closure method, by parsing the newly added keyframe into the current moment map model to generate a global keyframe descriptor, which is then used to query the potential match of the keyframe global database. In order to prevent false detection, the candidate object connected to the latest keyframe in the common view map is discarded in a discarding manner. If no loop candidate is found, which is also a common situation for most keyframes, the loop closure thread is terminated. If a loop candidate is detected, 3D-3D mapping point matching is performed between the loop candidate keyframe and the most recently added keyframe, and then random matching is used to correct the substation map model. The final result is a SLAM map of the substation scene.

[0053] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A multi-sensor fusion SLAM method for substation scene modeling, characterized in that: The following steps are involved: The improved RANSAC algorithm is used to segment the corner points and pixel blocks detected by multiple sensors and correlate them with each other; The obtained associated corner points and pixel blocks are used to perform joint multi-target pose estimation; The point cloud pose is estimated based on the plane scanning to establish the point cloud map of the substation scene; The obtained point cloud map is optimized and improved in a closed loop.

2. A multi-sensor fusion SLAM method for substation scene modeling according to claim 1, characterized in that: The improved RANSAC algorithm is used to segment the corner points and pixel blocks detected by multiple sensors and correlate them, specifically: The Rodrigues formula is used to rotate the plane in any direction and align it with the XYZ axes. This rotation is applied to all points on the plane, and the points on the plane are projected to R by removing the z value. 3 →R 2 , so that we can get in, represents the corner points of plane v; The improved RANSAC algorithm is used for the segmentation of pixel blocks. Assuming that each sensor point cloud is a point cloud candidate of a cylindrical feature, the center of mass c′ and the radius r′ of the cylinder are roughly approximated. Then, the parameters of the two are recalculated. The rough radius r′ is calculated from the maximum and minimum boundaries of the point cloud facing the sensor view. The specific formula is expressed as Among them, n fl is the vector pointing from the sensor origin to the i-th point cloud cluster of the sensor, proj floor It is the vector formed by projecting the unit vector of the sensor direction onto the plane of the floor. The specific formula is: The improved RANSAC algorithm extracts corner points and pixel blocks detected by multiple sensors, taking into account the features previously observed in the sensor In the previous measurement Based on this, the feature is updated as The improved Mahalanobis distance calculation method is used to correlate the corner points and pixel blocks. The specific formula is expressed as follows: where P is the covariance in the same coordinate system as the measurements.

3. The multi-sensor fusion SLAM method for substation scene modeling according to claim 1 is characterized in that: The joint multi-target pose estimation is performed based on the obtained associated corner points and pixel blocks, specifically: After the associated corner points and pixel blocks, the utility function is introduced to analyze the current observation value, and the weight of each residual type is modified accordingly as the optimization progresses. This scalar multi-objective optimization process is expressed as follows: Among them, K is the output of the utility function. When the number of feature matches between corner points and pixel blocks is low, the output K of the utility function is calculated based on the residual of the geometric structure. The specific formula is expressed as: Among them, l is the pyramid level to be optimized, N g is the number of current inner geometry matches; n p and n g are the counting types of corner point and pixel block features, respectively. and are the variances of the photometric and geometric residuals, respectively, ‖·‖ is the Huber norm, and the energy of the photometric and geometric residual characteristics is expressed as follows: e p (ε)=(r T Wr) p ,e g (ε)=(r T Wr) g Among them, r is the stacked residual vector of each feature type, r p represents the photometric residual, r g Represents the geometric reprojection residual; then, the weight matrix is ​​used to suppress the pose estimation error caused by large depth changes. The specific formula is expressed as in, Represents the pose estimation error caused by depth change.

4. The multi-sensor fusion SLAM method for substation scene modeling according to claim 1 is characterized in that: The point cloud pose estimation obtained is used to establish a point cloud map of the substation scene based on plane scanning, specifically: The virtual plane established after pose estimation is used to sample the substation scene, implicitly providing depth assumptions to reconstruct the final substation scene. The scene virtual plane is parameterized by the normal vector n in the XYZ axis, and the distance d along the normal vector to the sensor is obtained, combined with the intrinsic sensor pose matrix K between the matching pose estimate and the reference pose. k and the relative position R of the external sensor t , calculate the plane induced homography H, the specific formula can be expressed as: Among them, K ref is the intrinsic sensor reference pose matrix, t is the time, T is the transposed matrix of the normal vector n, and the plane induced homography formula can be used to match the current point cloud map to the global point cloud map.

5. The multi-sensor fusion SLAM method for substation scene modeling according to claim 1 is characterized in that: The closed-loop optimization and improvement of the obtained point cloud map is specifically as follows: A loop closure method is adopted to generate a global keyframe descriptor by parsing the newly added keyframe into the current moment map model, which is used to query the keyframe global database for potential matches; The candidate objects connected to the latest keyframe in the co-visibility graph are discarded in a discarding manner. If no loop candidate is found, the loop closure thread is terminated. If a loop candidate is detected, 3D-3D mapping point matching is performed between the loop candidate keyframe and the most recently added keyframe, and then random matching is used to correct the substation map model.

Images