Transformer substation modeling dynamic SLAM method for improving multi-view geometry

Abstract

The invention discloses a transformer substation modeling dynamic SLAM method for improving multi-view geometry, and belongs to the technical field of transformer substations, and the method comprises the steps: carrying out the spatial clustering of the point cloud data of a sensor, and obtaining the point cloud cluster data; performing optical flow consistency sensor pose estimation on the point cloud cluster data obtained in the step 1; establishing an improved residual model of a flexible threshold for the pose estimation of the sensor obtained in the step 2 to optimize the pose of the sensor; and SLAM map construction is carried out on the pose optimization model obtained in the step 3 by adopting an improved multi-view geometric algorithm. According to the method, many point cloud clusters are generated, and each point cloud cluster is regarded as a rigid body generated by converting non-rigid pixel-level point cloud data into a cluster-level rigid scene; false detection caused by single-point measurement is avoided, the complexity of sensor motion segmentation is reduced, and the accuracy of sensor pose estimation is improved; the sensor attitude estimation from the initial frame to the current frame is established, the sensor attitude estimation is improved, and the SLAM mapping speed of the substation scene is improved.

Description

Technical Field

[0001] The invention relates to the technical field of substations, and in particular to a substation modeling dynamic SLAM method with improved multi-view geometry. Background Art

[0002] As a place for transforming, concentrating and distributing voltage and current in the power system, substation plays an extremely important role. It not only ensures the stable operation of the transmission network, but also ensures the safety of power supply in the city. In modern society, electricity has become a necessity for people's production and life, and substation, as an indispensable part of the power system, is related to the country's economic construction and social stability. In recent years, with the rapid development of the national power grid construction, the power network has become more and more dense. Since 2016, the length of China's 35kV and above transmission line poles has increased from 1.581 billion meters to 1.882 billion meters, and the length of 35kV and above transmission line loops has increased from 1.756 billion meters to 2.156 billion meters. As the core content of the construction of the transmission network, the substation has a large number of power lines and complex substation equipment, which not only brings great challenges to the construction of the power system, but also puts forward higher requirements for the management and maintenance of substation equipment information. For substation modeling, the traditional two-dimensional GIS has great limitations in spatial expression and spatial analysis capabilities, and has shown great deficiencies in the general application of power transmission and transformation project information management. Because the two-dimensional GIS lacks intuitiveness, it is difficult to express the spatial relationship between power equipment. It can only present the plane position and basic information of the equipment, and cannot fully and accurately reflect the three-dimensional structure, operating status and spatial position of the equipment. It can no longer meet the needs of existing substation scene modeling. Therefore, it is necessary to introduce more advanced technologies and means, such as three-dimensional visualization fusion laser SLAM, to improve the efficiency and accuracy of substation scene modeling, so as to ensure the stable operation of the power system.

[0003] As a supporting technology for virtual reality, 3D visualization fusion laser SLAM is penetrating into the development of various industries, especially in urban planning and construction and other related industries that need to be supported by 3D model data. In particular, since the power sector has put forward relevant requirements for the establishment of a "strong smart grid", my country's power system has begun to develop in the direction of intelligence. Therefore, in the current national intelligent substation construction process, the use of 3D visualization fusion laser SLAM to establish substation scenes and build an intelligent substation equipment information management platform has become an inevitable trend of development. The 3D model data it uses has a more complete and detailed description of the substation scene, which can intuitively, comprehensively and accurately reflect the equipment information, operating status and spatial position inside the substation, helping staff to better understand the equipment operation, reduce the risk of operating errors and equipment damage, improve the safety and reliability of management work, and lay a data foundation for the intelligent management of the power system.

[0004] At present, many experts and scholars have conducted a lot of research on substation scene modeling, such as the existing patent: a method for holographic modeling of digital twins of substations (patent number: CN202310565082.3) and patent: a method for constructing a three-dimensional model of a substation based on radar point cloud data (patent number: CN202311859130.6). At present, there have been great breakthroughs in the research on substation scene modeling. There are also many existing solutions, but most of them have defects: the results generated by point cloud processing lack spatial correlation of the entire area, thereby introducing noise and reducing the accuracy of scene segmentation. The sensor movement is difficult to weaken, resulting in large errors in pose estimation. In order to solve the above problems, the present invention provides a dynamic SLAM method for substation modeling with improved multi-view geometry. Summary of the invention

[0005] In view of the above-mentioned technical deficiencies, the purpose of the present invention is to provide a dynamic SLAM method for substation modeling with improved multi-view geometry, which generates many point cloud clusters, and each point cloud cluster is regarded as a rigid body generated by converting non-rigid pixel-level point cloud data into a cluster-level rigid scene; avoids false detection caused by single-point measurement, greatly reduces the complexity of sensor motion segmentation; and solves the technical problems raised in the background technology.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solution: The present invention provides a substation modeling dynamic SLAM method with improved multi-view geometry, comprising:

[0007] Step 1: Perform spatial clustering on the point cloud data of the sensor to obtain point cloud cluster data;

[0008] Step 2: Perform optical flow consistency sensor pose estimation on the point cloud cluster data obtained in step 1;

[0009] Step 3: Based on the sensor pose estimation obtained in step 2, an improved residual model with a flexible threshold is established to optimize the sensor pose;

[0010] Step 4: Use the improved multi-view geometry algorithm to construct the SLAM map for the pose optimization model obtained in step 3.

[0011] Preferably, step one specifically includes the following process:

[0012] Step 1.1: Receive the depth point cloud data from the sensor as input. The depth point cloud data contains the depth value D(p) of each point cloud pixel. Initialize the K cluster centroids u1,u2,.....u in the corresponding 3D point cloud randomly. k Initially, each centroid u k Represents the average depth value of the point cloud cluster in 3D space;

[0013] Step 1.2: Each pixel p of the point cloud data is assigned to the cluster with the nearest centroid by calculating p and each centroid u k The squared Euclidean distance d(p,k) between them assigns the point cloud pixel p to cluster C. p , the distribution formula is: C p =arg min||D(p)-u k || 2 ;

[0014] Step 1.3: Take the centroid u k The update step is to transform the centroid u of each cluster k Recalculated as the average depth of all points assigned to the cluster, the specific formula is: k =kD(p) / N k ; That is, the centroid u of each cluster k is updated to the average of the depth D(p) at all pixels p, where the cluster assignment C p Equal to k, N k Represents the number of points assigned to cluster K. The algorithm iterates between the above two steps until convergence, thereby completing the clustering of the point cloud and finally obtaining the point cloud cluster data of the sensor.

[0015] Preferably, step 2 specifically includes the following process:

[0016] Step 2.1: The optical flow F∈R of the two frames I1 and I2 before and after the point cloud cluster data in step 1 W×H×2 Describes the point cloud pixel displacement, which aligns the first frame with the second frame, that is, at the point cloud pixel (x, y), according to the optical flow consistency, I1(x, y) = I2(x + F(x, y)x + F(x, y)y);

[0017] Step 2.2: Use the predicted shape code e of frame i i , and use a differentiable variant of marching cubes to extract the underlying grid from it. The specific formula can be expressed as M i =(V i ,T i ), where V i Represents the 3D vertices of the mesh, T i Represents its 3D triangle;

[0018] Step 2.3: Convert the mesh into a 3D scene to get the predicted sensor pose. The specific formula is: Where, represents the rotation matrix of the sensor;

[0019] Step 2.4: Get the sensor's position and posture by projecting each vertex. The specific formula is: Among them, represents the intrinsic matrix of the sensor; finally, the pose estimation of the sensor is obtained.

[0020] Preferably, step three includes the following process:

[0021] Step 3.1: Drive the threshold T by taking the best data c and multi-frame geometric threshold T g Thereby optimizing the posture;

[0022] Among them, the optimal data-driven threshold T c There are two unique peaks corresponding to the residuals in the static and dynamic regions. We apply Otsu to first optimize the best data-driven distribution pose and use the result to automatically calculate T c ; During the optimization process, T c and multi-frame geometric threshold T g , and estimated using the geometric persistence technique;

[0023] Step 3.2: Calculate the optimal data-driven threshold T using the Otsu method c , the specific formula is expressed as This automatically divides the distribution into static and dynamic components, minimizing the pose change, and the multi-frame geometric threshold T g is calculated as the 99th percentile displacement amplitude over a longer observation window, thus providing temporal stability, and is expressed as The final sensor pose T combines the two thresholds into a weighted sum, which is expressed as T = αT c +(1-α)T g .

[0024] Preferably, step 4 specifically includes the following process:

[0025] Step 4.1: Set the current frame F t and the corresponding depth map D t As input, point cloud cluster pixels are projected into 3D space to construct a geometric point cloud representation, thereby building a SLAM map of the substation scene;

[0026] Step 4.2: For each point cloud pixel position p, from the depth map D t Extract its depth D t (p), projecting the depth using the intrinsic parameters in K and π-1 of the point cloud cluster, converting the point cloud pixel position p to its associated 3D point X in metric coordinates relative to the center of the sensor p ;

[0027] Step 4.3: By repeating this operation for all point cloud pixel positions p, the 3D point cloud of the substation scene geometry is constructed. The specific formula is expressed as X p =Dt (p)*(k-1)*(π-1)(p).

[0028] The beneficial effects of the present invention are:

[0029] 1. The present invention generates many point cloud clusters, and each point cloud cluster is regarded as a rigid body generated by converting non-rigid pixel-level point cloud data into a cluster-level rigid scene; avoiding false detection caused by single-point measurement, greatly reducing the complexity of sensor motion segmentation, and improving the accuracy of sensor pose estimation.

[0030] 2. The present invention establishes the sensor pose estimation from the initial frame to the current frame through optical flow consistency sensor pose estimation, which helps to exclude dynamic objects during the sensor's self-motion calculation, thereby eliminating the pose estimation error caused by the sensor motion, and then eliminating motion noise, while improving the sensor pose estimation and increasing the speed of SLAM mapping of the substation scene.

[0031] 3. The present invention optimizes the pose by establishing an improved residual model with a flexible threshold, allowing a principled compromise between real-time and robustness, thereby reducing the time for pose estimation and improving the accuracy of pose estimation.

[0032] 4. The present invention adopts an improved multi-view geometry method to perform more accurate motion segmentation, eliminate the dynamic parts in the current frame and point cloud, thereby eliminating the influence of dynamic pedestrians and cars in the substation scene modeling process and improving the efficiency and accuracy of mapping. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 A structural schematic diagram of a dynamic SLAM method for substation modeling with improved multi-view geometry provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Example: Figure 1 As shown, the present invention provides a substation modeling dynamic SLAM method with improved multi-view geometry, comprising:

[0037] Step 1: spatially cluster the point cloud data of the sensor to obtain point cloud cluster data, which specifically includes the following process: In this embodiment, the point cloud data of the sensor is spatially clustered by an improved K-Means algorithm to avoid single pixel noise.

[0038] Step 1.1: Receive the depth point cloud data from the sensor as input. The depth point cloud data contains the depth value D(p) of each point cloud pixel. Initialize the K cluster centroids u1,u2,.....u in the corresponding 3D point cloud randomly. k Initially, each centroid u k Represents the average depth value of the point cloud cluster in 3D space;

[0039] Step 1.2: Each pixel p of the point cloud data is assigned to the cluster with the nearest centroid by calculating p and each centroid u k The squared Euclidean distance d(p,k) between them assigns the point cloud pixel p to the cluster C. p , the distribution formula is: C p =arg min||D(p)-u k || 2 ;

[0040] Step 1.3: Take the centroid u k The update step is to transform the centroid u of each cluster k Recalculated as the average depth of all points assigned to the cluster, the specific formula is: k =kD(p) / N k ; That is, the centroid u of each cluster k is updated to the average of the depth D(p) at all pixels p, where the cluster assignment C p Equal to k, N k Represents the number of points assigned to cluster K. The algorithm iterates between the above two steps until convergence, thereby completing the clustering of the point cloud and finally obtaining the point cloud cluster data of the sensor; the final result is the point cloud cluster data after spatial clustering. The beneficial effect of this part is: using a point cloud pixel segmentation algorithm in multiple point cloud frames, the result it generates lacks the spatial correlation of the entire area, which easily introduces noise and reduces the accuracy of scene segmentation. In contrast, this step generates many point cloud clusters, and each point cloud cluster can be regarded as a rigid body generated by converting non-rigid pixel-level point cloud data into a cluster-level rigid scene. In this way, false detection caused by single-point measurement can be avoided, the complexity of sensor motion segmentation can be greatly reduced, and the accuracy of sensor pose estimation can be improved.

[0041] Step 2: Perform optical flow consistency sensor pose estimation on the point cloud cluster data obtained in step 1. This embodiment uses an optical flow consistency algorithm to estimate the pose of the sensor, which specifically includes the following process:

[0042] Step 2.1: The optical flow F∈R of the two frames I1 and I2 before and after the point cloud cluster data in step 1 W×H×2 Describes the point cloud pixel displacement, which aligns the first frame with the second frame, that is, at the point cloud pixel (x, y), according to the optical flow consistency, I1(x, y) = I2(x + F(x, y)x + F(x, y)y);

[0043] Step 2.2: Use the predicted shape code e of frame i i , and use a differentiable variant of marching cubes to extract the underlying grid from it. The specific formula can be expressed as M i =(V i ,T i ), where V i Represents the 3D vertices of the mesh, T i Represents its 3D triangle;

[0044] Step 2.3: Convert the mesh into a 3D scene and get the predicted sensor pose. The specific formula is: Where, represents the rotation matrix of the sensor;

[0045] Step 2.4: Get the sensor's position and posture by projecting each vertex. The specific formula is: Among them, represents the intrinsic matrix of the sensor; finally, the pose estimation of the sensor is obtained. Some beneficial effects are as follows: this step will establish the sensor pose estimation from the initial frame to the current frame through the optical flow consistency sensor pose estimation, which helps to exclude dynamic objects during the self-motion calculation of the sensor, thereby eliminating the pose estimation error caused by the sensor motion, and then eliminating motion noise. At the same time, it can also improve the sensor pose estimation and increase the speed of SLAM mapping of the substation scene.

[0046] Step 3: Establishing an improved residual model with a flexible threshold for the pose estimation of the sensor obtained in step 2 to optimize the pose of the sensor; specifically including the following process: after dense optical tracking, there are still some dynamic points in the scene, which must be processed separately by establishing an improved residual model with a flexible threshold; the persistence of static objects is low or close to zero, because the durability distribution varies significantly with the environment; therefore, distinguishing between high durability and low durability by a constant threshold is problematic for the pose estimation of the sensor; to address this problem, this embodiment creates a flexible threshold technology to achieve pose optimization;

[0047] Step 3.1: Drive the threshold T by taking the best data c and multi-frame geometric threshold Tg Thereby optimizing the posture;

[0048] Among them, the optimal data-driven threshold T c There are two unique peaks, corresponding to the residuals in the static and dynamic regions, respectively. If the trough between the two peaks can be identified as a threshold, it can be easily distinguished. However, this distribution is sometimes discontinuous, which increases the ambiguity of the persistence curve and makes it difficult to determine the correct threshold. In solving this problem, this embodiment applies Otsu to first optimize the best data-driven distribution pose, and uses the result to automatically calculate T c ; Since posture optimization cannot distinguish between moving objects and static objects, T is used in the optimization process. c and multi-frame geometric threshold T g , and estimated using the geometric persistence technique;

[0049] Step 3.2: Calculate the optimal data-driven threshold T using the Otsu method c , the specific formula is expressed as This automatically divides the distribution into static and dynamic components, minimizing the pose change, and the multi-frame geometric threshold T g is calculated as the 99th percentile displacement amplitude over a longer observation window, thus providing temporal stability, and is expressed as The final sensor pose T combines the two thresholds into a weighted sum, which is expressed as T = αT c +(1-α)T g The final result is the sensor pose after optimization using the improved residual model with flexible threshold. Beneficial effect: This step optimizes the pose by establishing an improved residual model with flexible threshold, which allows a principled compromise between real-time performance and robustness, thereby reducing the pose estimation time and improving the pose estimation accuracy.

[0050] Step 4: Use the improved multi-view geometry algorithm to construct the SLAM map for the pose optimization model obtained in step 3. Step 4 specifically includes the following process:

[0051] Step 4.1: Set the current frame F t and the corresponding depth map D t As input, point cloud cluster pixels are projected into 3D space to construct a geometric point cloud representation, thereby building a SLAM map of the substation scene;

[0052] Step 4.2: For each point cloud pixel position p, from the depth map D t Extract its depth D t (p), projecting the depth using the intrinsic parameters in K and π-1 of the point cloud cluster, converting the point cloud pixel position p to its associated 3D point X in metric coordinates relative to the center of the sensorp ;

[0053] Step 4.3: By repeating this operation for all point cloud pixel positions p, the 3D point cloud of the substation scene geometry is constructed. The specific formula is expressed as X p =D t (p)*(k-1)*(π-1)(p). The final result is a SLAM map of the substation scene. Beneficial effects: This step uses an improved multi-view geometry method for more accurate motion segmentation, which can eliminate the dynamic part in the current frame and point cloud, thereby eliminating the influence of dynamic pedestrians and cars in the substation scene modeling process and improving the efficiency and accuracy of mapping.

[0054] When this embodiment is actually used, a point cloud pixel segmentation algorithm is used in multiple point cloud frames, and the result it generates lacks spatial correlation of the entire area, which easily introduces noise and reduces the accuracy of scene segmentation. In contrast, the present invention generates many point cloud clusters, and each point cloud cluster can be regarded as a rigid body generated by converting non-rigid pixel-level point cloud data into a cluster-level rigid scene. In this way, false detection caused by single-point measurement can be avoided, the complexity of sensor motion segmentation can be greatly reduced, and the accuracy of sensor pose estimation can be improved. Through optical flow consistency sensor pose estimation, the sensor pose estimation from the initial frame to the current frame will be established, which helps to exclude dynamic objects during the self-motion calculation of the sensor, thereby eliminating the error of pose estimation caused by sensor motion, and then eliminating motion noise. At the same time, it can also improve sensor pose estimation and improve the speed of SLAM mapping of substation scenes. By establishing an improved residual model with a flexible threshold to optimize the pose, it can allow a principled compromise between real-time and robustness, thereby reducing the time of pose estimation and improving the accuracy of pose estimation. The improved multi-view geometry method is used for more accurate motion segmentation, which can eliminate the dynamic parts in the current frame and point cloud, thereby eliminating the influence of dynamic pedestrians and cars in the substation scene modeling process and improving the efficiency and accuracy of mapping.

[0055] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A dynamic SLAM method for substation modeling based on improved multi-view geometry, characterized in that: include: Step 1: Perform spatial clustering on the point cloud data of the sensor to obtain point cloud cluster data; Step 2: Perform optical flow consistency sensor pose estimation on the point cloud cluster data obtained in step 1; Step 3: Based on the sensor pose estimation obtained in step 2, an improved residual model with a flexible threshold is established to optimize the sensor pose; Step 4: Use the improved multi-view geometry algorithm to construct the SLAM map for the pose optimization model obtained in step 3.

2. A substation modeling dynamic SLAM method with improved multi-view geometry as claimed in claim 1, characterized in that: Step 1 specifically includes the following process: Step 1.1: Receive the depth point cloud data from the sensor as input. The depth point cloud data contains the depth value D(p) of each point cloud pixel. Initialize the K cluster centroids u1,u2,.....u in the corresponding 3D point cloud randomly. k Initially, each centroid u k Represents the average depth value of the point cloud cluster in 3D space; Step 1.2: Each pixel p of the point cloud data is assigned to the cluster with the nearest centroid by calculating p and each centroid u k The squared Euclidean distance d(p,k) between them assigns the point cloud pixel p to cluster C. p , the distribution formula is: C p =argminD(p)-u k 2 ; Step 1.3: Take the centroid u k The update step is to transform the centroid u of each cluster k Recalculated as the average depth of all points assigned to the cluster, the specific formula is: k =kD(p)N k ; That is, the centroid u of each cluster k is updated to the average of the depth D(p) at all pixels p, where the cluster assignment C p Equal to k, N k Represents the number of points assigned to cluster K. The algorithm iterates between the above two steps until convergence, thereby completing the clustering of the point cloud and finally obtaining the point cloud cluster data of the sensor.

3. A substation modeling dynamic SLAM method with improved multi-view geometry as claimed in claim 2, characterized in that: Step 2 specifically includes the following process: Step 2.1: The optical flow F∈R of the two frames I1 and I2 before and after the point cloud cluster data in step 1 W×H×2 Describes the point cloud pixel displacement, which aligns the first frame with the second frame, that is, at the point cloud pixel (x, y), according to the optical flow consistency, I1(x, y) = I2(x + F(x, y)x + F(x, y)y); Step 2.2: Use the predicted shape code e of frame i i , and use a differentiable variant of marching cubes to extract the underlying grid from it. The specific formula can be expressed as M i =(V i ,T i ); Among them, V i Represents the 3D vertices of the mesh, T i Represents its 3D triangle; Step 2.3: Convert the mesh into a 3D scene to get the predicted sensor pose. The specific formula is: Where, represents the rotation matrix of the sensor; Step 2.4: Get the sensor's position and posture by projecting each vertex. The specific formula is: Among them, represents the intrinsic matrix of the sensor; finally, the pose estimation of the sensor is obtained.

4. A substation modeling dynamic SLAM method with improved multi-view geometry as claimed in claim 3, characterized in that: Step 3 includes the following process: Step 3.1: Drive the threshold T by taking the best data c and multi-frame geometric threshold T g Thereby optimizing the posture; Among them, the optimal data-driven threshold T c There are two unique peaks corresponding to the residuals in the static and dynamic regions. We apply Otsu to first optimize the best data-driven distribution pose and use the result to automatically calculate T c ; During the optimization process, T c and multi-frame geometric threshold T g , and estimated using the geometric persistence technique; Step 3.2: Calculate the optimal data-driven threshold T using the Otsu method c , the specific formula is expressed as This automatically divides the distribution into static and dynamic components, minimizing the pose change, and the multi-frame geometric threshold T g is calculated as the 99th percentile displacement amplitude over a longer observation window, thus providing temporal stability, and is expressed as The final sensor pose T combines the two thresholds into a weighted sum, which is expressed as T = αT c +(1-α)T g .

5. A substation modeling dynamic SLAM method with improved multi-view geometry as claimed in claim 4, characterized in that: Step 4 specifically includes the following process: Step 4.1: Set the current frame F t and the corresponding depth map D t As input, point cloud cluster pixels are projected into 3D space to construct a geometric point cloud representation, thereby building a SLAM map of the substation scene; Step 4.2: For each point cloud pixel position p, from the depth map D t Extract its depth D t (p), projecting the depth using the intrinsic parameters in K and π-1 of the point cloud cluster, converting the point cloud pixel position p to its associated 3D point X in metric coordinates relative to the center of the sensor p ; Step 4.3: By repeating this operation for all point cloud pixel positions p, the 3D point cloud of the substation scene geometry is constructed. The specific formula is expressed as X p =D t (p)*(k-1)*(π-1)(p).

Images