Three-dimensional reconstruction error compensation method, device and equipment of transformer substation, storage medium

Abstract

This application relates to a method, apparatus, equipment, and storage medium for error compensation in the 3D reconstruction of substations. The method includes: simultaneously acquiring spatially consistent LiDAR point clouds and binocular images; processing the binocular images using a stereo matching algorithm to generate a stereo-matched point cloud; fusing the LiDAR point cloud and the stereo-matched point cloud to obtain a fused point cloud; then, combining the geometric features of the slender target object and the color information of the binocular images, accurately extracting the target point cloud of this type of target object from the fused point cloud; finally, fitting the target point cloud to a geometric model representing its geometric shape, calculating the deviation between the target point cloud and the geometric model, and if the deviation does not meet a preset convergence condition, adjusting the positions of some points in the target point cloud or the parameters of the geometric model, iterating repeatedly until the deviation meets the preset convergence condition, thus completing the 3D reconstruction of the slender target object in the substation. This method can effectively improve the accuracy and reliability of the 3D reconstruction of slender target objects in substations.

Description

Technical Field

[0001] This application relates to the field of three-dimensional reconstruction technology of engineering structures, and in particular to a method, device, computer equipment, computer-readable storage medium and computer program product for three-dimensional reconstruction error compensation of substations. Background Technology

[0002] With the development of 3D reconstruction technology for engineering structures, LiDAR technology has emerged. By actively emitting laser pulses and measuring the reflection time, it can generate high-precision 3D point clouds and obtain accurate depth information of the environment, and its performance is unaffected by lighting conditions. However, when LiDAR acquires data on small targets (such as busbars and insulator strings), the laser beam has a certain divergence angle, which may cause the laser pulse to fail to effectively hit the target. This results in poor reconstruction accuracy due to missing or incomplete point clouds of the target equipment. Summary of the Invention

[0003] Therefore, it is necessary to provide a method, device, computer equipment, computer-readable storage medium, and computer program product for three-dimensional reconstruction error compensation of substations that can meet the high-precision requirements of the above-mentioned technical problems.

[0004] Firstly, this application provides a method for compensating for errors in the three-dimensional reconstruction of a substation, including:

[0005] Acquire lidar point cloud and binocular images of slender targets within a substation; wherein the lidar and binocular camera maintain spatial consistency during data acquisition.

[0006] Stereo matching algorithms are used to process binocular images to generate stereo matching point clouds containing depth information.

[0007] The lidar point cloud and the stereo matching point cloud are transformed into the same coordinate system and then fused to obtain the fused point cloud.

[0008] By combining the geometric features of the slender target object with color information from the binocular images, the target point cloud of the slender target object is extracted from the fused point cloud;

[0009] The target point cloud is fitted to a geometric model used to characterize the geometry of the slender target object. The deviation between the target point cloud and the geometric model is calculated. If the deviation does not meet the preset convergence condition, the positions of some points in the target point cloud or the parameters of the geometric model are adjusted until the deviation meets the preset convergence condition.

[0010] In one embodiment, a stereo matching algorithm is used to process the binocular images to generate a stereo matching point cloud containing depth information, including:

[0011] By projecting the portion of the LiDAR point cloud that overlaps with the field of view of the binocular camera onto the image plane of the binocular image, the depth prior information of the image region corresponding to the slender target object is obtained.

[0012] Based on depth prior information, the disparity search range of stereo matching is constrained. According to the disparity search range, the disparity map of the binocular image is calculated using the stereo matching algorithm.

[0013] By combining the intrinsic and extrinsic parameters of the binocular camera and the baseline length, the disparity map is converted into a stereo matching point cloud containing depth information.

[0014] In one embodiment, the lidar point cloud and the stereo matching point cloud are transformed to the same coordinate system and fused to obtain a fused point cloud, including:

[0015] The three-dimensional space is divided into voxel grids, and the lidar points and stereo matching points falling in each voxel grid are determined.

[0016] Based on the measurement uncertainty of each point in the lidar point cloud and the stereo matching point cloud, the corresponding weight of each point is determined; where the smaller the measurement uncertainty, the greater the weight; the weight of the lidar point is greater than the weight of the stereo matching point in the same voxel grid.

[0017] The three-dimensional information of each voxel grid is determined based on the weights and three-dimensional coordinates of the lidar points and stereo matching points that fall into each voxel grid.

[0018] The 3D information of all voxel meshes is integrated to obtain a fused point cloud.

[0019] In one embodiment, the target point cloud of the elongated object is extracted from the fused point cloud by combining the geometric features of the elongated object and color information from the binocular images, including:

[0020] For each point in the fused point cloud, the three-dimensional coordinates, normal vector, curvature, and associated RGB color values ​​of each point are extracted by combining the geometric features of the slender target object and the color information from the binocular images, and a multi-dimensional feature vector of each point is constructed.

[0021] The points in the fused point cloud are clustered based on the multidimensional feature vector of each point. Clusters that meet the geometric and color features of the slender target object are selected and used as the target point cloud.

[0022] In one embodiment, the geometric model is a catenary model; fitting the target point cloud to the geometric model used to characterize the geometry of the slender target object includes:

[0023] The target point cloud is denoised and then projected onto a vertical plane containing the slender target object.

[0024] Based on the catenary equation, the initial parameters of the catenary model are estimated. The catenary equation includes the sag coefficient and the coordinate parameters of the lowest point.

[0025] The parameters of the catenary model are iteratively optimized until the sum of squared distances between the catenary model and the target point cloud meets the preset fitting accuracy requirements, thus completing the model fitting of the target point cloud.

[0026] In one embodiment, the deviation between the target point cloud and the geometric model is calculated. If the deviation does not meet a preset convergence condition, the positions of some points in the target point cloud or the parameters of the geometric model are adjusted until the deviation meets the preset convergence condition, including:

[0027] Calculate the orthogonal distance from each point in the target point cloud to the geometric model, identify outliers whose orthogonal distance exceeds a preset threshold, and adjust the position of the outliers in the target point cloud or the parameters of the geometric model until the orthogonal distance from each point in the target point cloud to the geometric model does not exceed the preset threshold in two consecutive tests.

[0028] Secondly, this application also provides a three-dimensional reconstruction error compensation device for a substation, comprising:

[0029] The acquisition module is used to acquire lidar point clouds and binocular images of slender target objects within the substation; wherein, the lidar and the binocular camera have spatial consistency when acquiring data;

[0030] The processing module is used to process the stereo images using a stereo matching algorithm to generate a stereo matching point cloud containing depth information;

[0031] The fusion module is used to transform the lidar point cloud and the stereo matching point cloud into the same coordinate system for fusion, resulting in a fused point cloud.

[0032] The extraction module is used to extract the target point cloud of the slender target object from the fused point cloud by combining the geometric features of the slender target object and the color information from the binocular image.

[0033] The adjustment module is used to fit the target point cloud to a geometric model used to characterize the geometry of a slender target object, calculate the deviation between the target point cloud and the geometric model, and adjust the position of some points in the target point cloud or the parameters of the geometric model if the deviation does not meet the preset convergence conditions, until the deviation meets the preset convergence conditions.

[0034] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0035] Acquire lidar point cloud and binocular images of slender targets within a substation; wherein the lidar and binocular camera maintain spatial consistency during data acquisition.

[0036] Stereo matching algorithms are used to process binocular images to generate stereo matching point clouds containing depth information.

[0037] The lidar point cloud and the stereo matching point cloud are transformed into the same coordinate system and then fused to obtain the fused point cloud.

[0038] By combining the geometric features of the slender target object with color information from the binocular images, the target point cloud of the slender target object is extracted from the fused point cloud;

[0039] The target point cloud is fitted to a geometric model used to characterize the geometry of the slender target object. The deviation between the target point cloud and the geometric model is calculated. If the deviation does not meet the preset convergence condition, the positions of some points in the target point cloud or the parameters of the geometric model are adjusted until the deviation meets the preset convergence condition.

[0040] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0041] Acquire lidar point cloud and binocular images of slender targets within a substation; wherein the lidar and binocular camera maintain spatial consistency during data acquisition.

[0042] Stereo matching algorithms are used to process binocular images to generate stereo matching point clouds containing depth information.

[0043] The lidar point cloud and the stereo matching point cloud are transformed into the same coordinate system and then fused to obtain the fused point cloud.

[0044] By combining the geometric features of the slender target object with color information from the binocular images, the target point cloud of the slender target object is extracted from the fused point cloud;

[0045] The target point cloud is fitted to a geometric model used to characterize the geometry of the slender target object. The deviation between the target point cloud and the geometric model is calculated. If the deviation does not meet the preset convergence condition, the positions of some points in the target point cloud or the parameters of the geometric model are adjusted until the deviation meets the preset convergence condition.

[0046] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0047] Acquire lidar point cloud and binocular images of slender targets within a substation; wherein the lidar and binocular camera maintain spatial consistency during data acquisition.

[0048] Stereo matching algorithms are used to process binocular images to generate stereo matching point clouds containing depth information.

[0049] The lidar point cloud and the stereo matching point cloud are transformed into the same coordinate system and then fused to obtain the fused point cloud.

[0050] By combining the geometric features of the slender target object with color information from the binocular images, the target point cloud of the slender target object is extracted from the fused point cloud;

[0051] The target point cloud is fitted to a geometric model used to characterize the geometry of the slender target object. The deviation between the target point cloud and the geometric model is calculated. If the deviation does not meet the preset convergence condition, the positions of some points in the target point cloud or the parameters of the geometric model are adjusted until the deviation meets the preset convergence condition.

[0052] The aforementioned substation 3D reconstruction error compensation method, device, computer equipment, computer-readable storage medium, and computer program product first synchronously acquire spatially consistent LiDAR point clouds and binocular images. The binocular images are then processed using a stereo matching algorithm to generate a stereo matching point cloud carrying depth information. Subsequently, the LiDAR point cloud and the stereo matching point cloud are transformed to the same coordinate system and fused to obtain a fused point cloud. Then, combining the geometric features of slender target objects and the color information of the binocular images, the target point cloud of this type of target object is accurately extracted from the fused point cloud. By fusing the LiDAR point cloud and the stereo matching point cloud, the advantages of different sensor point clouds are fully utilized, effectively improving the overall quality of the point cloud and laying a solid data foundation for the accurate extraction of slender target objects in the substation. Simultaneously, combining the geometric features of the slender target and the color information of the binocular images to extract the target point cloud significantly improves the segmentation accuracy between the target point cloud and the background, and reduces interference from non-target point clouds. Finally, the target point cloud is fitted to a geometric model representing its geometric shape. The deviation between the target point cloud and the geometric model is calculated. If the deviation does not meet the preset convergence condition, the positions of some points in the target point cloud or the parameters of the geometric model are adjusted. This process is repeated iteratively until the deviation meets the preset convergence condition, thus completing the 3D reconstruction of the slender target object in the substation. By fitting the geometric model and iteratively adjusting based on the deviation, high-precision matching between the target point cloud and the geometric model is achieved. The final model can accurately represent the actual geometric shape of the target object in the substation, effectively improving the accuracy and reliability of the 3D reconstruction of the slender target object in the substation. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is an application environment diagram of a three-dimensional reconstruction error compensation method for a substation in one embodiment;

[0055] Figure 2 This is a flowchart illustrating a three-dimensional reconstruction error compensation method for a substation in one embodiment.

[0056] Figure 3 This is a structural block diagram of a three-dimensional reconstruction error compensation device for a substation in one embodiment;

[0057] Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0060] The three-dimensional reconstruction error compensation method for substations provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed in the cloud or on other network servers. First, spatially consistent LiDAR point clouds and binocular images are simultaneously acquired. The binocular images are processed using a stereo matching algorithm to generate a stereo matching point cloud carrying depth information. Then, the LiDAR point cloud and the stereo matching point cloud are transformed to the same coordinate system and fused to obtain a fused point cloud. Next, combining the geometric features of the slender target object and the color information of the binocular images, the target point cloud of this type of target object is accurately extracted from the fused point cloud. Finally, the target point cloud is fitted to a geometric model representing its geometric shape, and the deviation between the target point cloud and the geometric model is calculated. If the deviation does not meet the preset convergence condition, the positions of some points in the target point cloud or the parameters of the geometric model are adjusted, and the process is iterated repeatedly until the deviation meets the preset convergence condition, completing the three-dimensional reconstruction of the slender target object in the substation.

[0061] Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0062] In one exemplary embodiment, such as Figure 2 As shown, a method for three-dimensional reconstruction error compensation of a substation is provided, which is then applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 202 to 210. Wherein:

[0063] Step 202: Acquire lidar point cloud and binocular image of slender target object in substation; wherein, lidar and binocular camera have spatial consistency when acquiring data;

[0064] Among them, the lidar point cloud refers to the collection of a large number of three-dimensional spatial points obtained after scanning the scene within the substation by lidar equipment. The binocular image refers to two two-dimensional digital images simultaneously acquired by the left and right cameras of a binocular camera; it is the raw data for generating the stereo matching point cloud. The binocular camera can obtain the depth information of the scene by calculating the disparity between the two cameras. Spatial consistency is a key constraint for the data acquisition of lidar and binocular cameras, meaning that the lidar and binocular cameras are in the same spatiotemporal acquisition state when acquiring data. This has two meanings: first, temporal consistency, where the two types of sensors trigger acquisition synchronously, avoiding target position deviations caused by acquisition time differences in static substation scenes; second, spatial consistency, where the coordinate systems of the two types of sensors can be accurately transformed after extrinsic parameter calibration, and the acquisition field of view covers the same area within the substation, ensuring that the lidar point cloud and the binocular image represent the same physical space scene.

[0065] Optionally, image processing techniques or deep learning models are applied to the binocular images to detect the two-dimensional position of the slender target object, and the two-dimensional position is used as prior information for subsequent stereo matching point cloud generation or target point cloud extraction.

[0066] Two-dimensional object detection methods mainly fall into two categories. The first category comprises traditional image processing techniques, including edge detection: Generatrices typically appear as distinct edges in an image. Edge detection operators such as Canny and Sobel can be used to extract image edges. Hough transform: For generatrices that are approximately straight, Hough line detection is an effective method. For curved generatrices, generalized Hough transform or piecewise line approximation can be used. Morphological operations: Operations such as dilation, erosion, and skeletonization are performed on edge images to connect broken line segments, remove noise, and thin lines. Region growing / segmentation: If the generatrice color contrasts significantly with the background, region growing or threshold-based segmentation methods can be used. The second category comprises deep learning models, including object detection models such as YOLO (You Only Look Once), SSD (Single Shot MultiBoxDetector), and Faster R-CNN. These models can be trained to directly detect the location of generatrices in an image (outputting bounding boxes). Semantic segmentation / instance segmentation models, such as U-Net, Mask R-CNN, and DeepLab, can perform pixel-level segmentation of images, classifying each pixel as either a bus line or background, thereby obtaining an accurate mask of the bus line in the image.

[0067] The use of prior information, with the detected two-dimensional position of the bus line in the image (pixel coordinates, line segments, bounding boxes, pixel masks, etc.) as a strong constraint, guides subsequent point cloud processing steps in two main dimensions: First, when guiding the generation of stereo matching point clouds, the matching area is limited, with dense stereo matching only performed at the two-dimensional position of the bus line and its adjacent areas, while the background area is sparsely matched or not matched, saving computational resources and reducing the introduction of background noise; second, the matching parameters are optimized, and specific stereo matching parameters or algorithms are adapted for the weak texture characteristics of the bus line area to improve the matching accuracy of the bus line area; third, the disparity map post-processing is optimized, using the two-dimensional mask of the bus line to smooth and correct the disparity value of the bus line area, eliminating noise caused by incorrect matching, and ensuring the quality of the stereo matching point cloud. When guiding target point cloud extraction, firstly, the search range of the 3D point cloud is narrowed by using points in the fused point cloud that fall within the 2D detection area of ​​the bus line after being projected onto the image as candidate points, which significantly reduces the number of points to be processed and improves extraction efficiency and accuracy. Secondly, point cloud feature enhancement is achieved by using "whether the point is located within the detection area of ​​the bus line in the image after projection" as a new feature, which is combined with the geometric and color features of the point cloud for point cloud clustering or classification, enhancing the feature distinction between the target points on the bus line and the background points, and further improving the accuracy of target point cloud extraction.

[0068] Step 204: Process the stereo image using a stereo matching algorithm to generate a stereo matching point cloud containing depth information;

[0069] Stereo matching algorithms, which rely on two 2D images (left and right) acquired by a stereo camera, calculate the disparity between corresponding pixels in the images to solve for the 3D depth information of the scene. This enables the conversion from 2D images to 3D depth and is the core algorithm for generating stereo matching point clouds. Commonly used stereo matching algorithms include local matching algorithms (such as SAD, SSD, and NCC) or global / semi-global matching algorithms (such as dynamic programming, graph cut, and SGM). Depth information refers to the perpendicular distance from points on the surface of objects in the scene to the imaging plane of the stereo camera. It is the core spatial information for constructing 3D point clouds and is calculated by the stereo matching algorithm using pixel disparity combined with camera calibration parameters. A stereo matching point cloud is a set of 3D point clouds formed by converting the image pixel depth information calculated by the stereo matching algorithm into 3D spatial coordinates using pixel coordinates and camera intrinsic parameters.

[0070] Step 206: Convert the lidar point cloud and the stereo matching point cloud to the same coordinate system and fuse them to obtain the fused point cloud;

[0071] Among them, the fused point cloud is a new three-dimensional point cloud set obtained by unifying the coordinates and fusing the LiDAR point cloud and the stereo matching point cloud. It combines the advantages of the high depth and precision of the LiDAR point cloud with the advantages of the density and color information of the stereo matching point cloud.

[0072] Step 208: Combine the geometric features of the slender target object with the color information from the binocular images to extract the target point cloud of the slender target object from the fused point cloud;

[0073] Among them, slender target objects specifically refer to electrical equipment in substations that extend linearly or in a curved spatial shape and whose cross-sectional dimensions are much smaller than their length, with the core being busbars, flexible conductors, jumpers, insulator strings, etc.

[0074] Specifically, the original values, geometric features (normal vector, curvature), and RGB values ​​of each point in the point cloud data are read and concatenated into a feature vector. The feature vectors of all points are then input into the DBSCAN clustering algorithm, and the generalized calculation formula for "total distance" is redefined (total distance = spatial distance + weight coefficient 1 × color difference + weight coefficient 2 × normal vector difference). After the algorithm runs, it outputs several clusters. Because the parent line has specific color and linear characteristics, points that match these characteristics are automatically clustered into one class and thus extracted.

[0075] Step 210: Fit the target point cloud to the geometric model used to characterize the geometry of the slender target object, calculate the deviation between the target point cloud and the geometric model, and adjust the position of some points in the target point cloud or the parameters of the geometric model if the deviation does not meet the preset convergence condition until the deviation meets the preset convergence condition.

[0076] Among them, deviation is an index that quantifies the degree of fit between the target point cloud and the geometric model in spatial position. It refers to the spatial distance from the three-dimensional point in the target point cloud to the geometric model. Common forms of expression include single-point orthogonal distance, sum of squared distances from all points to the model, and average distance.

[0077] Optionally, the target point cloud of the bus line can be fitted to the catenary model. Three types of parameter estimation algorithms are commonly used: the least squares method minimizes the sum of squared distances from the point cloud to the model, and combines numerical optimization methods such as gradient descent and Gauss-Newton method to solve for the optimal parameters, which is suitable for high-quality point clouds with less noise; the RANSAC algorithm fits the initial model by randomly sampling a subset of the point cloud and iteratively verifies and selects the optimal model, which is more robust and suitable for point clouds with a lot of noise or outliers; in addition, other robust estimation methods such as M-estimation can also be used.

[0078] Fitting process: It is executed in three orderly steps. First, the target point cloud is preprocessed to perform operations such as noise reduction and sorting along the direction of the traverse. Second, the initial parameters of the model are estimated by the geometric features of the point cloud (such as the approximate lowest point and span) to accelerate the subsequent optimization process. Finally, the selected parameter estimation algorithm is executed to iteratively update the model parameters until the convergence condition is met and the parameter optimization is completed.

[0079] In the aforementioned substation 3D reconstruction error compensation method, firstly, spatially consistent LiDAR point clouds and stereo images are simultaneously acquired. The stereo images are then processed using a stereo matching algorithm to generate a stereo matching point cloud carrying depth information. Subsequently, the LiDAR point cloud and the stereo matching point cloud are transformed to the same coordinate system and fused to obtain a fused point cloud. Then, combining the geometric features of slender targets and the color information from the stereo images, the target point cloud of this type of target object is accurately extracted from the fused point cloud. By fusing the LiDAR point cloud and the stereo matching point cloud, the advantages of different sensor point clouds are fully utilized, effectively improving the overall quality of the point cloud and laying a solid data foundation for the accurate extraction of slender targets in the substation. Simultaneously, combining the geometric features of the slender target with the color information from the stereo images to extract the target point cloud significantly improves the segmentation accuracy between the target point cloud and the background, reducing interference from non-target point clouds. Finally, the target point cloud is fitted to a geometric model representing its geometric shape. The deviation between the target point cloud and the geometric model is calculated. If the deviation does not meet the preset convergence condition, the positions of some points in the target point cloud or the parameters of the geometric model are adjusted. This process is repeated iteratively until the deviation meets the preset convergence condition, thus completing the 3D reconstruction of the slender target object in the substation. By fitting the geometric model and iteratively adjusting based on the deviation, high-precision matching between the target point cloud and the geometric model is achieved. The final model can accurately represent the actual geometric shape of the target object in the substation, effectively improving the accuracy and reliability of the 3D reconstruction of the slender target object in the substation.

[0080] In one embodiment, a stereo matching algorithm is used to process the binocular images to generate a stereo matching point cloud containing depth information, including:

[0081] By projecting the portion of the LiDAR point cloud that overlaps with the field of view of the binocular camera onto the image plane of the binocular image, the depth prior information of the image region corresponding to the slender target object is obtained.

[0082] Based on depth prior information, the disparity search range of stereo matching is constrained. According to the disparity search range, the disparity map of the binocular image is calculated using the stereo matching algorithm.

[0083] By combining the intrinsic and extrinsic parameters of the binocular camera and the baseline length, the disparity map is converted into a stereo matching point cloud containing depth information.

[0084] Specifically, when generating stereo matching point clouds, the depth prior information of the LiDAR point cloud is introduced to achieve precise constraints for stereo matching. First, the part of the LiDAR point cloud that overlaps with the field of view of the binocular camera is projected onto the binocular image plane to extract the depth prior information of the image region corresponding to the slender target of the substation. Then, this information is used to limit the disparity search range of stereo matching. Based on this range, the disparity map of the binocular image is calculated by the stereo matching algorithm. Finally, the disparity map is converted into a stereo matching point cloud containing depth information by combining the intrinsic and extrinsic parameters of the binocular camera and the baseline length, thus completing the point cloud generation.

[0085] In the above embodiments, the high-precision depth prior of the LiDAR is projected onto the image plane and the disparity search range is constrained, which significantly reduces the computational load of stereo matching, saves computational resources, and avoids matching errors caused by invalid disparity searches. The depth constraint on the slender target region makes the stereo matching algorithm more accurate in weak texture regions such as the generatrix, effectively reducing the probability of false matching. The constrained disparity map is converted into a point cloud by combining camera parameters, and the depth information of the generated stereo matching point cloud is more accurate, which can accurately compensate for the problem of missing point clouds of small targets by the LiDAR.

[0086] In one embodiment, the lidar point cloud and the stereo matching point cloud are transformed to the same coordinate system and fused to obtain a fused point cloud, including:

[0087] The three-dimensional space is divided into voxel grids, and the lidar points and stereo matching points falling in each voxel grid are determined.

[0088] Based on the measurement uncertainty of each point in the lidar point cloud and the stereo matching point cloud, the corresponding weight of each point is determined; where the smaller the measurement uncertainty, the greater the weight; the weight of the lidar point is greater than the weight of the stereo matching point in the same voxel grid.

[0089] The three-dimensional information of each voxel grid is determined based on the weights and three-dimensional coordinates of the lidar points and stereo matching points that fall into each voxel grid.

[0090] The 3D information of all voxel meshes is integrated to obtain a fused point cloud.

[0091] Specifically, a voxel grid weighted fusion method is used to fuse LiDAR point clouds and stereo matching point clouds. First, the three-dimensional space is divided into voxel grids and the two types of point clouds falling within each grid are identified. Then, a weight is assigned to each point based on the measurement uncertainty, following the principle that the smaller the measurement uncertainty, the greater the weight, and setting the weight of LiDAR points to be greater than that of stereo matching points within the same voxel. Subsequently, the three-dimensional information of the voxel is calculated by combining the weights of the two types of point clouds within each voxel with the three-dimensional coordinates. Finally, the three-dimensional information of all voxel grids is integrated to obtain the fused point cloud.

[0092] In the above embodiments, the precise integration of two types of point clouds is achieved through the voxel grid weighted fusion technique, effectively solving the problems of spatial position deviation and poor fusion effect caused by the lack of differentiation of point cloud precision in traditional point cloud fusion, and improving the overall accuracy and reliability of the fused point cloud: dividing the three-dimensional space into voxel grids realizes the spatial regularization of point clouds, avoiding the spatial misalignment problem of direct fusion of discrete point clouds; assigning weights based on measurement uncertainty and giving higher weights to LiDAR points fully matches the higher depth accuracy of LiDAR point clouds, making the fusion result more consistent with the characteristics of high-precision data, while weakening the influence of points with large measurement errors; through the weighted calculation and overall integration of voxel grids, the advantages of the two types of point clouds are complemented, and the fused point cloud retains the high precision of LiDAR point clouds while also having the density of stereo matching point clouds, effectively making up for the problem of missing small target point clouds in LiDAR.

[0093] In one embodiment, the target point cloud of the elongated object is extracted from the fused point cloud by combining the geometric features of the elongated object and color information from the binocular images, including:

[0094] For each point in the fused point cloud, the three-dimensional coordinates, normal vector, curvature, and associated RGB color values ​​of each point are extracted by combining the geometric features of the slender target object and the color information from the binocular images, and a multi-dimensional feature vector of each point is constructed.

[0095] The points in the fused point cloud are clustered based on the multidimensional feature vector of each point. Clusters that meet the geometric and color features of the slender target object are selected and used as the target point cloud.

[0096] Specifically, when extracting the target point cloud of a slender object from the fused point cloud, for each point in the fused point cloud, the three-dimensional coordinates, normal vector, curvature, and associated RGB color values ​​of the point are extracted by combining the geometric features of the slender object with the color information of the binocular image. A multi-dimensional feature vector is constructed for each point. Then, based on the multi-dimensional feature vector, clustering processing is performed on the fused point cloud to select clusters that simultaneously meet the geometric and color features of the slender object, and these clusters are used as the target point cloud.

[0097] In the above embodiments, geometric features and color features are combined to construct a multi-dimensional feature vector, which makes up for the lack of distinguishability of a single feature and makes the feature differences between the slender target points and the background points more significant. Based on the clustering analysis of the multi-dimensional feature vector, the point set that matches the geometric and color features of the slender target can be accurately screened, effectively eliminating the interference of background point clouds such as towers, vegetation, and ground, and the extracted target point cloud has higher purity. The entire extraction process relies on feature quantization and algorithm clustering, which does not require manual intervention, has higher extraction efficiency, and provides a high-quality target point cloud data foundation for subsequent model fitting and parameter optimization, ensuring the overall accuracy of the subsequent 3D reconstruction of slender targets in substations.

[0098] In one embodiment, the geometric model is a catenary model; fitting the target point cloud to the geometric model used to characterize the geometry of the slender target object includes:

[0099] The target point cloud is denoised and then projected onto a vertical plane containing the slender target object.

[0100] Based on the catenary equation, the initial parameters of the catenary model are estimated. The catenary equation includes the sag coefficient and the coordinate parameters of the lowest point.

[0101] The parameters of the catenary model are iteratively optimized until the sum of squared distances between the catenary model and the target point cloud meets the preset fitting accuracy requirements, thus completing the model fitting of the target point cloud.

[0102] Specifically, firstly, RANSAC is used to fit the projection line of the point cloud onto the horizontal plane to determine the vertical plane where the traverse lies, and then the 3D point cloud is... Projected onto this plane, it is converted into two-dimensional coordinates. Select a two-dimensional point set The point with the smallest value (height) is taken as the vertex of the catenary. The initial guess values ​​are obtained; the objective function is defined as the sum of squared residuals from all points to the catenary equation; the initial guess values ​​and the objective function are input into the Levenberg-Marquardt algorithm, and the parameters are iteratively updated until the sum of squared residuals converges to the minimum value.

[0103] In the above embodiments, point cloud denoising processing eliminates noise interference with fitting in advance, and vertical plane projection simplifies the fitting process while ensuring fitting specificity, allowing the model to fit based on high-quality, highly relevant point cloud data. Initial parameters are estimated based on the catenary equation and iteratively optimized to a preset accuracy, enabling the fitted model to accurately fit the true physical shape of the flexible conductor under gravity, especially accurately restoring its sag characteristics, providing precise geometric basis for substation safety clearance distance assessment. Furthermore, the iterative optimization method, where the sum of squared distances meets accuracy requirements, results in a higher degree of fit between the model and the target point cloud. The final parameterized catenary model not only accurately represents the geometric shape of slender targets but also outputs physically meaningful model parameters.

[0104] In one embodiment, the deviation between the target point cloud and the geometric model is calculated. If the deviation does not meet a preset convergence condition, the positions of some points in the target point cloud or the parameters of the geometric model are adjusted until the deviation meets the preset convergence condition, including:

[0105] Calculate the orthogonal distance from each point in the target point cloud to the geometric model, identify outliers whose orthogonal distance exceeds a preset threshold, and adjust the position of the outliers in the target point cloud or the parameters of the geometric model until the orthogonal distance from each point in the target point cloud to the geometric model does not exceed the preset threshold in two consecutive tests.

[0106] Specifically, when optimizing the matching accuracy between the target point cloud and the geometric model, the orthogonal distance from each point in the target point cloud to the geometric model is first calculated to identify outliers whose orthogonal distance exceeds a preset threshold. Then, the positions of the outliers in the target point cloud or the parameters of the geometric model are adjusted accordingly. This calculation and adjustment operation is repeated until the orthogonal distance from all points in the target point cloud to the geometric model does not exceed the preset threshold in two consecutive calculations, thus satisfying the preset convergence condition.

[0107] In the above embodiments, the deviation is quantified by orthogonal distance, making the deviation judgment between the point cloud and the model more accurate and objective. The outlier identification based on the preset threshold can quickly locate abnormal points in the point cloud. By adjusting the position of outliers or optimizing the iteration method of model parameters, the deviation problem is specifically corrected. Furthermore, the convergence condition is that there are no points exceeding the threshold twice in a row, avoiding the randomness of fitting caused by a single achievement, thus ensuring the stability and reliability of model fitting. Ultimately, the geometric model can more accurately represent the true geometric shape of the slender target of the substation.

[0108] In one embodiment, a method for compensating for errors in the three-dimensional reconstruction of a substation in a specific embodiment includes:

[0109] S1: Acquire lidar point cloud and binocular images of slender target objects within the substation.

[0110] For slender targets in a substation environment, such as busbars, conductors, and insulator strings, it is first necessary to acquire their three-dimensional spatial and visual information. Specifically, this is achieved by integrating a lidar sensor and a binocular camera on the same platform using a ground-based mobile measurement system (such as an inspection robot) or a drone. The lidar sensor generates sparse but high-precision three-dimensional point cloud data (i.e., lidar point cloud) containing the target object and its surrounding environment by emitting laser pulses and receiving reflected signals. This point cloud data directly reflects the spatial coordinates of the object. Simultaneously, the binocular camera simultaneously captures the same scene from two different but known viewpoints, acquiring one or more pairs of binocular images with parallax information.

[0111] S1.1: Using drones or ground vehicles as mobile platforms, simultaneously acquire LiDAR point clouds and binocular images. Drones are suitable for large-scale, complex terrain areas inaccessible to humans, and can carry lightweight LiDAR and binocular cameras to collect data from the air along preset routes, offering advantages such as high efficiency, flexibility, and excellent observation angles. Ground vehicles are suitable for power corridors where road conditions permit, or for scenarios requiring higher point cloud density and ground detail. LiDAR and binocular cameras can be mounted on vehicles, inspection robots, and other equipment for data acquisition. Simultaneously, the LiDAR and binocular cameras must be rigidly integrated into the same mobile platform, achieving precise synchronization of their data acquisition. Synchronization methods include hardware synchronization and timestamp synchronization: Hardware synchronization uses hardware trigger signals such as GPS PPS second pulse signals and dedicated synchronization controllers to simultaneously trigger LiDAR scanning and binocular camera exposure; timestamp synchronization adds high-precision timestamps based on the same clock source or a precisely calibrated clock source to each LiDAR scan point / scan line and each frame image from the binocular camera, allowing for subsequent matching of LiDAR data and image data at the same moment based on the timestamps.

[0112] S1.2: Perform intrinsic and extrinsic parameter calibration on the lidar and the camera used to acquire binocular images to ensure spatial consistency of data from both sensors.

[0113] Regarding internal parameter calibration, the focal length of each camera in a stereo camera system needs to be calibrated. Main point Radial distortion parameters With tangential distortion parameters The calibration of external parameters is commonly accomplished using the Zhang Zhengyou checkerboard method, which also requires calibrating the baseline vectors and rotation relationships between the left and right cameras, as well as their relative poses. The internal scanning parameters of the LiDAR are generally calibrated by the manufacturer or are relatively fixed; for some types of LiDAR, internal parameter calibration is only required as needed. The core of external parameter calibration is solving for the relative rotation matrix and translation vector between the LiDAR coordinate system and the binocular camera coordinate system (usually with the left camera or virtual binocular center as a reference). There are two main calibration methods: one is calibration based on a specific calibration object, using a calibration board such as a checkerboard or circular array that can be clearly identified by both the LiDAR and the camera. Multiple sets of point clouds and images of the calibration board in different poses are collected, and the corresponding feature points are extracted. The external parameters are then obtained by solving the hand-eye calibration problem or optimizing the transformation parameters. The second method is self-calibration without a calibration object, utilizing the correspondences of natural features such as planes, lines, and corners in the scene, and estimating the external parameters through optimization algorithms.

[0114] S2: Generate stereo matching point cloud based on binocular images.

[0115] After acquiring binocular images, these image pairs are processed using a stereo matching algorithm. By performing correction, feature extraction, matching search, and disparity calculation on the binocular images, one or more dense disparity maps can be obtained. Subsequently, based on the intrinsic and extrinsic parameters of the binocular camera and the baseline length, the disparity map can be converted into a depth map, and further, the two-dimensional coordinates and depth values ​​of each pixel can be converted into three-dimensional spatial coordinates, thereby generating a dense three-dimensional point cloud with color and texture information, called a stereo matching point cloud.

[0116] S2.1: Project at least part of the lidar point cloud onto the image plane of the binocular image to obtain prior depth information of the slender target object in the corresponding image region.

[0117] After acquiring the lidar point cloud and stereo images, and completing sensor calibration, the portion of the lidar point cloud that overlaps with the stereo camera's field of view is selected. For these lidar points, using the calibrated camera intrinsic parameters and the extrinsic parameters between the lidar and the camera, their three-dimensional coordinates are projected onto the image plane of the left (or right, or both) image of the stereo image pair.

[0118] The projection process is as follows: Let the three-dimensional coordinates of a lidar point in the lidar coordinate system be... The transformation matrix between the lidar and the camera... (by rotation matrix) Translation vector (Construction), transforming it to three-dimensional coordinates in the camera coordinate system. Then, using the camera intrinsic parameter matrix... (Including focal length) and the main point ),Will Projected onto image pixel coordinates get For each LiDAR point projected onto the image, its depth value in the camera coordinate system. This constitutes the depth prior information for that pixel location. Since the depth measurement accuracy of LiDAR is usually high, the depth prior provided by these projected points is reliable. For slender targets such as busbars, although the LiDAR point cloud may be sparse, these sparse points can still provide depth information for key locations.

[0119] S2.2: Utilize depth prior information to guide the stereo matching process for binocular images.

[0120] Constrained parallax search range: For pixels that have obtained the LiDAR depth prior and its neighborhood, and its corresponding depth It can be converted into theoretical disparity values. ( (This refers to the baseline length of the binocular camera). In the disparity calculation stage of stereo matching, the disparity search range for this pixel can be narrowed from the traditional global range to... ,in It is a relatively small tolerance.

[0121] S3: Fuse the lidar point cloud and the stereo matching point cloud to obtain the fused point cloud.

[0122] Because LiDAR point clouds possess high-precision depth information but may not fully capture small targets, while stereo matching point clouds have high density but may have significant errors in depth estimation for slender targets with weak or repetitive textures, it is necessary to fuse the two. The purpose of fusion is to combine the advantages of both to generate a fused point cloud that includes both the high-precision skeleton information from LiDAR and the dense surface details from binocular vision. The fusion process typically needs to be performed in a unified coordinate system to ensure spatial alignment of the two point clouds. This can be achieved through point cloud registration algorithms (such as the ICP algorithm and its variants) or based on shared localization and orientation (POS) system parameters.

[0123] S3.1: A probabilistic framework is used to fuse the two point clouds.

[0124] The fusion process is not simply a point cloud merging process, but takes into account the reliability of the source of each data point. Probabilistic frameworks, such as Gaussian Process Regression (GPR) or ideas based on Gaussian Mixture Model (GMM), are used to estimate the depth / occupancy probability of each location (or voxel) in the space.

[0125] Uncertainty modeling first requires estimating the uncertainty for each point in both the LiDAR point cloud and the stereo matching point cloud. For LiDAR points, the uncertainty primarily stems from ranging errors, angular resolution limitations, and systematic errors inherent in the scanner itself. This uncertainty can typically be provided by the sensor manufacturer or estimated through calibration experiments and can be represented as a covariance matrix or a simplified standard deviation. For stereo matching point clouds, the uncertainty is related to factors such as image noise, the confidence level of the matching algorithm (e.g., the magnitude of the matching cost, the uniqueness of the disparity, etc.), the richness of the object's surface texture, and consistency with the LiDAR depth prior. For example, the uncertainty in depth estimation is greater in sparsely textured regions or regions with high matching costs.

[0126] Probabilistic fusion first voxelizes the unified 3D space containing the two types of point clouds, dividing the continuous space into a regular voxel grid. Then, it fuses information voxels one by one, collecting all LiDAR points and stereo matching points falling within each voxel. Next, it assigns weights based on the uncertainty quantification results, following the principle that "the smaller the uncertainty and the more reliable the measurement, the greater the weight," assigning corresponding fusion weights to each point within a voxel. Finally, it makes fusion decisions on the weighted point cloud data within the voxels using weighted averaging, Kalman filtering, or Bayesian fusion rules, determining the final state of each voxel, including the probability that the voxel is occupied by an object and the most likely depth value in space. The final states of all voxels together constitute the fused point cloud, achieving high-quality fusion of the two types of point clouds at the probabilistic level.

[0127] S3.2: Assign higher weights to the areas covered by the lidar point cloud than to the point cloud areas generated solely by stereo matching.

[0128] Region identification: The first step is to identify which regions are "covered" by the LiDAR point cloud. This can be done by projecting the LiDAR point cloud onto a reference plane or image plane and taking into account its influence range (e.g., a certain radius or voxel neighborhood around a point).

[0129] Weight Enhancement: During the fusion process, if a spatial location (or voxel) is supported by both LiDAR points and stereo matching points, the information from the LiDAR is given higher weight when calculating the final 3D information of that location. For example, in a weighted average, the weight coefficient of the LiDAR points... The weight coefficients significantly greater than those of the stereo matching points This is because lidar is generally considered to directly measure three-dimensional spatial coordinates, and its depth accuracy is generally superior to stereo matching depths indirectly calculated from images, especially for targets such as busbars. Even if the stereo matching point cloud density is higher, the depth provided by lidar points should be used as the standard.

[0130] Handling method:

[0131] 1. If a voxel contains both a lidar point and a stereo matching point, the representative point of the voxel can be preferentially taken from the lidar point, or from the weighted average value centered on the lidar point, where the weight of the lidar point is much greater than that of the stereo matching point.

[0132] 2. In some fusion algorithms, it can be directly set that if there are valid LiDAR points in a certain area, the depth information of that area is completely determined by the LiDAR, and the stereo matching points are only used to fill in the areas not covered by the LiDAR.

[0133] S4: Extract the target point cloud of the slender target object from the fused point cloud.

[0134] The fused point cloud contains not only points of the target object (such as a busbar) but also a large number of background points (such as supporting structures, the ground, and other equipment). To accurately reconstruct the target object, it needs to be segmented from the fused point cloud. Prior knowledge of the target object can be utilized, such as the linear geometric features of a slender target, point cloud density distribution characteristics, color features (if the stereo matching point cloud has color), or spatial continuity. Point cloud segmentation algorithms (such as region growing-based segmentation, clustering-based segmentation like DBSCAN, and linear / column structure extraction based on Random Sample Consensus RANSAC) can be used to identify and extract the set of point clouds belonging to the slender target object, i.e., the target point cloud.

[0135] S4.1: Combine the geometric features of the slender target object with color information from the binocular images to cluster or classify the fused point cloud in order to extract the target point cloud.

[0136] In fused point clouds, slender target objects such as busbars often exhibit unique geometric and visual characteristics, which can be used to distinguish them from the background (such as towers, vegetation, ground, buildings, etc.).

[0137] Utilization of geometric features:

[0138] 1. Linear / Curvilinear Features: The generatrix appears as a continuous line or a smooth curve (such as a catenary shape) in space. Local geometric descriptors for each point in the point cloud can be calculated, such as principal direction, curvature, and normal vector distribution. Points on the generatrix typically have a distinct principal direction and relatively small curvature (approximately straight lines within small local segments). RANSAC-based methods can be used to fit local line or curve segments, or variants of the Hough transform can be used to detect linear structures.

[0139] 2. Point density distribution: The point cloud of the bus line is usually sparse, but it has a certain continuity along its direction.

[0140] 3. Topology: Busbars are usually connected between towers, and this topology can be used to assist in extraction.

[0141] 4. Size constraints: The diameter of the busbar is usually within a known range, which can be used to filter out structures that are too thick or too thin.

[0142] Utilization of color information (from binocular images): If the stereo matching point cloud (and some points in the resulting fused point cloud) contains color information (i.e., each 3D point is associated with its RGB color value in the original image), then the color information can serve as a powerful clue to distinguish the target from the background.

[0143] 1. Color uniformity: The busbar itself usually has a relatively uniform color (e.g., silver-gray, black, etc., depending on the conductor material and aging). Background objects such as vegetation (green), sky (blue), and buildings (various colors) are different.

[0144] 2. Color contrast: There is usually a color contrast between the bus line and its immediate background (such as sky or vegetation).

[0145] 3. Color Priority: The typical color range of the busbar can be predefined.

[0146] Clustering or classification that combines geometric and color information:

[0147] 1. Feature extraction: For each point or local point set in the fused point cloud, extract its three-dimensional coordinates, normal vector, principal curvature, color value (RGB or HSV color space, etc.) and other features to form a multi-dimensional feature vector.

[0148] 2. Clustering methods:

[0149] Unsupervised clustering: Clustering algorithms such as DBSCAN (a density-based scanning algorithm, adept at discovering clusters of arbitrary shapes) or K-Means can be used. In DBSCAN, a custom distance metric function can be designed, which simultaneously considers the spatial distance between points, normal vector similarity, and color similarity. Points belonging to the generatrix will be clustered together due to their compactness in these feature spaces.

[0150] Graph-based methods: The point cloud is constructed into a graph, and the weight of the edges depends on the geometric and color similarity between the points. Then, the main lines are separated by graph segmentation algorithms (such as Normalized Cuts).

[0151] 3. Classification method:

[0152] Supervised classification: If labeled training data exists (i.e., the genesis and background in a portion of the point cloud are manually labeled), a classifier can be trained, such as a Support Vector Machine (SVM), Random Forest, or a deep learning point cloud segmentation model (e.g., PointNet, PointNet++, DGCNN, etc.). These models can directly learn the mapping from raw point cloud features (including coordinates and color) to class labels (genesis / non-genesis).

[0153] Rule classification: Based on experience, the discrimination rules are set, for example: "If the local neighborhood points of a point show a linear distribution and their average color is within the preset range of the bus line color, then the point belongs to the bus line."

[0154] S5: Fit the target point cloud to a geometric model used to characterize the geometry of the slender target object.

[0155] The geometric model should be able to describe the spatial morphology of a slender target object under specific physical conditions. For example, the natural drooping shape of a flexible conductor or jumper connecting two points in a substation under gravity can be approximated by a catenary model; for a rigid busbar, a straight line segment or cylindrical model can be used. After selecting a suitable model, parameter estimation algorithms (such as least squares method, RANSAC, etc.) are used to estimate the model parameters based on the target point cloud data, thereby obtaining an initial geometric model instance.

[0156] S5.1: Fit the target point cloud of the conductor to the catenary model.

[0157] A catenary is a curve formed by a flexible chain or cable fixed at both ends drooping naturally under its own uniform gravity. In substation engineering, under ideal conditions where complex external loads such as wind load and icing are not considered, the geometry of a single-span flexible conductor or jumper is very close to that of a catenary.

[0158] In a two-dimensional Cartesian coordinate system, the standard equation of a catenary can be expressed as:

[0159]

[0160] in, These are the coordinates of the lowest point (vertices) of the catenary. This is a catenary parameter, also known as the sag factor, which is related to the weight per unit length of the cable and the horizontal tension. ,in It is horizontal tension. (Weight per unit length) It is a hyperbolic cosine function.

[0161] S6: Based on the deviation between the target point cloud and the geometric model, iteratively adjust the positions of at least some points in the target point cloud, or adjust the parameters of the geometric model, or simultaneously adjust the positions of the points and the parameters of the model, until the deviation meets the preset convergence condition.

[0162] After obtaining the initial geometric model fitting results, the deviation of each point (or representative point) in the target point cloud from the geometric model is calculated (e.g., the closest distance from the point to the model). These deviations reflect the inconsistencies between the current point cloud and the ideal model, which may include measurement errors in the point cloud itself or inaccuracies in the model parameters.

[0163] Based on these deviations, iterative optimization is performed:

[0164] 1. Adjust the position of the point: For points with large deviations, if it is believed that the deviation from the true position is due to measurement noise or initial fusion error, the position can be "pulled" closer to the geometric model to make it more in line with the constraints of the model.

[0165] 2. Adjust model parameters: If there is a systematic deviation between the overall point cloud trend and the current model parameters, the model parameters (such as the lowest point position of the catenary, the sag coefficient, etc.) should be adjusted to make the model better fit most of the point cloud data.

[0166] 3. Simultaneous adjustment: In a more complex optimization framework, the position of the point and the model parameters can be optimized simultaneously. For example, in the energy minimization framework, the distance from the point to the model can be used as the energy term, while the smoothness or physical rationality of the model can be used as the regularization term.

[0167] This iterative process continues, with each iteration recalculating and adjusting the deviation until the overall deviation between the point cloud and the model (e.g., the root mean square value of all point deviations, the maximum deviation, or the proportion of points that meet a specific accuracy requirement) reaches a preset convergence condition (e.g., the deviation is less than a certain threshold, or the deviation changes very little between two consecutive iterations).

[0168] S6.1 (occurring before the original S5 fitting model or in the initial stage of the S6 iteration adjustment): Compare the differences between the depth information from the LiDAR and the depth information from the stereo matching in the fused point cloud to identify systematic errors or sensor biases.

[0169] Even with precise endo- and extrinsic parameter calibration of LiDAR and binocular cameras, small, systematic deviations may still exist between the depth information generated by them in practical applications due to environmental factors, sensor aging, imperfect calibration, or algorithm limitations. For example, stereo matching algorithms may tend to systematically overestimate or underestimate depth, or exhibit nonlinear biases within a specific distance range.

[0170] Selection of regions for difference calculation: To accurately estimate this systematic bias, regions that are well covered by both LiDAR and stereo matching point clouds should be selected. These regions are typically surfaces with relatively stable structures and rich textures in the scene, or, after using LiDAR-guided stereo matching, targets such as the bus line itself can also be used as comparison objects.

[0171] Corresponding point lookup: For a LiDAR point within a selected area, its corresponding point or depth value needs to be found in the stereo matching point cloud (or its generated depth map). This can be achieved by projecting the LiDAR point onto the stereo matching depth map, or by finding the nearest neighbor point in 3D space.

[0172] Deviation statistics: Collect depth values ​​of a large number of corresponding point pairs. , Calculate the differences between them Analyze the statistical properties of these differences, such as the mean, variance, median, and the trend of the differences with distance. If the differences exhibit a systematic pattern (e.g., a significantly non-zero mean, or a linear or non-linear relationship between the differences and depth), it indicates the presence of systematic errors or sensor bias.

[0173] S6.2: Estimate correction parameters based on differences and apply the correction parameters to the fused point cloud (especially the part derived from stereo matching).

[0174] Based on the identified systematic deviation patterns, a correction model is established. One model is the constant offset model: if the deviation mainly manifests as a fixed offset, then the correction parameter is a constant. Corrected depth The second is the scale factor model: if the deviation is proportional to the depth, then the correction parameter may be a scale factor. Corrected depth (Or multiply by a factor close to 1). Thirdly, linear / polynomial models: if there is a more complex relationship between bias and depth, a linear function can be used. Or a higher-order polynomial can be used to fit the correction relationship, where For calibration parameters.

[0175] Parameter estimation: Using the difference data collected in the previous step, the parameters of the calibration model are estimated by methods such as least squares fitting.

[0176] Application Correction: The estimated correction parameters are applied to the depth values ​​(or their 3D coordinates) of all points in the fused point cloud that originated from stereo matching. For points with both LiDAR and stereo matching information, if this bias has already been considered during fusion (e.g., through probabilistic fusion in S3.1), this step is primarily to correct points generated solely by stereo matching.

[0177] After completing this systematic bias correction, subsequent iterative adjustments are performed in S5 (fitting the target point cloud to the geometric model) and S6 (adjusting point positions or model parameters based on the bias between the point cloud and the model). Since the systematic bias of the point cloud data has been initially compensated, subsequent model fitting and iterative optimization will be based on a more consistent dataset, thereby enabling more accurate estimation of model parameters and correction of random errors.

[0178] S6.3 (as part of the S6 iteration loop): For points that deviate from the geometric model by more than a preset distance threshold, identify them as noise or errors and adjust their positions to better conform to the geometric model, or remove them from the target point cloud.

[0179] In each iteration, the deviation of each point in the target point cloud from the current geometric model is calculated (e.g., the orthogonal distance from the point to the model). back:

[0180] Set distance threshold The threshold is set based on the specific application scenario, the characteristics of the target object, the accuracy of the sensor, and the desired reconstruction accuracy. For example, for bus reconstruction, if the bus diameter and allowable measurement error are known, a reasonable distance threshold can be set accordingly. This threshold can also be dynamic, for example, using a larger threshold in the first few iterations and gradually decreasing it thereafter.

[0181] Identify outliers: For any point in the target point cloud If its distance to the current geometric model Then that point It was initially identified as an outlier, that is, a potential noise point or an error point caused by extraction error or fusion error.

[0182] Handling outliers:

[0183] 1. Adjusting Position (Pull Back): Instead of directly deleting the point, its position is "pulled back" towards the geometric model. For example, it can be moved to the nearest point on the model, or moved a certain proportion of the distance along its normal to the model, making its new position closer to the model. The adjusted point position is as follows:

[0184]

[0185] in, This is the adjusted point position. yes The normal vector to the nearest point in the model. It is an adjustment factor. It is the expected distance from the point to the model (for example, it can be set to 0 or a small value).

[0186] 2. Removal (Deletion): Identified outliers are permanently removed from the target point cloud. This method is suitable when it is certain that these points are obvious noise or errors, and retaining them would negatively impact subsequent model parameter optimization or point position adjustment. After removal, these points will no longer be considered in subsequent iterations.

[0187] 3. Reduce weights: In some iterative optimization algorithms (such as iterative reweighted least squares), it is not necessary to adjust the physical location of outliers or remove them directly. Instead, when calculating model parameters or updating the positions of other points, these outliers are given very low weights to minimize their impact on the overall optimization results.

[0188] After processing outliers (by adjusting their positions or removing them), the updated target point cloud (or the point cloud with updated weights) is used to re-estimate the model parameters for the next round (if the iterative adjustment includes adjusting the model parameters), or the positions of the remaining points are adjusted until the convergence condition is met.

[0189] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0190] Based on the same inventive concept, this application also provides a substation three-dimensional reconstruction error compensation device for implementing the above-mentioned substation three-dimensional reconstruction error compensation method. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more substation three-dimensional reconstruction error compensation device embodiments provided below can be found in the limitations of the substation three-dimensional reconstruction error compensation method described above, and will not be repeated here.

[0191] In one exemplary embodiment, such as Figure 3 As shown, a three-dimensional reconstruction error compensation device for a substation is provided, comprising: an acquisition module 302, a processing module 304, a fusion module 306, an extraction module 308, and an adjustment module 310, wherein:

[0192] The acquisition module 302 is used to acquire lidar point cloud and binocular image of slender target objects in the substation; wherein the lidar and the binocular camera have spatial consistency when acquiring data;

[0193] Processing module 304 is used to process binocular images using a stereo matching algorithm to generate a stereo matching point cloud containing depth information;

[0194] The fusion module 306 is used to convert the lidar point cloud and the stereo matching point cloud to the same coordinate system for fusion to obtain a fused point cloud.

[0195] Extraction module 308 is used to extract the target point cloud of the slender target object from the fused point cloud by combining the geometric features of the slender target object and the color information from the binocular image.

[0196] The adjustment module 310 is used to fit the target point cloud to a geometric model used to characterize the geometric shape of the slender target object, calculate the deviation between the target point cloud and the geometric model, and adjust the position of some points in the target point cloud or the parameters of the geometric model if the deviation does not meet the preset convergence condition, until the deviation meets the preset convergence condition.

[0197] In one embodiment, the processing module 304 is further configured to:

[0198] By projecting the portion of the LiDAR point cloud that overlaps with the field of view of the binocular camera onto the image plane of the binocular image, the depth prior information of the image region corresponding to the slender target object is obtained.

[0199] Based on depth prior information, the disparity search range of stereo matching is constrained. According to the disparity search range, the disparity map of the binocular image is calculated using the stereo matching algorithm.

[0200] By combining the intrinsic and extrinsic parameters of the binocular camera and the baseline length, the disparity map is converted into a stereo matching point cloud containing depth information.

[0201] In one embodiment, the fusion module 306 is further configured to:

[0202] The three-dimensional space is divided into voxel grids, and the lidar points and stereo matching points falling in each voxel grid are determined.

[0203] Based on the measurement uncertainty of each point in the lidar point cloud and the stereo matching point cloud, the corresponding weight of each point is determined; where the smaller the measurement uncertainty, the greater the weight; the weight of the lidar point is greater than the weight of the stereo matching point in the same voxel grid.

[0204] The three-dimensional information of each voxel grid is determined based on the weights and three-dimensional coordinates of the lidar points and stereo matching points that fall into each voxel grid.

[0205] The 3D information of all voxel meshes is integrated to obtain a fused point cloud.

[0206] In one embodiment, the extraction module 308 is further configured to:

[0207] For each point in the fused point cloud, the three-dimensional coordinates, normal vector, curvature, and associated RGB color values ​​of each point are extracted by combining the geometric features of the slender target object and the color information from the binocular images, and a multi-dimensional feature vector of each point is constructed.

[0208] The points in the fused point cloud are clustered based on the multidimensional feature vector of each point. Clusters that meet the geometric and color features of the slender target object are selected and used as the target point cloud.

[0209] In one embodiment, the geometric model is a catenary model; the adjustment module 310 is also used for:

[0210] The target point cloud is denoised and then projected onto a vertical plane containing the slender target object.

[0211] Based on the catenary equation, the initial parameters of the catenary model are estimated. The catenary equation includes the sag coefficient and the coordinate parameters of the lowest point.

[0212] The parameters of the catenary model are iteratively optimized until the sum of squared distances between the catenary model and the target point cloud meets the preset fitting accuracy requirements, thus completing the model fitting of the target point cloud.

[0213] In one embodiment, the adjustment module 310 is further configured to:

[0214] Calculate the orthogonal distance from each point in the target point cloud to the geometric model, identify outliers whose orthogonal distance exceeds a preset threshold, and adjust the position of the outliers in the target point cloud or the parameters of the geometric model until the orthogonal distance from each point in the target point cloud to the geometric model does not exceed the preset threshold in two consecutive tests.

[0215] The modules in the aforementioned three-dimensional reconstruction error compensation device for substations can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0216] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for three-dimensional reconstruction error compensation of a substation. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0217] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0218] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0219] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0220] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0221] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0222] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0223] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0224] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for compensating for errors in the three-dimensional reconstruction of a substation, characterized in that, The method includes: Acquire lidar point cloud and binocular images of slender targets within a substation; wherein the lidar and binocular camera maintain spatial consistency during data acquisition. The stereo image is processed using a stereo matching algorithm to generate a stereo matching point cloud containing depth information; The lidar point cloud and the stereo matching point cloud are transformed to the same coordinate system and then fused to obtain a fused point cloud. By combining the geometric features of the slender target object with color information from the binocular image, the target point cloud of the slender target object is extracted from the fused point cloud; The target point cloud is fitted to a geometric model used to characterize the geometry of the slender target object. The deviation between the target point cloud and the geometric model is calculated. If the deviation does not meet the preset convergence condition, the positions of some points in the target point cloud or the parameters of the geometric model are adjusted until the deviation meets the preset convergence condition.

2. The method according to claim 1, characterized in that, The step of processing the stereo image using a stereo matching algorithm to generate a stereo matching point cloud containing depth information includes: The portion of the lidar point cloud that overlaps with the field of view of the binocular camera is projected onto the image plane of the binocular image to obtain the depth prior information of the image region corresponding to the slender target object. Based on the depth prior information, the disparity search range of stereo matching is constrained, and the disparity map of the binocular image is calculated using the stereo matching algorithm according to the disparity search range. By combining the intrinsic and extrinsic parameters of the binocular camera and the baseline length, the disparity map is converted into a stereo matching point cloud containing depth information.

3. The method according to claim 1, characterized in that, The step of converting the lidar point cloud and the stereo matching point cloud to the same coordinate system and fusing them to obtain a fused point cloud includes: The three-dimensional space is divided into voxel grids, and the lidar points and stereo matching points falling in each voxel grid are determined. Based on the measurement uncertainty of each point in the lidar point cloud and the stereo matching point cloud, the corresponding weight of each point is determined; wherein, the smaller the measurement uncertainty, the greater the weight; the weight of the lidar point is greater than the weight of the stereo matching point in the same voxel grid. The three-dimensional information of each voxel grid is determined based on the weights and three-dimensional coordinates of the lidar points and stereo matching points that fall into each voxel grid. The 3D information of all voxel meshes is integrated to obtain a fused point cloud.

4. The method according to claim 1, characterized in that, The step of extracting the target point cloud of the slender target object from the fused point cloud by combining the geometric features of the slender target object and the color information from the binocular image includes: For each point in the fused point cloud, combining the geometric features of the slender target object and the color information from the binocular image, the three-dimensional coordinates, normal vector, curvature, and associated RGB color values ​​of each point are extracted to construct a multi-dimensional feature vector for each point; The points in the fused point cloud are clustered based on the multidimensional feature vector of each point, and clusters that conform to the geometric and color features of the slender target object are selected and used as the target point cloud.

5. The method according to claim 1, characterized in that, The geometric model is a catenary model; fitting the target point cloud to the geometric model used to characterize the geometry of the slender target object includes: The target point cloud is denoised, and then projected onto a vertical plane containing the elongated target object. Based on the catenary equation, the initial parameters of the catenary model are estimated, wherein the catenary equation includes the sag coefficient and the coordinate parameters of the lowest point; The parameters of the catenary model are iteratively optimized until the sum of squared distances between the catenary model and the target point cloud meets the preset fitting accuracy requirements, thus completing the model fitting of the target point cloud.

6. The method according to claim 1, characterized in that, The step of calculating the deviation between the target point cloud and the geometric model, and adjusting the positions of some points in the target point cloud or the parameters of the geometric model if the deviation does not meet the preset convergence condition, includes: Calculate the orthogonal distance from each point in the target point cloud to the geometric model, identify outliers whose orthogonal distance exceeds a preset threshold, and adjust the position of the outliers in the target point cloud or the parameters of the geometric model until the orthogonal distance from each point in the target point cloud to the geometric model does not exceed the preset threshold for two consecutive times.

7. A three-dimensional reconstruction error compensation device for a substation, characterized in that, The device includes: The acquisition module is used to acquire lidar point clouds and binocular images of slender target objects within the substation; wherein, the lidar and the binocular camera have spatial consistency when acquiring data; The processing module is used to process the binocular image using a stereo matching algorithm to generate a stereo matching point cloud containing depth information; The fusion module is used to transform the lidar point cloud and the stereo matching point cloud to the same coordinate system for fusion to obtain a fused point cloud. An extraction module is used to extract the target point cloud of the slender target object from the fused point cloud by combining the geometric features of the slender target object and the color information from the binocular image; An adjustment module is used to fit the target point cloud to a geometric model that characterizes the geometric shape of the slender target object, calculate the deviation between the target point cloud and the geometric model, and adjust the position of some points in the target point cloud or the parameters of the geometric model if the deviation does not meet the preset convergence condition, until the deviation meets the preset convergence condition.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images