Three-dimensional Gaussian sputtering power line reconstruction method and system based on catenary physical constraint
The 3D Gaussian sputtering electric field line reconstruction method based on catenary physical constraints, combined with semantic segmentation and physical model fitting, solves the problems of noise and insufficient geometric accuracy in the reconstruction of weak texture regions by traditional methods, and achieves high-precision and continuous 3D electric field line reconstruction, which is suitable for live robot operations.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
- Filing Date
- 2025-12-31
- Publication Date
- 2026-05-05
AI Technical Summary
Traditional power line reconstruction methods are prone to generating noise that deviates from the actual position in areas with weak texture. The geometric accuracy cannot meet the requirements of robot operations, and LiDAR equipment is expensive and difficult to adapt to lightweight robot platforms.
Combining semantic segmentation, motion recovery structure, and physical model fitting, a three-dimensional Gaussian sputtering electric field line reconstruction method is constructed using catenary physical constraints. The theoretical trajectory of the electric field line is fitted using the catenary equation parameters, and physical distance constraints are introduced during the optimization process to eliminate breaks and artifacts, thereby achieving high-precision reconstruction.
It achieves high-precision, continuous 3D reconstruction of power lines under weak texture conditions, improves geometric accuracy and robustness, reduces data acquisition costs and algorithm deployment difficulty, and meets the environmental perception needs of autonomous robot operation.
Smart Images
Figure CN121982199A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and smart grid inspection technology, specifically a three-dimensional Gaussian sputtering power line reconstruction method based on catenary physical constraints. Background Technology
[0002] Power lines are a core component of the power grid system, and their safe operation is directly related to national welfare and people's livelihood. With the development of smart grids, the use of live-line working robots to replace manual labor for high-voltage line inspection and maintenance has become an inevitable trend. The prerequisite for robot operation is the ability to accurately perceive the surrounding environment, especially to obtain high-precision 3D models of power lines.
[0003] Traditional power line reconstruction primarily relies on LiDAR, but these devices are expensive and bulky, making them difficult to adapt to lightweight robotic platforms. Traditional vision-based 3D reconstruction methods (such as MVS) are limited by the sparse texture and elongated structure of power lines, often resulting in fragmented and incomplete reconstructions. While the recently proposed 3D Gaussian sputtering (3DGS) technology offers excellent rendering results, it is essentially data-driven and prone to generating numerous "floating" noise points in weakly textured areas, deviating from their true positions, and its geometric accuracy cannot meet the requirements of robotic operations.
[0004] This invention addresses the issue of weak texture and strong reflection in electric field line images in natural scenes. To improve the spatial continuity and geometric accuracy of the reconstruction, a three-dimensional Gaussian sputtering electric field line reconstruction method and system based on catenary physical constraints is proposed. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention aims to provide a 3D Gaussian sputtering electric field line reconstruction method based on catenary physical constraints. This method combines semantic segmentation, structure-of-motion (SOMO) reconstruction, physical model fitting, and neural rendering techniques to construct an integrated reconstruction framework. For monocular image sequences acquired by a robot, the electric field line region is first extracted using an improved semantic segmentation model, and initial information is obtained by combining it with SFM (Single-Factor Metrics and Functions). Then, the physical equations of the catenary are fitted using sparse point clouds to obtain the theoretical trajectory. This trajectory is then used to guide 3DGS initialization, and physical distance constraints are introduced during the optimization process to force the point cloud to converge to the true physical shape. Compared to traditional pure vision methods, this invention can effectively eliminate breaks and artifacts, achieving high-precision 3D electric field line reconstruction.
[0006] This application achieves the above effects through the following technical solution: a three-dimensional Gaussian sputtering electric field line reconstruction method based on catenary physical constraints, the method comprising the following steps:
[0007] S1. Obtain Image1, a monocular image sequence collected for power line inspection scenarios, and construct a power line image sample library based on the monocular images;
[0008] S2 performs semantic segmentation on the image sequence Image1 to extract the power line mask image Image2; and uses the structure of motion recovery algorithm SfM to solve the image sequence Image1 to obtain the camera pose parameters Pose1 and the initial sparse point cloud Data1 of the scene.
[0009] S3 performs physical parameter fitting based on the initial sparse point cloud Data1, calculates the catenary equation parameters, and constructs the three-dimensional theoretical spatial trajectory Curve1 of the electric field line.
[0010] S4. Using the three-dimensional theoretical spatial trajectory Curve1 as the center, generate a three-dimensional Gaussian sphere along the trajectory spatial position to complete the physical prior initialization of the three-dimensional Gaussian sputtering model.
[0011] S5 Construct a loss function that includes the catenary physical constraint term, and use the image sequence Image1 and camera pose Pose1 to iteratively train the initialized 3D Gaussian sputtering model. When the training loss converges, the final 3D electric field line reconstruction model Model1 is obtained.
[0012] S6 Analyzes the spatial geometric parameters of the power line according to the three-dimensional reconstruction model Model1 of the power line to realize the reconstruction of the three-dimensional shape of the power line.
[0013] Furthermore, S2 specifically includes:
[0014] S21 acquires monocular images of diverse scenes containing power lines, and constructs a power line image dataset (Image-dataset) after preprocessing the images;
[0015] S22 Use image annotation software to annotate the power line regions in a single image Image_open in the Image-dataset, set them as labels to represent power line categories, save the annotation file in the same path as the image, and generate a JSON format annotation file containing power line categories;
[0016] S23 The labeled files are batch converted into binary mask format files required for training, and a semantic segmentation labeled dataset Labeling1_dataset containing original image-mask pairs is constructed;
[0017] S24 Constructs a U-Net network based on an encoder-decoder structure, embedding a convolutional block attention (CBAM) module in the bottleneck layer (Bottleneck) of the U-Net network. The CBAM module consists of cascaded channel attention sub-modules. Spatial attention submodule Composition, Feature Map The computational logic after processing by this module is as follows:
[0018] ;
[0019] ;
[0020] in, For the input feature map, Channel attention-weighted features, For the final output features, This indicates element-wise multiplication;
[0021] S25 divides the semantic segmentation annotation dataset Labeling1_dataset into a training set and a test set, and constructs a model using binary cross-entropy loss. and Dice coefficient loss Weighted Mixed Loss Function This formula is used to solve the sample imbalance problem where the proportion of pixels in the electric field lines is extremely small.
[0022] ;
[0023] ;
[0024] in, To balance the weighting coefficients, For the first The predicted probability value of each pixel. No. The actual label value of each pixel. Total number of pixels For smoothing terms;
[0025] S26 The U-Net network is iteratively trained using the training set and the hybrid loss function. When the Intersection over Union (IoU) index of the test set reaches a preset threshold, the optimal model weights are saved. The optimal model weights are loaded to perform inference on the acquired image sequence Image1 and output the power line mask image Image2.
[0026] S27 For the image sequence Image1, the key points and descriptors of each frame are extracted using the SIFT algorithm, feature matching is performed using the nearest neighbor ratio method, and mismatched point pairs are removed using the RANSAC algorithm.
[0027] S28 Initializes images with a wide baseline by using epipolar geometry constraints to solve the essential matrix to recover the initial pose; then registers the remaining images frame by frame using the n-point perspective algorithm PnP and calculates the three-dimensional coordinates of spatial points using triangulation to generate the initial sparse point cloud Data1.
[0028] S29 Constructs an objective function to minimize the reprojection error, performing joint nonlinear optimization on the pose parameters Pose1 of all cameras and the spatial coordinates of the initial sparse point cloud Data1. The specific objective function is as follows:
[0029] ;
[0030] in, This represents the total weight projection error. The total number of images, This represents the total number of three-dimensional points. For visibility identification, when the camera Observed three-dimensional points The value is 1 if the condition is met, otherwise it is 0. For three-dimensional points In the camera The observed pixel coordinates; For the camera projection model, the three-dimensional points Project onto the pixel coordinate system; after optimization and convergence, output the final camera pose parameters Pose1 and the initial sparse point cloud Data1.
[0031] Furthermore, S3 specifically refers to:
[0032] 31. Traverse each 3D point in the initial sparse point cloud Data1, project it onto the power line mask image Image2, retain the 3D points whose projection coordinates are located within the effective mask area, and form a power line candidate point set Points1 that is consistent with the image semantics from these points.
[0033] S32 uses the Random Sample Consensus Algorithm (RANSAC) to process the candidate point set Points1 of the power line. During the iteration process, the number of interior points falling within the threshold range of the plane model is counted. The plane containing the most interior points is selected as the best vertical plane Plane1, and points that are more than a preset threshold away from the plane are removed as outliers to obtain the optimized interior point set.
[0034] S33 analyzes the spatial geometric properties of the vertical plane Plane1 in the world coordinate system and constructs a set of locally orthogonal basis vectors describing the shape of electric field lines, specifically including:
[0035] Calculate the geometric centroid of the interior point set as the reference origin. Define the normal vector of the perpendicular plane Plane1 as... The unit vector of the direction of gravity is A horizontal unit vector along the span of the power line is constructed using the vector cross product operation. Construct a vertical unit vector perpendicular to the span direction. Thus establishing a system from the origin and basis vectors Defined local geometric reference frame;
[0036] S34 sets the points in the interior point set. Decomposed to the local geometric reference frame to obtain scalar coordinates ,in A catenary equation model is constructed based on the decomposed scalar coordinates. The optimal shape parameters are calculated using a nonlinear regression equation. and position parameters This yields the equation of the curve in the two-dimensional plane;
[0037] S34 Based on the solved equation parameters and local orthogonal basis vectors Within the span of the power line, a sampling step size is set, and the three-dimensional theoretical spatial trajectory Curve1 of the power line is directly constructed using linear vector combination. Any point on the trajectory... The formula for calculating three-dimensional coordinates is:
[0038] ;
[0039] in, For along the horizontal vector The sampled value of the independent variable for direction.
[0040] Furthermore, S4 specifically includes:
[0041] S41 Based on the three-dimensional trajectory equation obtained in step S3 Calculate the total arc length of the curve and set the linear sampling density along the curve path, in the parameters Uniform sampling is performed within the domain to generate a series of discrete three-dimensional coordinate point sets. Each coordinate point in the three-dimensional coordinate point set is used as the initial position mean parameter of an independent three-dimensional Gaussian sphere. This initializes a set of electric field line Gaussian point clouds composed of multiple three-dimensional Gaussian spheres.
[0042] S42 Iterate through each 3D Gaussian sphere in the set of Gaussian point clouds of electric lines, and calculate the position mean of the 3D Gaussian sphere. The corresponding curve parameters, for the three-dimensional trajectory equation Find information about the parameters The first derivative is obtained and normalized to obtain the unit tangent vector of the point at that sampling location. Construct a vector with the tangent vector The local rotation matrix in the X-axis principal axis direction and the rotation matrix Convert to unit rotation quaternion As the initial rotation parameter of the three-dimensional Gaussian sphere, the principal axis direction of the Gaussian sphere is strictly aligned with the local spatial orientation of the electric field lines;
[0043] S43 assigns anisotropic initial scale parameters to each three-dimensional Gaussian sphere in the set. Specifically, the scale vector is defined. ,in The longitudinal scale is along the direction of the tangent vector. Define the radial dimension perpendicular to the tangent vector plane and set constraints. This causes the initialized three-dimensional Gaussian sphere to exhibit a slender ellipsoidal geometry that extends tangentially along the electric field lines;
[0044] S44 sets the initial opacity parameters for all three-dimensional Gaussian spheres. Set the confidence level to a high value and initialize the initial three-dimensional Gaussian spherical harmonic function coefficients SH to zero or according to the average color value of the projected image to complete the physical guidance initialization construction of the three-dimensional Gaussian sputtering model.
[0045] Furthermore, S5 specifically includes:
[0046] The S51 uses a differentiable rasterizer to project a three-dimensional Gaussian sphere onto a two-dimensional image plane to obtain a rendered image from the current viewpoint. And calculate its comparison with the real image. Loss of photometric consistency between them;
[0047] S52 Constructing Physical Constraint Loss Based on Geometric Distance Field This is used to constrain the center of the Gaussian sphere of the electric field line from deviating from the theoretical catenary trajectory.
[0048] S53 Define the total loss function ,in For luminance loss, For structural similarity loss, Loss due to physical constraints;
[0049] S54 Calculate Physical Constraint Loss Specifically, this involves calculating the mean of the shortest Euclidean distance from the center of each three-dimensional Gaussian sphere belonging to the electric field line category to the three-dimensional theoretical space trajectory Curve1 in the current iteration step.
[0050] S55 uses the backpropagation algorithm to update the position, rotation, scaling and spherical harmonic coefficient properties of the three-dimensional Gaussian sphere, and fine-tunes the catenary equation parameters according to the current Gaussian sphere distribution every preset number of iterations, so that the physical model can dynamically adapt to the fine-tuning of the three-dimensional Gaussian distribution until the total loss function converges.
[0051] S56 When the total loss function converges, the iterative optimization process stops, and the final optimized 3D Gaussian scene representation is output; the geometric and appearance attributes of each 3D Gaussian sphere after convergence are retained; this series of discrete but continuously rendering anisotropic Gaussian spheres form a 3D electric line reconstruction model Model1 with catenary morphological characteristics.
[0052] Furthermore, S6 specifically includes:
[0053] S61 Extract the center coordinates of all Gaussian spheres in the finally converged 3D reconstruction model Model1 of the electric line, construct a 3D point cloud set of the electric line, use principal component analysis (PCA) to calculate the first principal direction of the 3D point cloud set of the electric line as the overall spatial direction vector of the electric line, identify the endpoint coordinates of the point cloud set at both ends of the direction, and calculate the Euclidean distance between the two endpoints to determine the span parameters of the electric line.
[0054] S62 Project the three-dimensional point cloud set of the power line onto a vertical plane composed of the gravity direction vector and the spatial direction vector. Calculate the vertical distance from the lowest point of the point cloud to the line connecting the two endpoints in the two-dimensional projection plane, i.e., the maximum sag height. Combined with the known ambient temperature and meteorological conditions, use the state equation to verify whether the sag parameters meet the safe operation standards.
[0055] S63 performs equidistant slice sampling on the 3D reconstruction model Model1 along the span direction of the power line, calculates the line connecting the centroids of the Gaussian spheres in each slice section, generates a smooth central skeleton line, compares the coordinates of the central skeleton line with the true value data measured by lidar or total station, evaluates the geometric accuracy of the reconstruction model, and finally outputs an automated power line reconstruction report containing a 3D model visualization view and geometric parameters of sag, span, and direction.
[0056] On the other hand, this application also provides a three-dimensional Gaussian sputtering electric field line reconstruction system based on catenary physical constraints, the system comprising:
[0057] The data acquisition module is configured to acquire a monocular image sequence Image1 captured for power line inspection scenarios;
[0058] The data preprocessing module is configured to perform semantic segmentation on the image sequence Image1 based on a deep learning model to extract the power line mask image Image2, and use the motion reconstruction structure algorithm to solve the camera pose parameter Pose1 and the initial sparse point cloud Data1 of the scene.
[0059] The physical modeling module is configured to perform catenary parameter regression based on the initial sparse point cloud Data1 to construct a three-dimensional theoretical spatial trajectory Curve1 of the electric field line that conforms to gravity constraints.
[0060] The physics-guided initialization module is configured to generate an initialized three-dimensional Gaussian sphere along the three-dimensional theoretical space trajectory Curve1 and endow it with an anisotropic geometric shape to complete the physical prior initialization of the three-dimensional Gaussian sputtering model.
[0061] The model joint optimization module is configured to construct a hybrid objective function that includes photometric consistency loss and catenary geometric constraint loss. It uses the image sequence Image1 and camera pose Pose1 to iteratively train and update the parameters of the initialized model, and outputs the electric power line 3D reconstruction model Model1.
[0062] The parameter parsing module is configured to parse and output the spatial geometric parameters of the electric power line based on the three-dimensional reconstruction model Model1 of the electric power line.
[0063] Furthermore, the data preprocessing module includes a dataset construction unit, a labeling processing unit, a model training unit, a mask inference unit, and a pose calculation unit;
[0064] The dataset construction unit is used to acquire public datasets and self-built robot perspective datasets to construct an image dataset of power lines.
[0065] The annotation processing unit is used to use an image annotation tool to perform pixel-level annotation on the power line regions in the power line image dataset, generate annotation files, and convert them into a mask format.
[0066] The model training unit is used to divide the dataset into a training set and a test set, use the training set to train the improved U-Net model with embedded CBAM modules, and save the weights when the IoU of the test set reaches a set threshold.
[0067] The mask inference unit is used to load the stored weights, segment the image sequence Image1, and output the power line mask image Image2.
[0068] The pose calculation unit is used to process the image sequence Image1 using the incremental SfM algorithm, and outputs sparse point cloud Data1 and camera pose Pose1.
[0069] Furthermore, the physical guidance initialization module and the model joint optimization module include a physical seeding unit, a geometric initialization unit, a loss calculation unit, and a parameter update unit;
[0070] The physical seeding unit is used to set the sampling density according to the total arc length of the catenary trajectory Curve1 and uniformly generate the initial center position of the three-dimensional Gaussian sphere along the trajectory.
[0071] The geometric initialization unit is used to calculate the tangent vector at each point on the trajectory, construct a rotation quaternion to align the principal axis of the Gaussian sphere with the tangent, and endow the Gaussian sphere with an anisotropic scale property of an elongated ellipsoid.
[0072] The loss calculation unit is used to calculate the shortest Euclidean distance from the center of the Gaussian sphere belonging to the electric field line category to the theoretical trajectory Curve1 in real time during training iterations, as a physical constraint loss term;
[0073] The parameter update unit is used to construct a total loss function that includes physical constraints, and uses the backpropagation algorithm and gradient descent strategy to update the position, rotation, scaling and spherical harmonic coefficients of the Gaussian sphere, and outputs the converged 3D electric field model Model1.
[0074] The innovation of this invention lies in:
[0075] 1. A strategy for 3D reconstruction of power lines that integrates physical and visual methods is proposed.
[0076] 2. A 3DGS initialization and constraint optimization mechanism based on catenary priors was constructed.
[0077] 3. High-precision, continuous 3D reconstruction of power lines under weak texture conditions has been achieved.
[0078] 4. Compared with traditional pure vision reconstruction methods, it has higher geometric accuracy and robustness. Attached Figure Description
[0079] Figure 1 This is a flowchart of the method of the present invention;
[0080] Figure 2 A diagram of the improved U-Net network model for embedding the CBAM attention mechanism. Detailed Implementation
[0081] This invention extends, recombines, and improves upon semantic segmentation and 3D reconstruction models. While it shares many similarities with the original technology, it also incorporates numerous improvements, which are hereby explicitly stated. The invention will be further illustrated below with reference to specific embodiments.
[0082] Example 1
[0083] This application provides a method for reconstructing three-dimensional Gaussian sputtering electric field lines based on catenary physical constraints. The flowchart of the method is shown below. Figure 1 As shown, the overall implementation path includes the following steps:
[0084] S1 Acquires Image1, a monocular image sequence collected for power line inspection scenarios;
[0085] S2 performs semantic segmentation on the image sequence Image1 to extract the power line mask image Image2; at the same time, it uses the Structure from Motion (SfM) algorithm to solve the image sequence Image1 and outputs the camera pose parameter Pose1 and the initial sparse point cloud Data1 of the scene.
[0086] S3 performs physical parameter fitting based on the initial sparse point cloud Data1, calculates the catenary equation parameters, and constructs the three-dimensional theoretical spatial trajectory Curve1 of the electric field line.
[0087] S4. Using the three-dimensional theoretical spatial trajectory Curve1 as the center, generate a three-dimensional Gaussian sphere along the trajectory spatial position to complete the physical prior initialization of the three-dimensional Gaussian sputtering model.
[0088] S5 Construct a loss function that includes the catenary physical constraint term, and use the image sequence Image1 and camera pose Pose1 to iteratively train the initialized 3D Gaussian sputtering model. When the training loss converges, the final 3D electric field line reconstruction model Model1 is obtained.
[0089] S6 Analyzes the spatial geometric parameters of the power line according to the three-dimensional reconstruction model Model1 of the power line to realize the reconstruction of the three-dimensional shape of the power line.
[0090] Firstly, step S2 of the above scheme provides a semantic segmentation and pose calculation method based on an improved U-Net. This involves constructing and labeling a diverse power line sample library, creating an "original image-mask" pair dataset, and employing an improved U-Net network with an embedded CBAM attention mechanism, such as... Figure 2 As shown, targeted training is performed using a hybrid loss function to solve the imbalance problem of slender target samples and output a high-precision mask Image2. Simultaneously, the incremental SfM algorithm is used to accurately solve the camera pose Pose1 and the initial sparse point cloud Data1 through SIFT feature matching, RANSAC elimination and PnP registration, and global bundle adjustment (BA) optimization.
[0091] S2 includes the following:
[0092] S21 Construct a power line image sample library. By acquiring diverse scene images containing power lines (Image_open), and then preprocessing the images, construct the power line image dataset (Image-dataset).
[0093] S22 Use image annotation software to annotate the power line regions in a single image Image_open in the power line image dataset, set them as labels to represent power line categories, save the annotation file in the same path as the image, and generate a JSON format annotation file containing power line categories;
[0094] S23 converts the JSON format annotation files into binary mask format files required for training in batches, and constructs a semantic segmentation annotation dataset Labeling1_dataset containing "original image-mask" pairs;
[0095] S24 constructs a U-Net network based on an encoder-decoder structure and embeds a Convolutional Block Attention (CBAM) module in its bottleneck layer. The CBAM module consists of cascaded channel attention sub-modules. Spatial attention submodule Composition, Feature Map The computational logic after processing by this module is as follows:
[0096]
[0097]
[0098] in, For the input feature map, Channel attention-weighted features, For the final output features, This indicates element-wise multiplication;
[0099] S25 divides the semantic segmentation annotation dataset Labeling1_dataset into a training set and a test set, and constructs a model using binary cross-entropy loss. and Dice coefficient loss Weighted Mixed Loss Function This formula is used to solve the sample imbalance problem where the proportion of pixels in the electric field lines is extremely small.
[0100]
[0101]
[0102] in, To balance the weighting coefficients, For the first The predicted probability value of each pixel. No. The actual label value of each pixel. Total number of pixels For smoothing terms
[0103] S26 The improved U-Net network is iteratively trained using the training set and the hybrid loss function. When the intersection-over-union (IoU) index of the test set reaches a preset threshold, the optimal model weights are saved. The weights are then loaded to perform inference on the acquired image sequence Image1 and output the power line mask image Image2.
[0104] S27 For the image sequence Image1, the key points and descriptors of each frame are extracted using the SIFT algorithm, feature matching is performed using the nearest neighbor ratio method, and mismatched point pairs are removed using the RANSAC algorithm.
[0105] S28 Initializes images with a wide baseline by solving the essential matrix using epipolar geometric constraints to recover the initial pose; then registers the remaining images frame by frame using the PnP (Perspective-n-Point) algorithm and calculates the three-dimensional coordinates of spatial points using triangulation to generate the initial sparse point cloud Data1.
[0106] S29 Constructs an objective function to minimize the reprojection error, performing joint nonlinear optimization on the pose parameters Pose1 of all cameras and the spatial coordinates of the initial sparse point cloud Data1. The specific objective function is as follows:
[0107]
[0108] in, This represents the total weight projection error. The total number of images, This represents the total number of three-dimensional points. For visibility identification, when the camera Observed three-dimensional points The value is 1 if the condition is met, otherwise it is 0. For three-dimensional points In the camera The observed pixel coordinates; For the camera projection model, the three-dimensional points Project onto the pixel coordinate system; after optimization and convergence, output the final camera pose parameters Pose1 and the initial sparse point cloud Data1.
[0109] Secondly, step S3 of the above scheme provides a physical modeling method based on the geometric basis vectors of the catenary. It uses mask consistency checks to filter out background noise in the sparse point cloud, and employs the RANSAC algorithm to fit the optimal vertical plane and remove outliers. The core lies in constructing a set of locally orthogonal basis vectors, and establishing the horizontal span direction based on the cross product of the plane normal vector and the gravity vector. and vertical The three-dimensional point set is mapped to local scalar coordinates; based on this, a nonlinear regression algorithm is applied to solve the catenary equation parameters, and the parameters are obtained through linear combination of basis vectors. This generates a three-dimensional theoretical spatial trajectory of electric field lines, Curve1, that conforms to the laws of gravity.
[0110] S3 includes the following:
[0111] S31 For each 3D point in the initial sparse point cloud Data1, map it to the electric field mask image Image2, determine whether its projection coordinates fall within the effective mask area of Image2, and remove points that fall into the background area as noise, thereby extracting the electric field candidate point set Points1 that is consistent with the image semantics.
[0112] S32 uses the Random Sample Consensus Algorithm (RANSAC) to process the candidate point set Points1 of the power line. During the iteration process, the number of interior points falling within the threshold range of the plane model is counted. The plane containing the most interior points is selected as the best vertical plane Plane1, and points that are more than a preset threshold away from the plane are removed as outliers to obtain the optimized interior point set.
[0113] S33 analyzes the spatial geometric properties of the vertical plane Plane1 in the world coordinate system and constructs a set of locally orthogonal basis vectors describing the shape of the electric field lines. Specifically, this includes calculating the geometric centroid of the interior point set as the reference origin. Define the normal vector of the perpendicular plane Plane1 as... The unit vector of the direction of gravity is A horizontal unit vector along the span of the power line is constructed using the vector cross product operation. Construct a vertical unit vector perpendicular to the span direction. Thus establishing a system from the origin and basis vectors Defined local geometric reference frame;
[0114] S34 sets the points in the interior point set. Decomposed to the local geometric reference frame to obtain scalar coordinates ,in A catenary equation model is constructed based on the decomposed scalar coordinates. The optimal shape parameters are calculated using a nonlinear regression equation. and position parameters This yields the equation of the curve in the two-dimensional plane;
[0115] S34 Based on the solved equation parameters and the local geometric basis vectors, a sampling step size is set within the span of the electric line. The three-dimensional theoretical spatial trajectory Curve1 of the electric line is directly constructed using linear vector combination. Any point on the trajectory... The formula for calculating three-dimensional coordinates is:
[0116]
[0117] in, For along the horizontal vector The sampled value of the independent variable for direction.
[0118] Thirdly, step S4 of the above scheme provides a physical guidance initialization method for the three-dimensional Gaussian sputtering model. First, uniform discretization sampling is performed based on the total arc length of the theoretical trajectory Curve1 to directly determine the initial center position of the Gaussian sphere. Secondly, by differentiating the trajectory equation, the tangent vector is obtained, and a rotation quaternion is constructed. This ensures that the principal axis of the Gaussian sphere is strictly aligned with the local orientation of the electric field lines. Finally, the Gaussian sphere is given anisotropic scale properties, with the longitudinal scale along the tangential direction set to be much larger than the radial scale, so that its initial shape is a slender ellipsoid that fits the geometry of the electric field lines. High confidence opacity is initialized, thereby significantly improving the convergence speed and topological accuracy of the model.
[0119] S4 includes the following:
[0120] S41 Based on the three-dimensional trajectory equation obtained in step S3 Calculate the total arc length of the curve and set the linear sampling density along the curve path, in the parameters Uniform sampling is performed within the domain to generate a series of discrete three-dimensional coordinate point sets. Each coordinate point in the three-dimensional coordinate point set is used as the initial position mean parameter of an independent three-dimensional Gaussian sphere. This initializes a set of electric field line Gaussian point clouds composed of multiple three-dimensional Gaussian spheres.
[0121] S42 Iterate through each 3D Gaussian sphere in the set of Gaussian point clouds of electric lines, and calculate the position mean of the 3D Gaussian sphere. The corresponding curve parameters, for the three-dimensional trajectory equation Find information about the parameters The first derivative is obtained and normalized to obtain the unit tangent vector of the point at that sampling location. Construct a vector with the tangent vector The local rotation matrix in the X-axis principal axis direction and the rotation matrix Convert to unit rotation quaternion As the initial rotation parameter of the three-dimensional Gaussian sphere, the principal axis direction of the Gaussian sphere is strictly aligned with the local spatial orientation of the electric field lines;
[0122] S43 assigns anisotropic initial scale parameters to each three-dimensional Gaussian sphere in the set. Specifically, the scale vector is defined. ,in The longitudinal scale is along the direction of the tangent vector. Define the radial dimension perpendicular to the tangent vector plane and set constraints. This causes the initialized three-dimensional Gaussian sphere to exhibit a slender ellipsoidal geometry that extends tangentially along the electric field lines;
[0123] S44 sets the initial opacity parameters for all three-dimensional Gaussian spheres. Set the confidence level to a high value and initialize the initial three-dimensional Gaussian spherical harmonic function coefficients SH to zero or according to the average color value of the projected image to complete the physical guidance initialization construction of the three-dimensional Gaussian sputtering model.
[0124] Fourthly, step S5 of the above scheme provides a hybrid loss function and joint optimization strategy that includes geometric constraint terms. Specifically, the physical constraint loss is calculated as the mean of the shortest Euclidean distance from the center of the Gaussian sphere (belonging to the electric field line category) to the theoretical trajectory Curve1, which serves as a penalty to eliminate geometric drift. During iterative training using a differentiable rasterizer, the backpropagation algorithm is used to update the Gaussian sphere properties, and an alternating optimization strategy is executed. That is, while optimizing the Gaussian parameters, the catenary equation is dynamically fine-tuned according to the Gaussian sphere distribution to ensure that the reconstructed model remains continuous and conforms to physical laws in weakly textured regions until the model converges.
[0125] S5 includes the following:
[0126] The S51 uses a differentiable rasterizer to project a three-dimensional Gaussian sphere onto a two-dimensional image plane to obtain a rendered image from the current viewpoint. And calculate its comparison with the real image. Photometric consistency loss between parameters; S52 constructs physical constraint loss based on geometric distance field. This is used to constrain the center of the Gaussian sphere of the electric field line from deviating from the theoretical catenary trajectory.
[0127] S53 Define the total loss function ,in For luminance loss, For structural similarity loss, Loss due to physical constraints;
[0128] S54 Calculate Physical Constraint Loss Specifically, this involves calculating the mean of the shortest Euclidean distance from the center of each three-dimensional Gaussian sphere belonging to the electric field line category to the three-dimensional theoretical space trajectory Curve1 in the current iteration step.
[0129] S55 uses the backpropagation algorithm to update the position, rotation, scaling, and spherical harmonic coefficients of the three-dimensional Gaussian sphere, and fine-tunes the catenary equation parameters every preset number of iterations according to the current Gaussian sphere distribution, so that the physical model can dynamically adapt to the fine-tuning of the three-dimensional Gaussian distribution until the total loss function converges.
[0130] Example 2
[0131] This application also provides a three-dimensional Gaussian sputtering electric field line reconstruction system based on catenary physical constraints, the system comprising:
[0132] The data acquisition module is configured to acquire a monocular image sequence Image1 captured for power line inspection scenarios;
[0133] The data preprocessing module is configured to perform semantic segmentation on the image sequence Image1 based on a deep learning model to extract the power line mask image Image2, and use the motion reconstruction structure algorithm to solve the camera pose parameter Pose1 and the initial sparse point cloud Data1 of the scene.
[0134] The physical modeling module is configured to perform catenary parameter regression based on the initial sparse point cloud Data1 to construct a three-dimensional theoretical spatial trajectory Curve1 of the electric field line that conforms to gravity constraints.
[0135] The physics-guided initialization module is configured to generate an initialized three-dimensional Gaussian sphere along the three-dimensional theoretical space trajectory Curve1 and endow it with an anisotropic geometric shape to complete the physical prior initialization of the three-dimensional Gaussian sputtering model.
[0136] The model joint optimization module is configured to construct a hybrid objective function that includes photometric consistency loss and catenary geometric constraint loss. It uses the image sequence Image1 and camera pose Pose1 to iteratively train and update the parameters of the initialized model, and outputs the electric power line 3D reconstruction model Model1.
[0137] The parameter parsing module is configured to parse and output the spatial geometric parameters of the electric power line based on the three-dimensional reconstruction model Model1 of the electric power line.
[0138] This application presents a method and system for reconstructing 3D Gaussian sputtering electric lines based on catenary physical constraints. This method combines semantic segmentation, structure-of-motion (SOG) reconstruction, physical model fitting, and neural rendering techniques to construct an integrated framework for achieving refined 3D reconstruction of electric lines in live-line working scenarios. For weakly textured, highly reflective electric line images acquired by a live-line working robot using monocular vision, an improved U-Net segmentation network is first used to remove complex backgrounds, obtaining the electric line mask and initial sparse point cloud. Then, the least squares method is used to fit the catenary parameters of the sparse point cloud to obtain the theoretical 3D spatial trajectory of the electric line. This theoretical trajectory is then used to guide the initial distribution of a 3D Gaussian sphere. Finally, physical constraints are introduced into the loss function to iteratively optimize the 3D Gaussian sputtering model, outputting a continuous and geometrically accurate 3D electric line model. Compared to traditional pure vision-based 3D reconstruction methods, which are susceptible to weak textures leading to reconstruction breaks, and standard 3D Gaussian sputtering methods, which are prone to generating floating noise, the method proposed in this invention, which incorporates the physical priors of the catenary into the initialization and optimization process, offers higher geometric continuity and spatial accuracy, as well as stronger robustness compared to traditional data-driven methods. Furthermore, when faced with line reconstruction tasks of different voltage levels or different sag shapes, it only requires automatic adjustment of fitting parameters based on the physical model, without the need for re-collecting massive amounts of high-precision LiDAR ground truth data for fully supervised training in specific scenarios. This significantly reduces data acquisition costs and algorithm deployment difficulty, and can meet the real-time and safety requirements of robot autonomous operation for environmental perception.
Claims
1. A method for reconstructing three-dimensional Gaussian sputtering electric field lines based on catenary physical constraints, characterized in that, The method includes the following steps: S1. Obtain Image1, a monocular image sequence collected for power line inspection scenarios, and construct a power line image sample library based on the monocular images; S2 performs semantic segmentation on the image sequence Image1 to extract the power line mask image Image2; and uses the structure of motion recovery algorithm SfM to solve the image sequence Image1 to obtain the camera pose parameters Pose1 and the initial sparse point cloud Data1 of the scene. S3 performs physical parameter fitting based on the initial sparse point cloud Data1, calculates the catenary equation parameters, and constructs the three-dimensional theoretical spatial trajectory Curve1 of the electric field line. S4. Using the three-dimensional theoretical spatial trajectory Curve1 as the center, generate a three-dimensional Gaussian sphere along the trajectory spatial position to complete the physical prior initialization of the three-dimensional Gaussian sputtering model. S5 Construct a loss function that includes the catenary physical constraint term, and use the image sequence Image1 and camera pose Pose1 to iteratively train the initialized 3D Gaussian sputtering model. When the training loss converges, the final 3D electric field line reconstruction model Model1 is obtained. S6 Analyzes the spatial geometric parameters of the power line according to the three-dimensional reconstruction model Model1 of the power line to realize the reconstruction of the three-dimensional shape of the power line.
2. The method for reconstructing three-dimensional Gaussian sputtering electric field lines based on catenary physical constraints according to claim 1, characterized in that, Specifically, S2 is: S21 acquires monocular images of diverse scenes containing power lines, and constructs a power line image dataset (Image-dataset) after preprocessing the images; S22 Use image annotation software to annotate the power line regions in a single image Image_open in the Image-dataset, set them as labels to represent power line categories, save the annotation file in the same path as the image, and generate a JSON format annotation file containing power line categories; S23 The labeled files are batch converted into binary mask format files required for training, and a semantic segmentation labeled dataset Labeling1_dataset containing original image-mask pairs is constructed; S24 Constructs a U-Net network based on an encoder-decoder structure, embedding a convolutional block attention (CBAM) module in the bottleneck layer (Bottleneck) of the U-Net network. The CBAM module consists of cascaded channel attention sub-modules. Spatial attention submodule Composition, Feature Map The computational logic after processing by this module is as follows: ； ； in, For the input feature map, Channel attention-weighted features, For the final output features, This indicates element-wise multiplication; S25 divides the semantic segmentation annotation dataset Labeling1_dataset into a training set and a test set, and constructs a model using binary cross-entropy loss. and Dice coefficient loss Weighted Mixed Loss Function This formula is used to solve the sample imbalance problem where the proportion of pixels in the electric field lines is extremely small. ； ； in, To balance the weighting coefficients, For the first The predicted probability value of each pixel. No. The actual label value of each pixel. Total number of pixels For smoothing terms; S26 The U-Net network is iteratively trained using the training set and the hybrid loss function. When the Intersection over Union (IoU) index of the test set reaches a preset threshold, the optimal model weights are saved. The optimal model weights are loaded to perform inference on the acquired image sequence Image1 and output the power line mask image Image2. S27 For the image sequence Image1, the key points and descriptors of each frame are extracted using the SIFT algorithm, feature matching is performed using the nearest neighbor ratio method, and mismatched point pairs are removed using the RANSAC algorithm. S28 Initializes images with a wide baseline by using epipolar geometry constraints to solve the essential matrix to recover the initial pose; then registers the remaining images frame by frame using the n-point perspective algorithm PnP and calculates the three-dimensional coordinates of spatial points using triangulation to generate the initial sparse point cloud Data1. S29 Constructs an objective function to minimize the reprojection error, performing joint nonlinear optimization on the pose parameters Pose1 of all cameras and the spatial coordinates of the initial sparse point cloud Data1. The specific objective function is as follows: ； in, This represents the total weight projection error. The total number of images, This represents the total number of three-dimensional points. For visibility identification, when the camera Observed three-dimensional points The value is 1 if the condition is met, otherwise it is 0. For three-dimensional points In the camera The observed pixel coordinates; For the camera projection model, the three-dimensional points Project onto the pixel coordinate system; after optimization and convergence, output the final camera pose parameters Pose1 and the initial sparse point cloud Data1.
3. The method for reconstructing three-dimensional Gaussian sputtering electric field lines based on catenary physical constraints according to claim 1, characterized in that, Specifically, S3 is: S31 Iterate through each 3D point in the initial sparse point cloud Data1, project it onto the power line mask image Image2, retain the 3D points whose projected coordinates are located within the effective mask area, and form a power line candidate point set Points1 that is consistent with the image semantics from these points; S32 uses the Random Sample Consensus Algorithm (RANSAC) to process the candidate point set Points1 of the power line. During the iteration process, the number of interior points falling within the threshold range of the plane model is counted. The plane containing the most interior points is selected as the best vertical plane Plane1, and points that are more than a preset threshold away from the plane are removed as outliers to obtain the optimized interior point set. S33 analyzes the spatial geometric properties of the vertical plane Plane1 in the world coordinate system and constructs a set of locally orthogonal basis vectors describing the shape of electric field lines, specifically including: Calculate the geometric centroid of the interior point set as the reference origin. Define the normal vector of the perpendicular plane Plane1 as... The unit vector of the direction of gravity is A horizontal unit vector along the span of the power line is constructed using the vector cross product operation. Construct a vertical unit vector perpendicular to the span direction. Thus establishing a system from the origin and basis vectors Defined local geometric reference frame; S34 sets the points in the interior point set. Decomposed to the local geometric reference frame to obtain scalar coordinates ,in A catenary equation model is constructed based on the decomposed scalar coordinates. The optimal shape parameters are calculated using a nonlinear regression equation. and position parameters This yields the equation of the curve in the two-dimensional plane; S34 Based on the solved equation parameters and local orthogonal basis vectors Within the span of the power line, a sampling step size is set, and the three-dimensional theoretical spatial trajectory Curve1 of the power line is directly constructed using linear vector combination. Any point on the trajectory... The formula for calculating three-dimensional coordinates is: ； in, For along the horizontal vector The sampled value of the independent variable for direction.
4. The method for reconstructing three-dimensional Gaussian sputtering electric field lines based on catenary physical constraints according to claim 1, characterized in that, Specifically, S4 is: S41 Based on the three-dimensional trajectory equation obtained in step S3 Calculate the total arc length of the curve and set the linear sampling density along the curve path, in the parameters Uniform sampling is performed within the domain to generate a series of discrete three-dimensional coordinate point sets. Each coordinate point in the three-dimensional coordinate point set is used as the initial position mean parameter of an independent three-dimensional Gaussian sphere. This initializes a set of electric field line Gaussian point clouds composed of multiple three-dimensional Gaussian spheres. S42 Iterate through each 3D Gaussian sphere in the set of Gaussian point clouds of electric lines, and calculate the position mean of the 3D Gaussian sphere. The corresponding curve parameters, for the three-dimensional trajectory equation Find information about the parameters The first derivative is obtained and normalized to obtain the unit tangent vector of the point at that sampling location. Construct a vector with the tangent vector The local rotation matrix in the X-axis principal axis direction and the rotation matrix Convert to unit rotation quaternion As the initial rotation parameter of the three-dimensional Gaussian sphere, the principal axis direction of the Gaussian sphere is strictly aligned with the local spatial orientation of the electric field lines; S43 assigns anisotropic initial scale parameters to each three-dimensional Gaussian sphere in the set. Specifically, the scale vector is defined. ,in The longitudinal scale is along the direction of the tangent vector. Define the radial dimension perpendicular to the tangent vector plane and set constraints. This causes the initialized three-dimensional Gaussian sphere to exhibit a slender ellipsoidal geometry that extends tangentially along the electric field lines; S44 sets the initial opacity parameters for all three-dimensional Gaussian spheres. Set the confidence level to a high value and initialize the initial three-dimensional Gaussian spherical harmonic function coefficients SH to zero or according to the average color value of the projected image to complete the physical guidance initialization construction of the three-dimensional Gaussian sputtering model.
5. The method for reconstructing three-dimensional Gaussian sputtering electric field lines based on catenary physical constraints according to claim 1, characterized in that, Specifically, S5 is: The S51 uses a differentiable rasterizer to project a three-dimensional Gaussian sphere onto a two-dimensional image plane to obtain a rendered image from the current viewpoint. And calculate its comparison with the real image. Loss of photometric consistency between them; S52 Constructing Physical Constraint Loss Based on Geometric Distance Field This is used to constrain the center of the Gaussian sphere of the electric field line from deviating from the theoretical catenary trajectory. S53 Define the total loss function ,in For luminance loss, For structural similarity loss, Loss due to physical constraints; S54 Calculate Physical Constraint Loss Specifically, this involves calculating the mean of the shortest Euclidean distance from the center of each three-dimensional Gaussian sphere belonging to the electric field line category to the three-dimensional theoretical space trajectory Curve1 in the current iteration step. S55 uses the backpropagation algorithm to update the position, rotation, scaling and spherical harmonic coefficient properties of the three-dimensional Gaussian sphere, and fine-tunes the catenary equation parameters according to the current Gaussian sphere distribution every preset number of iterations, so that the physical model can dynamically adapt to the fine-tuning of the three-dimensional Gaussian distribution until the total loss function converges. S56 When the total loss function converges, the iterative optimization process stops, and the final optimized 3D Gaussian scene representation is output; the geometric and appearance attributes of each 3D Gaussian sphere after convergence are retained; this series of discrete but continuously rendering anisotropic Gaussian spheres form a 3D electric line reconstruction model Model1 with catenary morphological characteristics.
6. The method for reconstructing three-dimensional Gaussian sputtering electric field lines based on catenary physical constraints according to claim 1, characterized in that, Specifically, S6 is: S61 Extract the center coordinates of all Gaussian spheres in the finally converged 3D reconstruction model Model1 of the electric line, construct a 3D point cloud set of the electric line, use principal component analysis (PCA) to calculate the first principal direction of the 3D point cloud set of the electric line as the overall spatial direction vector of the electric line, identify the endpoint coordinates of the point cloud set at both ends of the direction, and calculate the Euclidean distance between the two endpoints to determine the span parameters of the electric line. S62 Project the three-dimensional point cloud set of the power line onto a vertical plane composed of the gravity direction vector and the spatial direction vector. Calculate the vertical distance from the lowest point of the point cloud to the line connecting the two endpoints in the two-dimensional projection plane, i.e., the maximum sag height. Combined with the known ambient temperature and meteorological conditions, use the state equation to verify whether the sag parameters meet the safe operation standards. S63 performs equidistant slice sampling on the 3D reconstruction model Model1 along the span direction of the power line, calculates the line connecting the centroids of the Gaussian spheres in each slice section, generates a smooth central skeleton line, compares the coordinates of the central skeleton line with the true value data measured by lidar or total station, evaluates the geometric accuracy of the reconstruction model, and finally outputs an automated power line reconstruction report containing a 3D model visualization view and geometric parameters of sag, span, and direction.
7. A three-dimensional Gaussian sputtering electric field line reconstruction system based on catenary physical constraints, characterized in that, The system includes: The data acquisition module is configured to acquire a monocular image sequence Image1 captured for power line inspection scenarios; The data preprocessing module is configured to perform semantic segmentation on the image sequence Image1 based on a deep learning model to extract the power line mask image Image2, and use the motion reconstruction structure algorithm to solve the camera pose parameter Pose1 and the initial sparse point cloud Data1 of the scene. The physical modeling module is configured to perform catenary parameter regression based on the initial sparse point cloud Data1 to construct a three-dimensional theoretical spatial trajectory Curve1 of the electric field line that conforms to gravity constraints. The physics-guided initialization module is configured to generate an initialized three-dimensional Gaussian sphere along the three-dimensional theoretical space trajectory Curve1 and endow it with an anisotropic geometric shape to complete the physical prior initialization of the three-dimensional Gaussian sputtering model. The model joint optimization module is configured to construct a hybrid objective function that includes photometric consistency loss and catenary geometric constraint loss. It uses the image sequence Image1 and camera pose Pose1 to iteratively train and update the parameters of the initialized model, and outputs the electric power line 3D reconstruction model Model1. The parameter parsing module is configured to parse and output the spatial geometric parameters of the electric power line based on the three-dimensional reconstruction model Model1 of the electric power line.
8. The three-dimensional Gaussian sputtering electric field line reconstruction system based on catenary physical constraints according to claim 7, characterized in that, The data preprocessing module includes a dataset construction unit, a labeling processing unit, a model training unit, a mask inference unit, and a pose calculation unit. The dataset construction unit is used to acquire public datasets and self-built robot perspective datasets to construct an image dataset of power lines. The annotation processing unit is used to use an image annotation tool to perform pixel-level annotation on the power line regions in the power line image dataset, generate annotation files, and convert them into a mask format. The model training unit is used to divide the dataset into a training set and a test set, use the training set to train the improved U-Net model with embedded CBAM modules, and save the weights when the IoU of the test set reaches a set threshold. The mask inference unit is used to load the stored weights, segment the image sequence Image1, and output the power line mask image Image2. The pose calculation unit is used to process the image sequence Image1 using the incremental SfM algorithm, and outputs sparse point cloud Data1 and camera pose Pose1.
9. A three-dimensional Gaussian sputtering electric field line reconstruction system based on catenary physical constraints according to claim 7, characterized in that, The physical guidance initialization module and the model joint optimization module include a physical seeding unit, a geometric initialization unit, a loss calculation unit, and a parameter update unit. The physical seeding unit is used to set the sampling density according to the total arc length of the catenary trajectory Curve1 and uniformly generate the initial center position of the three-dimensional Gaussian sphere along the trajectory. The geometric initialization unit is used to calculate the tangent vector at each point on the trajectory, construct a rotation quaternion to align the principal axis of the Gaussian sphere with the tangent, and endow the Gaussian sphere with an anisotropic scale property of an elongated ellipsoid. The loss calculation unit is used to calculate the shortest Euclidean distance from the center of the Gaussian sphere belonging to the electric field line category to the theoretical trajectory Curve1 in real time during training iterations, as a physical constraint loss term; The parameter update unit is used to construct a total loss function that includes physical constraints, and uses the backpropagation algorithm and gradient descent strategy to update the position, rotation, scaling and spherical harmonic coefficients of the Gaussian sphere, and outputs the converged 3D electric field model Model1.