Power transmission line defect generation method based on physical agent model and differentiable rendering

Abstract

This invention discloses a method for generating transmission line defects based on a physical proxy model and differentiable rendering, relating to the fields of computer vision and power system defect detection technology. The method constructs a 3D model of the transmission line incorporating defect parameters, uses a graph neural network to predict the node displacement field satisfying physical constraints, and generates a point cloud of the defect scene. It then combines curvature-adaptive 3D Gaussian differentiable rendering to output a preliminary rendered image and depth map. Texture is enhanced through a latent diffusion model, and a geometric anchoring mechanism based on edges and depth is introduced to maintain structural consistency. Finally, a visibility-weighted mask is generated using a depth cache, supplemented by a dynamic threshold for closed-loop evaluation and fine-tuning. This invention effectively generates labeled defect data that conforms to physical and mechanical properties and has a stable geometric structure, improving the training effect of defect detection models.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and power system defect detection technology, and in particular to a method for generating transmission line defects based on a physical proxy model and differentiable rendering. Background Technology

[0002] With the rapid development of smart grids, utilizing drones for inspection combined with deep learning models for transmission line defect detection has become a crucial aspect of ensuring the safe and stable operation of power systems. In this process, the generalization ability and detection accuracy of deep learning models are highly dependent on the scale and quality of the input training data. In actual transmission line inspection operations, a large number of images are collected in normal conditions, while samples containing real defects such as insulator damage, broken conductor strands, or deformed hardware are relatively scarce. This often leads to the challenge of an imbalanced positive and negative sample ratio during model training.

[0003] To expand defect sample sets, researchers often employ basic geometric and pixel enhancement methods such as image flipping, cropping, and color transformation, or use generative models to synthesize images. However, basic geometric enhancement methods remain at the two-dimensional pixel level and cannot generate defect samples with three-dimensional physical and mechanical deformation characteristics, making it difficult for synthesized images to conform to actual physical laws in terms of mechanical structure. Furthermore, when existing 3D differentiable rendering and 2D image generation models are used in combination, they often struggle to maintain accurate topological structure and relative spatial position in complex transmission line scenarios, easily resulting in poor physical consistency and geometric feature drift.

[0004] Furthermore, the complex structure of transmission line components and their susceptibility to mutual occlusion make it difficult for traditional data augmentation methods to automatically generate accurate pixel-level visibility labels and distortion-free semantic segmentation masks. This increases the cost of subsequent manual annotation and limits the feature learning effectiveness of detection models in complex occlusion scenarios. There is an urgent need in the industry for a data augmentation solution that combines physical and mechanical principles with high-fidelity rendering to generate high-quality defect images and automated annotation data. Summary of the Invention

[0005] The purpose of this invention is to provide a method for generating transmission line defects based on a physical proxy model and differentiable rendering, so as to solve the technical problems in the prior art, such as the lack of training data for transmission line defects, and the lack of physical and mechanical deformation characteristics, poor physical consistency, and easy drift of geometric topology in conventional data augmentation and generation models.

[0006] This invention provides a method for generating transmission line defects based on a physical proxy model and differentiable rendering, comprising the following steps: A three-dimensional parametric model of the transmission line components is constructed and defect parameters are introduced. The physical properties of the corresponding components are extracted to construct the basic three-dimensional model. The nodal displacement field of the basic 3D model is predicted using a trained graph neural network surrogate model. The prediction process of the nodal displacement field is to forcefully satisfy the static equilibrium equation constraint by minimizing the deviation between the predicted nodal internal force and the external load, and to update the vertex coordinates based on the predicted nodal displacement field to generate a point cloud of the defect scene. The camera pose is set around the point cloud of the defect scene for three-dimensional Gaussian sputtering differentiable rendering. During the rendering initialization stage, the anisotropic covariance matrix of the Gaussian sphere is adaptively set according to the local principal curvature direction of the component to force the major axis direction of the Gaussian sphere to be aligned with the tangent of the slender component, thereby generating a defect rendering image and a rendering depth map. The defect rendering image is input into the latent diffusion model for texture and lighting enhancement. A geometric anchoring mechanism is introduced in the inverse denoising latent space of the latent diffusion model. The edge feature map of the defect rendering image is extracted as a hard spatial constraint, and the rendering depth map is extracted simultaneously to constrain the relative depth relationship of the output image. In this way, the topological structure and spatial position of the defect are locked during the texture enhancement process, and the final defect synthesis image is generated. The depth information of the point cloud of the defect scene under the corresponding camera view is extracted by using a rasterized depth buffer, the pixel-level visibility of the defect area relative to the occluder is calculated, and a visibility-weighted mask with continuous weights and automated annotation data are generated. By extracting statistical features from historical qualified image batches to construct a dynamic adaptive threshold, the distribution distance of the defective synthetic images is evaluated in a closed loop and the model is fine-tuned as needed. Finally, the qualified images are used to train the transmission line defect detection model.

[0007] Optionally, the forced satisfaction of the static equilibrium equation constraint is achieved by introducing a residual physical loss function during the training phase of the graph neural network surrogate model. The residual physical loss function calculates the sum of the products of the connection stiffness of local network nodes and the relative displacement difference between adjacent nodes, and calculates the residual between the sum of these products and the external physical loads on the nodes. By optimizing the network parameters, the residual is made to approach zero, so as to avoid distortion deformation that violates physical properties.

[0008] Optionally, an extrapolation degradation protection mechanism is provided during the prediction of the nodal displacement field: The input defect parameters are monitored in real time; When the defect parameter exceeds the boundary of the set confidence interval covered during the training of the graph neural network surrogate model, the direct inference of the graph neural network surrogate model is interrupted, and the algorithm automatically switches to a reduced-order finite element solver to perform corrective solution operations on the local region that exceeds the interval.

[0009] Optionally, the rendering initialization phase also includes surface sampling and transparency prior steps: Seed points are adaptively sampled from the triangular mesh surface of the basic 3D model according to curvature, and the sampling density is increased in high curvature regions. Based on the preset real material properties and background properties of the transmission line components, independent initial opacity values ​​are set for the metal surface, the semi-transparent glass insulator surface, and the background sky sphere to suppress floating artifacts during rendering.

[0010] Optionally, the loss function corresponding to the geometric anchoring mechanism in the potential diffusion model includes the standard potential diffusion loss, edge consistency loss, and depth consistency loss; The edge consistency loss calculation outputs the mean square error between the image and the defect rendering image at the edge feature level. The depth consistency loss extracts the predicted depth of the output image through a pre-trained monocular depth estimation network and calculates the absolute error between the predicted depth and the rendered depth map to perform depth anti-drift correction.

[0011] Optionally, the texture and lighting enhancement of the latent diffusion model are constrained by a multimodal conditional control mechanism, specifically including: The edge feature map of the defective rendered image is extracted as an image control condition; The structured text prompts containing weather conditions, lighting conditions, defect type descriptions, surface detail features, and background environment descriptions are analyzed to generate text feature vectors as text control conditions. The inverse denoising process is driven under the dual guidance of the image control conditions and the text control conditions.

[0012] Optionally, the step of generating a visibility-weighted mask with continuous weights specifically includes: The point cloud of the defect scene is rendered into a scene depth map, and the set of pixel coordinates after the three-dimensional bounding box of the defect region is projected onto the image plane is obtained. For each pixel in the pixel coordinate set, compare the scene depth with the actual depth of the defect; When occlusion exists, the difference between the scene depth and the actual depth of the defect is calculated. The pixel-level visibility in the zero-to-one range is calculated by using the ratio of this difference to the preset maximum occlusion attenuation distance. Multiply the pixel-level visibility by the pixel's basic geometric mask to output the visibility-weighted mask of the semi-transparent soft label shape.

[0013] Optionally, the step of generating automated annotation data incorporates distortion correction processing for virtual camera lenses: Obtain lens distortion parameters including radial and tangential distortion coefficients; In the normalized plane, the distortion parameters are used to perform polynomial spatial position correction on the normalized coordinates. Then, the corrected coordinates are mapped to the final pixel coordinates through the camera intrinsic parameter matrix to generate distortion-free semantic segmentation masks and bounding boxes.

[0014] Optionally, the step of constructing a dynamic adaptive threshold and triggering model fine-tuning on demand includes: Calculate the mean distribution distance of historical qualified image batches, and add twice the standard deviation of historical distribution distance to this mean as the dynamically updated adaptive judgment threshold; If the distribution distance of the current batch exceeds the adaptive judgment threshold, the parameters of the backbone network of the latent diffusion model and the graph neural network surrogate model are kept frozen. Features are extracted only from samples with qualified physical structures in the current batch, and the parameters of the low-rank adaptation layer of the latent diffusion model are fine-tuned iteratively until the batch distribution distance drops to within the threshold and converges.

[0015] Optionally, before using the qualified images for training the transmission line defect detection model, a sample matching construction step is also included: A hybrid dataset of the synthesized defect image and the real-collected defect image is established, and the mixing ratio of the two is dynamically adjusted according to a preset defect type scarcity index. For defect types that are determined to be rare or scarce, the threshold for the proportion of their corresponding defect synthetic images in the current mixed dataset is increased. For defects classified as "normal", their percentage threshold is limited to a preset "normal" range.

[0016] The present invention has achieved the following beneficial effects: This invention constructs a three-dimensional parametric model containing physical properties and uses a graph neural network to predict the nodal displacement field that satisfies mechanical equilibrium constraints. This enables the generated defect samples to possess physical and mechanical deformation characteristics that conform to objective laws, overcoming the shortcomings of conventional two-dimensional geometric enhancement in mechanical performance. By combining three-dimensional Gaussian differentiable rendering with a latent diffusion model that introduces a geometric anchoring mechanism, the topological structure and spatial location of defects are locked while enhancing image texture and lighting details. This improves the poor physical consistency and geometric feature drift of traditional generative models in complex power transmission scenarios. By using rasterized depth buffers to generate continuously weighted visibility-weighted masks and automatically performing lens distortion correction, accurate labels are automatically output, reducing manual annotation costs and improving the usability of samples in multi-occlusion scenarios. In addition, the introduction of a closed-loop evaluation and fine-tuning mechanism with dynamic adaptive thresholds and a dynamic adjustment strategy for sample ratios ensures the overall distribution quality of the synthesized images, providing rich and reliable training data support for downstream power transmission line defect detection models. The generated data not only optimized the model parameter distribution at the algorithm level, but also produced significant and measurable technological effects in the physical world: the defect detection model trained using the aforementioned high-fidelity and mechanically continuous synthetic data was deployed in physical inspection equipment such as drones and inspection robots on real transmission lines, significantly improving the recognition accuracy of physical inspection terminals for minor physical deformations (such as initial conductor strand breakage, slight tensile deformation, and initial insulator cracks). In actual power grid inspection operations, it effectively overcame the vulnerability of traditional data-driven models in harsh field environments such as strong light reflection and complex background obstruction, significantly reducing the false alarm and missed detection rates of actual physical equipment. Thus, the leap in data quality was substantially transformed into an enhanced guarantee of safe operation of transmission lines, a heavy asset physical facility.

[0017] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 The overall flowchart of the method for generating transmission line defects based on a physical proxy model and differentiable rendering provided in the embodiments of the present invention is shown below. Figure 2This is a flowchart illustrating the construction of a basic three-dimensional model and the introduction of physical defect parameters in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the prediction of node displacement fields using a graph neural network surrogate model in an embodiment of the present invention. Figure 4 This is a flowchart of the process of three-dimensional Gaussian sputtering differentiable rendering and potential diffusion model generation in an embodiment of the present invention; Figure 5 This is a flowchart illustrating the process of generating automated annotation data and performing lens distortion correction in an embodiment of the present invention; Figure 6 This is a flowchart illustrating the closed-loop evaluation and model fine-tuning mechanism for constructing a dynamic adaptive threshold in an embodiment of the present invention. Detailed Implementation

[0020] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0021] To address the lack of training data for existing transmission line inspection target detection models, the inability of conventional geometric data augmentation methods to generate defect samples with physical and mechanical deformation characteristics, and the problems of poor physical consistency and geometric feature drift in traditional differentiable rendering and generative models, such as... Figure 1 As shown in the embodiments of this application, a method for generating transmission line defects based on a physical proxy model and differentiable rendering is provided. The execution entity of this method can be a server node or computing cluster configured with a central processing unit, a graphics processing unit, and a large-capacity random access memory. The execution entity calls pre-written program instructions to perform steps covering parametric modeling, physical field proxy prediction, differentiable rendering, diffusion generation, and visibility annotation to output an annotated synthetic image of transmission line defects.

[0022] In the specific implementation process, such as Figure 2As shown, the method first includes constructing a three-dimensional parametric model of transmission line components and introducing defect parameters, extracting the physical properties of the corresponding components to construct a basic three-dimensional model. Specifically, the executing entity obtains geometric structure drawing data and surface reflectivity measurement data of standard transmission line components through a data interface. These standard components include suspension insulator strings, multi-split conductors, vibration dampers, connecting hardware, and tension clamps. Based on the geometric structure drawing data, the executing entity generates a three-dimensional basic mesh model of each component in an internal three-dimensional Cartesian coordinate system, and configures the spatial resolution of the surface triangular facets of the basic mesh model within a preset range of 0.5 mm to 2 mm. The executing entity reads the surface material texture attribute file corresponding to the aforementioned three-dimensional basic mesh model, specifically including a diffuse texture matrix, a normal texture matrix, and a surface roughness texture matrix, and maps and binds the coordinate data of the aforementioned texture matrices to the corresponding three-dimensional mesh vertex data structure.

[0023] After generating the basic mesh model, the execution entity configures parameterized definition interfaces for each model instance, constructing a geometric parameter space containing multiple dimensions. Taking the suspension insulator model as an example, its configured parameter space variables include integer variables for the number of insulators (ranging from 1 to 30), floating-point variables for skirt diameter and structural height, and material type variables represented by discrete codes. For the conductor model, the parameter space variables include conductor model code variables, floating-point variables for sag (ranging from 0 to 15 meters), and integer variables for the number of splits (ranging from 1 to 4). For the hardware model, the parameter configuration includes model specification data, spatial attachment point coordinate data in the local coordinate system, and a topological connection matrix. The execution entity serializes the model data configured with parameterized interfaces and stores it in a storage medium, forming a parameterized 3D model library. Based on this, the execution entity receives scene assembly configuration parameters, reads the corresponding parameter space variable values, and performs translation and rotation matrix transformations on the 3D mesh models of different components to assemble and generate a basic 3D model of the entire coupled scene, including transmission towers, insulator strings, and conductors.

[0024] After constructing the basic 3D model, the execution entity initializes the corresponding defect parameter space in memory. This defect parameter space is represented by a multidimensional tensor data structure. The defect category variables are discretized and stored using a one-hot encoding format. In this embodiment, the defect category encoding covers insulator skirt breakage, insulator steel foot corrosion, conductor strand breakage, vibration damper slippage, and hardware deformation. For conductor strand breakage, the number of broken strands is configured as a discrete integer variable of 1 to 20 strands; for vibration damper slippage, the slippage distance is configured as a continuous floating-point variable of 0 to 500 mm; and for hardware deformation, the bending angle is configured as a continuous floating-point variable of 0 to 30 degrees. The physical location of the defect is defined using a 3D spatial offset vector data relative to the origin of the local coordinate system of the basic 3D model. The severity of the defect is calculated using continuous scalar values.

[0025] Understandably, the execution entity invokes the Latin hypercube sampling algorithm during the preprocessing stage. This algorithm divides the defect parameter space into multiple equally probable multidimensional grid intervals and randomly extracts numerical vectors from each interval to generate a defect parameter feature set containing specific categories, locations, and severity. The execution entity parses and maps the sampled defect parameter feature set to the memory address of the corresponding grid cell in the basic 3D model. By modifying the underlying material property tensor of the grid cells in the defect-affected region, the physical defect injection mapping is performed. For example, when the parsed defect category is insulator skirt breakage, the execution entity reduces the local elastic modulus parameter value of the grid cells within the corresponding spatial coordinate range; when the defect category is conductor strand breakage, the execution entity proportionally reduces the cross-sectional moment of inertia constant of the corresponding 3D cylindrical cell section according to the strand breakage ratio parameter. Specifically, the reduction of the local elastic modulus parameter value is linearly reduced based on the proportion of the broken volume, with a reduction coefficient ranging from [0.4, 0.9]; the formula for calculating the reduction ratio of the cross-sectional moment of inertia constant is as follows: ,in The number of shares lost. This represents the total number of shares. After completing the physical property mapping of the defect parameters, the execution entity allocates a virtual physical simulation environment data structure, transforming the load data of the external environment into external load boundary conditions acting on the mesh nodes of the basic 3D model. The load data includes wind speed variables, wind direction angle variables, temperature variables, as well as the gravity load vector calculated from material density and gravitational acceleration, and the icing mass load vector transformed from the external icing thickness.

[0026] After configuring the load and defect boundary conditions, such as Figure 3As shown, the executing entity uses a trained graph neural network surrogate model to predict the nodal displacement field of the underlying 3D model. Specifically, the executing entity transforms the finite element mesh data structure of the underlying 3D model into graph structure data. The set of nodes in the graph structure corresponds to the set of vertices in the finite element mesh, and the set of edges corresponds to the topological connection lines of the mesh's faces. During the forward inference phase of the graph neural network surrogate model, the encoder layer receives the aforementioned defect parameter vector, performs dimensionality reduction projection, and outputs the initial feature vectors of each node to the processor layer. The processor layer contains a multi-layered cascaded message passing network structure. In the message passing of each layer, the state features of a node are iteratively updated according to its own state and the states of its neighboring nodes. The calculation formula for its data flow is as follows: ; In the above formula, Indicates the first in the figure The node completes the first... The node feature vector output after layer message passing calculation; This represents the first multilayer perceptron network, used to perform nonlinear mapping operations on the aggregated neighbor features and the current node features; Indicates the first The node at the th The feature vector of the layer input; Indicates targeting the node All first-order neighbor nodes The feature output vector is subjected to traversal aggregation and summation operations; Represents a node The set of neighbor node indexes; This represents the second multilayer perceptron network, used to compute the feature vector of a single message pass between a node and its neighboring nodes; the first multilayer perceptron network ( The second multilayer perceptron network (MLB) consists of two fully connected layers and a ReLU activation function, with both the input and output feature vectors having a dimension of 128. It also contains two fully connected layers and a ReLU activation function. The input dimension is the sum of the concatenated 256-dimensional node features and edge features, and the output feature vector dimension is 128. Representing neighboring nodes In the The feature vector of the layer; Indicates the connection node With nodes The edge feature vector matrix contains the initial Euclidean space distance between nodes and the equivalent spring stiffness physical quantity.

[0027] The graph neural network surrogate model's loss function during the training phase forces the predicted node's internal forces to external loads, thereby forcibly satisfying the static equilibrium equation constraints. The total loss function calculated by the executing entity... The weighted sum of the standard mean square error loss term and the residual physical loss term is calculated as follows: ; In the above formula, This represents the global total loss function value used to update the network weight parameters during the backpropagation algorithm. The scalar value representing the standard mean square error loss function is used to quantify the data deviation between the model output displacement and the reference true displacement. This represents the dimensionless hyperparameter set as the balancing weight coefficient for the physical loss term; considering the significant order of magnitude difference between the rigid constraints dominated by the physical stiffness matrix and the high-frequency distributions such as data-driven latent variable denoising during backpropagation, if... Setting these values ​​to static values ​​can easily lead to gradient explosion or network optimization collapse. Therefore, the Grad Norm gradient dynamic balancing scheduling mechanism is embedded in the model training pipeline. During each backpropagation iteration, the execution entity calculates the standard mean squared error loss gradient in real time. Norm and Gradient of Residual Physical Loss The norm is used to dynamically calculate and update the physical loss weight coefficients for the current time step by minimizing the relative difference between the loss of each task and the target equilibrium gradient. This mechanism forces the gradient magnitudes of all losses to remain on the same order of magnitude during backpropagation in the shared network layer, thereby ensuring the smooth convergence of the rigid constraints of physical laws and the fitting of high-frequency features of the generative model during the joint optimization process. This represents the calculated scalar value of the residual physical loss function, used to constrain the displacement field output by the network to satisfy the mechanical equilibrium condition.

[0028] The above standard mean square error loss term The calculation formula is as follows: ; In the above formula, This represents the result of the mean square error calculation; Represents the training batch dataset The number of samples in the sample; Indicates the training dataset Each group contains a defect parameter vector. With the target displacement vector The paired data is traversed and accumulated. Indicates the current state of the weight matrix. Below, the model responds to the input parameters The predicted three-dimensional displacement field matrix; The target displacement field matrix represents the reference. This indicates that the 2-norm square operation is performed on the difference between the prediction matrix and the reference matrix.

[0029] The above residual physical loss function The calculation formula is as follows: ; In the above formula, This represents the root mean square value of the calculated internal force value at a local node and the residual of the external load matrix; This represents the total number of nodes contained in the graph data structure; This represents all nodes within the graph structure. Perform traversal and accumulation; Indicates targeting the node For all its connected neighbor nodes Perform the traversal and summation; Indicates the connection node with neighboring nodes The local equivalent connection stiffness constant between them; The node representing the predicted output The three-dimensional displacement vector; The node representing the predicted output The three-dimensional displacement vector; This represents the relative displacement difference vector between two nodes; Indicates external application applied to the node The three-dimensional environmental load vector on; This indicates that the L2 norm square operation is performed on the mechanical residual vector.

[0030] During the prediction of the nodal displacement field, the execution entity is equipped with an extrapolation degradation protection mechanism. The execution entity extracts the input defect parameter data and compares and calculates it with the preset confidence interval envelope boundary data.

[0031] When the input defect parameter tensor exceeds the confidence interval envelope boundary, the execution entity triggers a heterogeneous computing asynchronous scheduling mechanism: On the GPU, the inference thread of the current graph neural network is suspended, its computational context is pushed onto the stack, and the boundary tensor data in the current video memory is asynchronously copied to the system main memory via the PCIe bus. Simultaneously, the reduced-order finite element solver (ROM) daemon process on the CPU is awakened. The reduced-order finite element solver performs a corrected solution operation on the linear algebraic equations based on the stiffness matrix and load vector of the out-of-limit local region mesh elements.

[0032] To address the issue of physical continuity between the reduced-order finite element method (FEB) solution and the surrounding displacement field generated by the graph neural network (GNN) in the boundary region, and to avoid geometric discontinuities and stress singularities caused by direct memory splicing, the execution entity constructs a boundary region with a width of [missing information - likely a value] between the FEB solution region and the GNN prediction region. The overlapping transition region. Within this region, the executing entity employs a cubic spline interpolation smoothing algorithm based on spatial distance to perform weighted fusion calculations on the displacement vectors of both components. Its spatial weighting coefficients... satisfy The continuity condition (i.e., smooth transition of the first derivatives of displacement and strain) is specifically calculated using the following formula: ; In the above formula, This represents the physically continuous node displacement vector that is finally output after smoothing and fusion calculation. This represents the cubic spline interpolation spatial weighting coefficient calculated based on the spatial distance distribution of nodes. Its value smoothly transitions between 0 and 1 and satisfies the continuity of the first derivative. This indicates the spatial relative position or distance coordinates of a grid node within a defined overlapping transition zone; This represents the local corrected displacement vector calculated and output by the reduced-order finite element solver (ROM) for the over-limit region; This represents the corresponding node displacement vector predicted by the Graph Neural Network (GNN) surrogate model.

[0033] Through the smooth fusion operation of the displacement and strain fields described above, the executing entity generates a nodal displacement field matrix with strict physical continuity, thereby updating the vertex coordinates of the basic three-dimensional model and outputting point cloud data of the defect scene without mechanical faults.

[0034] After acquiring point cloud data of the defect scene, such as Figure 4 As shown, the execution entity sets the camera pose around the point cloud of the defect scene for 3D Gaussian sputtering differentiable rendering. The execution entity configures the translation vector, rotation quaternion, and internal optical matrix parameters of the virtual camera in the coordinate system. During the rendering initialization phase, the execution entity executes an adaptive surface curvature sampling rule. For regions where the average principal curvature exceeds a first preset curvature threshold, the execution entity increases the sampling step size and seed point density; for tubular geometric regions where the principal curvature is below a second preset curvature threshold, the axial sampling interval is increased. The first preset curvature threshold is set to 0.15. The second preset curvature threshold is set to 0.02. Both methods are obtained by statistically averaging the local curvature of CAD drawings of standard components of transmission lines and applying an empirical fluctuation range of ±20%. The execution entity calculates the Jacobian matrix of the local mesh of the basic 3D model to extract the principal curvature direction vector, and sets the covariance matrix of the anisotropic Gaussian sphere based on the principal curvature direction vector. For slender structural components, the system aligns the direction vector of the major semi-axis of the Gaussian sphere with the tangential vector of the component's central axis through matrix multiplication, and sets the scalar parameters of the major and minor semi-axis respectively to generate needle-like anisotropic 3D Gaussian distribution characteristics. The execution entity synchronously reads the material encoding matrix and assigns initial opacity attributes based on preset constants to Gaussian sphere instances representing metal, glass, and sky backgrounds respectively.

[0035] After initializing the Gaussian sphere parameters, the execution entity emits virtual projection lines from the camera's projection center and calculates the color matrix values ​​of the corresponding pixels based on the characteristic properties of the intersecting Gaussian spheres. The color synthesis calculation formula based on alpha blending is as follows: ; In the above formula, Represents pixel coordinates on the image plane The final output color vector; This indicates that for all pixels that penetrate this pixel... The Gaussian sphere along the ray path undergoes traversal and cumulative calculation. Indicates the first The opacity scalar parameter of a three-dimensional Gaussian sphere; Indicates the first After a 3D Gaussian sphere undergoes perspective projection transformation by a camera, its pixel coordinates are... Two-dimensional Gaussian distribution probability weights on a plane; Indicates the first The primary color vectors of a three-dimensional Gaussian sphere are calculated using spherical harmonic functions.

[0036] During the training and optimization phases of the differentiable rendering model, the execution entity configuration includes image pyramid data structures with various resolutions. The loss function of the rendering network... The calculation formula is as follows: ; In the above formula, This represents the comprehensive rendering loss function used to update the 3D Gaussian property parameters; This indicates the set full resolution, half resolution, and quarter resolution scales. The hierarchy is traversed and accumulated. Indicated at the resolution scale The initial image matrix generated by the prediction; Representation and Scale The corresponding reference image matrix; The L1 norm is used to calculate the difference between corresponding pixel positions in the predicted image and the reference image. This represents the set structural similarity index loss weight constant; This represents the scalar value calculated for the structural similarity index between the predicted image and the reference image at full resolution. After iterative calculation, the execution subject outputs the converged defect rendering image and the corresponding viewpoint rendering depth map.

[0037] The execution entity inputs the defect rendering image into the latent diffusion model for texture and lighting enhancement, and introduces a geometric anchoring mechanism during the reverse denoising process. The execution entity invokes pre-configured diffusion model network components. It converts a structured text prompt sequence containing environmental parameters, lighting parameters, and defect material parameters into word embedding feature vectors, which are then input into the diffusion model backbone network. Simultaneously, the execution entity invokes an edge detection algorithm operator to extract a binarized edge feature map matrix from the defect rendering image, which is then input into the model's spatial control channel.

[0038] In the inverse denoising iteration of the latent space of the latent diffusion model, the conditional probability prediction formula of the network is as follows: ; In the above formula, Indicates the weight state is Network based on condition sets Predicted Gaussian noise residual tensor; This represents the denoising backbone network of the diffusion model; Indicates being at a discrete time step The low-dimensional latent variable matrix; The time step index scalar represents the denoising process. This represents the edge feature map matrix calculated from the defect-rendered image, serving as the image spatial control condition; This represents the semantic feature vector output by the text encoder after processing structured text prompts, serving as a text control condition. In the denoising backbone network... In the fusion calculation, the image spatial control conditions After processing with a zero-convolutional layer, the data is directly concatenated (concat) along the channel dimension to the feature map downsampled by the network; the text control conditions Then, the cross-attention matrix multiplication is used to map and inject the feature into the intermediate feature layer of the network layer by layer.

[0039] Two-dimensional edge feature maps with pixel-level distribution To address the mismatch between the scale of the feature distribution in the latent space compressed by the variational autoencoder and the actual feature distribution, the execution entity pre-configures a spatial dimensionality reduction feature pyramid mapping mechanism. Specifically, the extracted pixel-level features... The input is a downsampling module containing multiple layers of 3×3 convolutional kernels with a stride of 2, and it performs 8x downsampling compression in the spatial dimension. The resolution scale, and simultaneously utilize Convolution maps its number of channels to the same target dimension as the latent variable.

[0040] After the explicit alignment of feature scale and dimension is completed, the dimension-reduced edge feature tensor is then processed by a zero convolution layer with weights initialized to zero. Then, it is directly concatenated (concat) into the feature tensor downsampled by the denoising backbone network in the corresponding channel dimension of the feature latent space, thereby closing the hard constraint transformation mapping from pixel space to latent space at the feature level.

[0041] To limit geometric drift in the generated image, the executing agent configures a geometric anchoring mechanism. The total geometric anchoring loss function... The calculation formula is as follows: ; In the above formula, This represents the objective function value for geometric anchoring global constraints; This represents the standard latent variable denoising loss value calculated by the latent diffusion model; A weighted constant scalar representing the set tolerance for control edge deformation; This represents the calculated value of the two-dimensional edge consistency loss; A weighted constant scalar representing the set tolerance for control depth coordinate offset; This represents the calculated value of the 3D depth consistency loss. In this embodiment, The value range is [0.5, 1.5], and the recommended value is 1.0; The value range is [0.1, 0.5], and the recommended value is 0.2.

[0042] The above formula includes an edge consistency loss term. The calculation formula is as follows: ; In the above formula, This represents the calculation result of the edge feature deviation; This indicates that the Canny operator is invoked to modify the current predicted image of the diffusion network. Extract the generated dynamic prediction edge matrix; This represents a pre-extracted and fixed reference edge feature map matrix; This indicates that the 2-norm square operation is performed on the difference matrix of the two marginal matrices.

[0043] The above formula includes a deep consistency loss term. The calculation formula is as follows: ; In the above formula, This represents the calculation results of the depth structure deviation; This represents the predicted depth map matrix output after the network predicts the image and inputs it into a pre-trained monocular depth estimation network; the pre-trained monocular depth estimation network specifically adopts the MiDaSv3.1 network architecture and loads the DPT-Large pre-trained weight parameter model. Represents the physical reference depth map matrix derived from the differentiable rendering pipeline; This indicates that the L1 norm operation is performed on the difference matrix of the two depth map matrices. After the constraint calculation process converges, the execution entity outputs the defect synthesis image after decoding by the variational autoencoder.

[0044] After the image is generated, such as Figure 5 As shown, the execution entity extracts depth information using a rasterized depth buffer to generate automated annotation data. The execution entity initiates the graphics processor's rasterization pipeline, projects the defect scene point cloud data from perspective, and reads and writes to the depth buffer to generate a full-scene absolute depth map array corresponding to the image coordinates. The execution entity obtains the set of pixel coordinates mapped from the defect's 3D bounding box to a 2D plane. While traversing this coordinate set, the execution entity performs the following mask matrix multiplication calculation formula: ; In the above formula, Indicates pixel coordinates The specific floating-point values ​​of the visibility-weighted mask matrix elements calculated at the output; Represents the elements of the basic two-dimensional binary geometric mask matrix generated by vertex perspective projection; This represents the pixel-level visibility scalar of the physical entity corresponding to the pixel that is not obscured by the foreground.

[0045] pixel-level visibility The specific attenuation calculation formula is as follows: ; In the above formula, This represents the calculated pixel visibility value. This represents a non-linear truncation function used to keep the output result greater than or equal to zero; Represents the coordinates read from the depth buffer matrix. The actual visible foreground occlusion depth value at the location; This represents the theoretically unoccluded physical depth of the defect surface, calculated using 3D bounding box perspective projection. This represents the occlusion parallax attenuation threshold constant set in the configuration file. The occlusion parallax attenuation threshold constant... The setting is 150 mm, meaning that when the difference between the theoretical physical depth and the actual occlusion depth reaches 150 mm, the pixel-level visibility decay is 0.

[0046] To correct the spherical refraction deviation of the virtual camera lens, the execution entity performs distortion correction processing on the camera's intrinsic parameter data. The execution entity extracts first-order and second-order radial distortion coefficient arrays and tangential distortion coefficient arrays. Let the basic coordinate vector of the normalized imaging plane be... The executing entity first calculates the square of the normalized coordinate center distance vector. The calculation formula is as follows: ; In the above formula, It represents the squared scalar of the length of the vector from the origin to the pixel in a two-dimensional plane; This represents the value of the horizontal component in the normalized coordinate system. This represents the value of the vertical component in the normalized coordinate system.

[0047] The executing entity calculates the lateral correction coordinate values. The formula is as follows: ; In the above formula, This represents the normalized x-axis position data after distortion polynomial compensation. and These represent the original x and y coordinates without correction. and These represent the first-order and second-order radial distortion correction coefficients, respectively; and These represent the first-order and second-order tangential distortion correction coefficients, respectively. and This represents the square of the center distance vector and its fourth power.

[0048] The executing entity calculates the longitudinal correction coordinate values. The formula is as follows: ; In the above formula, This represents the normalized ordinate position data after distortion compensation; the meanings and sources of the other variable identifiers are consistent with the physical parameters in the abscissa correction formula.

[0049] After completing the distortion correction of the normalized plane, the execution entity calls the camera intrinsic parameter mapping matrix to perform a linear transformation, the transformation formula of which is as follows: ; In the above formula, This represents the transpose vector of a two-dimensional matrix in the image pixel coordinate system. This represents the combination of sensor focal length and optical center constant. The intrinsic parameter matrix of the camera; This represents the transpose of a homogeneous coordinate system, including distortion correction results and supplemented with scale parameters. The executing entity outputs the mapped coordinates as pixel-level semantic segmentation mask files and target bounding box files.

[0050] In the image output stage, such as Figure 6 As shown, the execution entity constructs a dynamic adaptive threshold and triggers a model fine-tuning mechanism as needed. The execution entity loads the evaluation feature extraction network module to extract the depth feature matrices of the current batch of synthetic defect images and the baseline real defect images, and calculates the Fréchet Inception Distance index between the two feature distributions. The index calculation formula is as follows: ; In the above formula, An evaluation scalar representing the distance between the distributional differences of characteristic features; This represents the true feature mean vector obtained by performing a mean operation on the feature vector data extracted from the real defect image sequence; This represents the mean vector of generated features calculated from the sequence of synthesized defect images; The L2 norm square operation represents the difference between two feature mean vectors; This indicates that an algebraic operation to find the trace of a square matrix is ​​performed. This represents the covariance matrix generated from the feature tensors of the real data. This represents the covariance matrix generated by the feature tensor of the generated data; It represents the principal square root matrix after the product of two covariance matrices.

[0051] The executing entity calculates an adaptive judgment threshold based on the characteristic distribution of the system operation records, and the calculation formula is as follows: ; In the above formula, Represents a dynamic adaptive decision threshold scalar used to determine the state of image batches; This represents the average statistical value of the distribution difference distance index of images that have been determined to be qualified batches and extracted from the historical database. This represents the standard deviation component of the volatility calculated from the above historical indicator series.

[0052] The executing entity compares the indicator data of the current batch with the adaptive judgment threshold. If the calculated result is greater than the threshold, the executing entity sends an interrupt request to the fine-tuning scheduling process. The system freezes the gradient update permissions of the graph neural network and the diffusion model backbone, extracts the synthetic image tensor containing specific error features, and calls the optimization operator to perform backpropagation weight parameter fine-tuning only on the low-rank adaptation (LoRA) layer of the diffusion model bypass until the evaluation scalar of the verification batch converges to the adaptive judgment threshold region. The executing entity reads the defect scarcity ratio lookup table. For low-frequency rare defect categories, the threshold setting parameter for the proportion of its synthetic images in the mixed dataset is increased; for regular defects, the corresponding ratio upper limit parameter is decreased. The specific judgment logic and corresponding relationship table are as follows: when the total number of samples of a certain type of defect (such as vibration damper slippage) in the real dataset is less than 5%, it is judged as a rare defect, and the threshold for the proportion of its synthetic images in the mixed dataset is increased to the range of [30%, 50%]; when the proportion of a certain type of defect is greater than 20%, it is judged as a regular defect, and its proportion threshold is forcibly limited to the regular range of [10%, 20%]. The generated data is packaged and fed into a downstream object detection framework for iterative calculation and training of the loss function.

[0053] Specifically, taking the generation of conductor strand breakage defects as an example, the execution entity extracts the mesh data array of multi-strand stranded aluminum conductors from the library, configures the basic parameters, and then injects the strand breakage ratio and offset coordinate parameters. During load distribution, the gravity scalar and static wind load coefficient are input. When the graph neural network calculates the displacement field, it determines that the input reduced section moment of inertia variable exceeds the lower limit of the confidence interval envelope set for safety. The system suspends the inference of the current tensor node and calls the reduced-order finite element subroutine matrix solver to calculate the tear displacement tensor data of the local fracture mesh. In the rendering step, the ray projection generates the corresponding needle-shaped Gaussian covariance mapping along the conductor's central axis and calculates the projection. Subsequently, the diffusion pipeline loads a string feature vector containing "steel core oxidation rust and edge breakage" for noise stripping. The visibility calculation module, by comparing the buffer depth, applies a continuous variable attenuation numerical calculation with a damping coefficient to the bottom steel core mesh within the fracture depression, which is obscured by the outer aluminum strand, generating a gradually changing calibration mask label. After the calculation module confirms that the indicator result is below the adaptive threshold line, the file enters the packaging sequence.

[0054] The method steps described in the foregoing embodiments can be implemented at the physical hardware level by an electronic device with logic processing capabilities. This electronic device includes a bus architecture, with the bus connecting at least one central processing unit (CPU), and volatile random access memory and non-volatile read-only memory for storing instruction sets and data structures. The electronic device also includes a network communication interface for data throughput of configuration parameter packets and raw mesh packets, and is equipped with display and input peripheral interfaces. The processor reads the program code stored in memory through internal registers, driving its arithmetic logic unit and floating-point unit to execute instructions for extracting parameter matrices, graph network connection data, image pixel tensor data, Gaussian parameter kernel functions, and fine-tuning gradients, as well as multiply-accumulate and nonlinear function mapping instructions, thereby realizing the aforementioned business process flow including 3D generation and loop closure detection.

[0055] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for generating transmission line defects based on a physical proxy model and differentiable rendering, characterized in that, Includes the following steps: A three-dimensional parametric model of the transmission line components is constructed and defect parameters are introduced. The physical properties of the corresponding components are extracted to construct the basic three-dimensional model. The nodal displacement field of the basic 3D model is predicted using a trained graph neural network surrogate model. The prediction process of the nodal displacement field is to forcefully satisfy the static equilibrium equation constraint by minimizing the deviation between the predicted nodal internal force and the external load, and to update the vertex coordinates based on the predicted nodal displacement field to generate a point cloud of the defect scene. The camera pose is set around the point cloud of the defect scene for three-dimensional Gaussian sputtering differentiable rendering. During the rendering initialization stage, the anisotropic covariance matrix of the Gaussian sphere is adaptively set according to the local principal curvature direction of the component to force the major axis direction of the Gaussian sphere to be aligned with the tangent of the slender component, thereby generating a defect rendering image and a rendering depth map. The defect rendering image is input into the latent diffusion model for texture and lighting enhancement. A geometric anchoring mechanism is introduced in the inverse denoising latent space of the latent diffusion model. The edge feature map of the defect rendering image is extracted as a hard spatial constraint, and the rendering depth map is extracted simultaneously to constrain the relative depth relationship of the output image. In this way, the topological structure and spatial position of the defect are locked during the texture enhancement process, and the final defect synthesis image is generated. The depth information of the point cloud of the defect scene under the corresponding camera view is extracted by using a rasterized depth buffer, the pixel-level visibility of the defect area relative to the occluder is calculated, and a visibility-weighted mask with continuous weights and automated annotation data are generated. By extracting statistical features from historical qualified image batches to construct a dynamic adaptive threshold, the distribution distance of the defective synthetic images is evaluated in a closed loop and the model is fine-tuned as needed. Finally, the qualified images are used to train the transmission line defect detection model.

2. The method for generating transmission line defects based on a physical proxy model and differentiable rendering according to claim 1, characterized in that, The method for forcibly satisfying the static equilibrium equation constraint is as follows: during the training phase of the graph neural network surrogate model, a residual physical loss function is introduced. The residual physical loss function calculates the sum of the products of the connection stiffness of local network nodes and the relative displacement difference between adjacent nodes, and calculates the residual between the sum of these products and the external physical loads on the nodes. By optimizing the network parameters, the residual is made to approach zero, so as to avoid distortion deformation that violates physical properties.

3. The method for generating transmission line defects based on a physical proxy model and differentiable rendering according to claim 1, characterized in that, An extrapolation degradation protection mechanism is implemented during the prediction of the nodal displacement field. The input defect parameters are monitored in real time; When the defect parameter exceeds the boundary of the set confidence interval covered during the training of the graph neural network surrogate model, the direct inference of the graph neural network surrogate model is interrupted, and the algorithm automatically switches to a reduced-order finite element solver to perform corrective solution operations on the local region that exceeds the interval.

4. The method for generating transmission line defects based on a physical proxy model and differentiable rendering according to claim 1, characterized in that, The rendering initialization phase also includes surface sampling and transparency prior steps: Seed points are adaptively sampled from the triangular mesh surface of the basic 3D model according to curvature, and the sampling density is increased in high curvature regions. Based on the preset real material properties and background properties of the transmission line components, independent initial opacity values ​​are set for the metal surface, the semi-transparent glass insulator surface, and the background sky sphere to suppress floating artifacts during rendering.

5. The method for generating transmission line defects based on a physical proxy model and differentiable rendering according to claim 1, characterized in that, The loss functions corresponding to the geometric anchoring mechanism in the potential diffusion model include standard potential diffusion loss, edge consistency loss, and depth consistency loss. The edge consistency loss calculation outputs the mean square error between the image and the defect rendering image at the edge feature level. The depth consistency loss extracts the predicted depth of the output image through a pre-trained monocular depth estimation network and calculates the absolute error between the predicted depth and the rendered depth map to perform depth anti-drift correction.

6. The method for generating transmission line defects based on a physical proxy model and differentiable rendering according to claim 1, characterized in that, The texture and lighting enhancement of the latent diffusion model are constrained by a multimodal conditional control mechanism, specifically including: The edge feature map of the defective rendered image is extracted as an image control condition; The structured text prompts containing weather conditions, lighting conditions, defect type descriptions, surface detail features, and background environment descriptions are analyzed to generate text feature vectors as text control conditions. The inverse denoising process is driven under the dual guidance of the image control conditions and the text control conditions.

7. The method for generating transmission line defects based on a physical proxy model and differentiable rendering according to claim 1, characterized in that, The step of generating a visibility-weighted mask with continuous weights specifically includes: The point cloud of the defect scene is rendered into a scene depth map, and the set of pixel coordinates after the three-dimensional bounding box of the defect region is projected onto the image plane is obtained. For each pixel in the pixel coordinate set, compare the scene depth with the actual depth of the defect; When occlusion exists, the difference between the scene depth and the actual depth of the defect is calculated. The pixel-level visibility in the zero-to-one range is calculated by using the ratio of this difference to the preset maximum occlusion attenuation distance. Multiply the pixel-level visibility by the pixel's basic geometric mask to output the visibility-weighted mask of the semi-transparent soft label shape.

8. The method for generating transmission line defects based on a physical proxy model and differentiable rendering according to claim 1, characterized in that, The step of generating automated labeled data incorporates distortion correction processing for virtual camera lenses: Obtain lens distortion parameters including radial and tangential distortion coefficients; In the normalized plane, the distortion parameters are used to perform polynomial spatial position correction on the normalized coordinates. Then, the corrected coordinates are mapped to the final pixel coordinates through the camera intrinsic parameter matrix to generate distortion-free semantic segmentation masks and bounding boxes.

9. The method for generating transmission line defects based on a physical proxy model and differentiable rendering according to claim 1, characterized in that, The steps of constructing a dynamic adaptive threshold and triggering model fine-tuning as needed include: Calculate the mean distribution distance of historical qualified image batches, and add twice the standard deviation of historical distribution distance to this mean as the dynamically updated adaptive judgment threshold; If the distribution distance of the current batch exceeds the adaptive judgment threshold, the parameters of the backbone network of the latent diffusion model and the graph neural network surrogate model are kept frozen. Features are extracted only from samples with qualified physical structures in the current batch, and the parameters of the low-rank adaptation layer of the latent diffusion model are fine-tuned iteratively until the batch distribution distance drops to within the threshold and converges.

10. The method for generating transmission line defects based on a physical proxy model and differentiable rendering according to claim 1, characterized in that, Before using the qualified images for training the transmission line defect detection model, a sample matching construction step is also included: A hybrid dataset of the synthesized defect image and the real-collected defect image is established, and the mixing ratio of the two is dynamically adjusted according to a preset defect type scarcity index. For defect types that are determined to be rare or scarce, the threshold for the proportion of their corresponding defect synthetic images in the current mixed dataset is increased. For defects classified as "normal", their percentage threshold is limited to a preset "normal" range.

Images