An unsupervised detection method for power distribution equipment based on three-dimensional model rendering

Abstract

The application discloses a kind of based on three-dimensional model guide's distribution network equipment unsupervised detection method, this method combines distribution network equipment three-dimensional shape priori with unsupervised learning framework, realizes the unlabelled detection of distribution network equipment.Through constructing distribution network equipment three-dimensional model library and rendering generation with accurate label synthetic sample, combined with pseudo-label iterative updating and texture library dynamic optimization mechanism, adapt image geometric transformation using three-dimensional enhancement technology, accurately predict equipment category, three-dimensional pose and two-dimensional bounding box through three-dimensional detection network.The application realizes that distribution network equipment detection is completely separated from artificial labeling, with strong generalization ability, strong detection performance, wide application scene and the like advantages, suitable for power distribution network unmanned aerial vehicle inspection, multi-type equipment identification, cross voltage grade detection and other various power inspection scenes.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and power line inspection, specifically to an unsupervised inspection method for power distribution equipment based on 3D model rendering. Background Technology

[0002] Power equipment inspection is a fundamental task for defect identification in power distribution network inspection images. Existing inspection technologies mostly rely on fully supervised learning models, requiring large amounts of labeled datasets to ensure the model's generalization ability. However, while power distribution network inspection images can be rapidly accumulated through regular collection by drones, the labeling work is limited by professionalism and confidentiality, and can only be completed by power experts or inspection personnel. This results in high labor costs, insufficient labeled data, and the labeling granularity is mostly at the image level, which cannot meet the training requirements of detectors.

[0003] To alleviate the dependence on annotations, existing technologies propose methods such as pre-training, weakly supervised learning, and data augmentation. However, these methods need to be designed for specific equipment categories and rely on a certain proportion of labeled data, making it difficult to adapt to the inspection needs of dozens of types of equipment in the power distribution network (crossarm insulators, clamps, vibration dampers, etc.). Their generalization ability is limited at new voltage levels or equipment categories. General unsupervised inspection methods rely on general visual priors. Due to the significant differences in the distribution between inspection images and general images, they cannot effectively extract semantic features of power equipment, resulting in poor detection performance.

[0004] Therefore, there is an urgent need for a universal unsupervised inspection method that is free from the dependence on manual annotation and adaptable to complex power distribution network scenarios. Power equipment possesses three-dimensional shape consistency and rigidity invariance; equipment on the same line exhibits highly uniform form; and equipment in different regions has similar size and appearance. This characteristic provides a core prior for unsupervised inspection. Based on this, this invention designs an unsupervised inspection method for power distribution equipment based on 3D model rendering. Summary of the Invention

[0005] To address the shortcomings of existing distribution network equipment detection methods, such as reliance on manual labeling, weak generalization ability, and poor adaptability, this invention provides an unsupervised detection method for distribution network equipment based on a 3D model. This method utilizes the prior knowledge of the 3D shape of power equipment, generates labeled samples through 3D model rendering, and combines an iterative training framework to achieve unsupervised detection, thereby improving data utilization and detection generalization ability.

[0006] The present invention proposes an unsupervised detection method for distribution network equipment based on a three-dimensional model, which includes the following steps: Step 1: Establish a two-dimensional image coordinate system and a three-dimensional camera coordinate system and determine the intrinsic parameter matrix. Based on the public dimensions of the power distribution equipment, construct a set of triangular rigid body mesh models and define three-dimensional pose labels and two-dimensional bounding box labels. Step 2: Construct a 3D detection network for predicting 3D pose and 2D bounding boxes; Step 3: Build and initialize the texture library. The texture library stores multiple textures of the triangular rigid body mesh model. It is obtained or updated through random indexing. Initialization is completed based on the natural material color of the device. For multi-material devices, multiple colors are used for uniform random initialization. Step 4: Based on the 3D detection network from Step 2, the texture library from Step 3, and the renderer, construct an unsupervised 3D detection training network and perform unsupervised iterative training. During the cyclic training process, 3D labels are randomly generated for each unlabeled image and the synthesized image is rendered. A training set is constructed based on the synthesized image, the previous round of pseudo-labels, and the synthesized labels. After 3D enhancement, the 3D detection training network is trained. The prediction results are filtered to generate the current round of pseudo-labels and the texture library is updated. The cycle is iterated until convergence. Finally, the trained unsupervised 3D detection network is used to detect images of distribution network equipment.

[0007] Preferably, the step one of establishing the two-dimensional image coordinate system and the three-dimensional camera coordinate system and determining the intrinsic parameter matrix includes: Given dimensions Images of distribution network equipment The origin is the top left corner of the image. The right direction is the horizontal axis. The downward direction is the vertical axis. Establish a two-dimensional image coordinate system Establish a 3D camera coordinate system The horizontal and vertical axes are aligned with the coordinate system of the two-dimensional image, i.e. , vertical axis Pointing inside the image plane; camera origin The vertical projection onto the image is the image center. And the projection distance is the focal length. Fixed image size , and focal length The intrinsic parameter matrix is ​​obtained as follows: .

[0008] Preferably, the step one of constructing a triangular rigid body mesh model set based on the publicly disclosed dimensions of the distribution network equipment includes: constructing a triangular rigid body mesh model set based on the dimensions of the distribution network equipment disclosed by the power equipment manufacturer. Each model ,in For vertex set, For triangle face indexing, For vertex textures.

[0009] Preferably, the definition of the 3D pose label and the 2D bounding box label in step one includes: Triangular Rigid Body Mesh Model Set In the 3D camera coordinate system Rigid body motion in [the context] is equivalent to [the motion of] the set of vertices. The transformations corresponding to rigid body motion are determined by orthogonal rotation matrices. Translation vector Sure; for Each point It can be transformed into the following ways : ; Define the 3D pose label of an object ,in, It is a quaternion, where For the real part, It is the imaginary part; Let the translation vector be defined; define the 2D bounding box label. Used to characterize the foreground region of objects in an image of power distribution network equipment. These correspond to the left, top, right, and bottom coordinates of the border, respectively.

[0010] Preferably, the 3D detection network in step two includes a backbone network and a 3D detection layer. The backbone network outputs multi-scale feature maps, and the 3D detection layer has four decoupled branches: a classification branch and a classification branch. Translation regression branch Quaternion Regression Branch Boundary regression branch ; The classification branch activates the sigmoid function to output class probabilities; the quaternion regression branch predicts quaternions. The bounding box regression branch predicts the offset from the grid center to the 2D bounding box; the translation regression branch first predicts the translation center. In the two-dimensional image coordinate system The projection in, and then through Map the projection onto the 3D camera coordinate system All three regression branches employ a discrete-to-continuous D2C process, through... Regression of continuous variables is achieved by summing the probabilities of discrete points.

[0011] Preferably, in step three, the texture library storage Each model Each texture is obtained or updated through random indexing of the corresponding model texture; the texture library is initialized based on the natural material color of the distribution network equipment, and multiple materials of the equipment are uniformly and randomly initialized with multiple colors to avoid the network overfitting to a single texture.

[0012] Preferably, in the cyclic training process of step four, each cycle includes the following steps: 41) No. In each loop, a random number is generated for each unlabeled distribution network image. 3D pose label Through the renderer Synthetic image , This is a set of rigid body mesh models with triangular faces. For texture library; Random masks are used during image synthesis. The foreground is randomly discarded to simulate the situation where the device is occluded; random color perturbation is used to enhance the diversity of the synthesized samples, and level detail technology is used to accelerate the overall rendering process; 42) Merge the composite image with the original image ,Right now ,in Indicates will Paste to Above, attach the label from step 41). Compared to the previous round of pseudo-labels ,Right now , This means concatenating and merging the two types of labels; merging the image and labels to form the training set. ; 43) Based on the training set and 3D enhancement methods Training a 3D detector : ; in It is a loss function. This is a 3D enhancement method. In the 3D enhancement method, the label 3D transformation matrix corresponding to the 2D transformation of the image is constructed through intrinsic parameter matrix and matrix orthogonalization method to ensure the consistency between the label and the image content before and after augmentation. 44) Filter the prediction results of the 3D detector and retain the image with the highest confidence level. Each prediction and the top of each category One prediction, generating pseudo-labels. ,according to Sample new texture And update the texture library Repeat the training loop until... Convergence or reaching the maximum number of iterations .

[0013] Compared with the prior art, the present invention has the following significant advantages: (1) Complete elimination of labeling dependence: This invention generates synthetic samples with precise labels through 3D model rendering and combines a pseudo-label iterative update mechanism, eliminating the need for manual labeling of distribution network inspection images, and perfectly solving the core pain points of high labeling cost, insufficient quantity and insufficient granularity in distribution network scenarios; (2) Comprehensive upgrade of generalization ability: Based on the prior of the consistency of the three-dimensional shape of power equipment, this invention does not require redesigning the method or network for specific equipment categories. It can directly adapt to the detection of various equipment in the distribution network through the model and can be generalized to inspection scenarios of different voltage levels and different regions. Its adaptability is far superior to traditional methods. (3) Significantly improved detection accuracy: This invention integrates key technologies such as 3D enhancement to enhance the model's adaptability to changes in device scale, pose differences, and occlusion scenes. The detection accuracy of 3D pose and 2D bounding box is significantly better than that of general unsupervised detection methods. (4) Wide range of practical value: This invention can be used as an unsupervised pre-training method to build a power equipment-specific feature extraction capability for the detection network. Its output three-dimensional labels can also support downstream tasks such as depth estimation, spatial perception, and defect diagnosis, and are fully adapted to actual industrial application scenarios such as UAV inspection. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the two-dimensional image coordinate system and the three-dimensional camera coordinate system of the present invention; Figure 2 It is the architecture of a 3D detector; Figure 3 It is an unsupervised 3D detection training framework; Figure 4 It is the 3D model rendering process; Figure 5 These are models with different levels of detail; Figure 6 It is an enhancement method for 3D labels; Figure 7 These are example images from the power distribution network inspection dataset; Figure 8 It is a three-dimensional model of power distribution network equipment; Figure 9 It refers to the changes in detector performance during the training process; Figure 10 These are the prediction results for two-dimensional and three-dimensional labels on unlabeled images. Detailed Implementation

[0015] The present invention will be further described and illustrated below with reference to specific embodiments. The embodiments described are merely examples of the content of this disclosure and do not limit the scope of the invention. The technical features of each embodiment in the present invention can be combined accordingly, provided that there is no mutual conflict.

[0016] This invention relates to an unsupervised detection method for power distribution equipment based on 3D model rendering, which mainly includes four steps: coordinate system and model construction, 3D detection network construction, texture library construction, and unsupervised training. The following is a detailed description of each step.

[0017] Step 1: Constructing the coordinate system and 3D model library for image cameras of power distribution network equipment (1) Given dimensions are Images of distribution network equipment The origin of this invention is the upper left corner of the image. The right direction is the horizontal axis. The downward direction is the vertical axis. Establish a two-dimensional image coordinate system ,like Figure 1 As shown.

[0018] (2) To further represent the three-dimensional objects projected on the image, this invention establishes another three-dimensional camera coordinate system. The horizontal and vertical axes are aligned with the aforementioned two-dimensional image coordinate system, i.e. , vertical axis Pointing inside the image plane. Camera origin. The vertical projection onto the image is the image center. And the projection distance is the focal length. The present invention is fixed. , and To obtain a constant intrinsic parameter matrix .

[0019] For the 3D camera coordinate system Each point in Its coordinate system in two-dimensional image Projection in It is possible The calculation yielded: in It is a projection function. It is to pass the input coordinates through its A function that normalizes the components.

[0020] (3) The present invention uses a rigid body mesh composed of triangular faces as a model to represent arbitrary objects. The three-dimensional shape is formally defined as ,in express One vertex, express Vertex indices of a triangle face, texture This represents the color of each vertex.

[0021] In the 3D camera coordinate system Rigid body motion in [the context] is equivalent to [the motion of] the set of vertices. The transformations corresponding to rigid body motion are determined by orthogonal rotation matrices. Translation vector Confirmed. Regarding... Each point It can be transformed into the following ways : in It is the pose transformation function. It describes an invariant rigid body model viewed from any angle or distance. And obtain its image projection , equivalent to In the 3D camera coordinate system After performing arbitrary rigid body transformations, the image is projected onto a two-dimensional image coordinate system. In, the transformation corresponding It can uniquely determine its location Projection in : Conversely, if it is known and It can also be deduced from the reverse. This allows us to obtain the object's pose in the camera coordinate system.

[0022] when Images rendered onto power distribution network equipment At that time, the corresponding two-dimensional vertex set in the image The pixels will be filled with the corresponding color. Similarly, it can also be derived from the given... , and sampling function China Resumption : Considering orthogonal matrices Equivalent to quaternions ( For real part and , (The imaginary part), this invention uses Three-dimensional pose labels for objects are represented using two-dimensional bounding box labels. Represents the foreground region of objects in an image of power distribution network equipment. These correspond to the left, top, right, and bottom coordinates of the border, respectively.

[0023] (4) Since distribution network equipment usually has a fixed three-dimensional shape and a fixed size and model under a specific voltage level, the equipment size published by the manufacturer can be collected, and the equipment model can be constructed using three-dimensional modeling software to establish a model library containing a variety of distribution network equipment.

[0024] Step 2: Construct a 3D detection network This invention constructs a three-dimensional detector Used for predicting 3D pose and two-dimensional border This invention relates only to the three-dimensional detection layer of the detector, whose backbone network can be arbitrarily selected, such as... Figure 2 As shown. Given an input image The detector backbone network outputs multi-scale feature maps. For a stride of... The channel is Feature map The 3D detection layer will convert the grid vectors of the feature map Convert to predicted labels ,in These represent the predicted classification probability, 3D translation, 3D rotation quaternion, and 2D bounding box, respectively. The 3D detection layer has four decoupled branches, namely the classification branch. Translation regression branch Quaternion Regression Branch and bounding box regression branch .

[0025] (1) Branch output The probability of each category (model) .

[0026] (2) Branch prediction quaternion To facilitate detector convergence, this invention regresses continuous variables in discrete form. Branches (including subsequent branches) and Each predicted value in the branch is obtained through The probability and form of each discrete point are obtained, which is abbreviated as the discrete-to-continuous (D2C) process, such as... Figure 2 As shown.

[0027] in For the pure neural network part, Convert to . Then Transform into .remember for In the index The value below in It is evenly distributed in The range of values Within range A discrete value It is along The dimension is used to make Normalize the probability distribution of each discrete value, and finally sum them to obtain the value to be regressed. .

[0028] (3) Branch prediction 2D bounding box The specific method is to predict the grid center. Offset to bounding box: (4) The branch first predicts the translation center. In the image coordinate system The projection in, and then through Map the projection to This output design and Consistent branching patterns facilitate detector learning, among which This is the predicted projection point offset. Projection distance: Step 3: Build and initialize the texture library (1) This invention constructs a texture library Used to store sampled textures for each model. Because vertex colors are used. Representation Model Textures, texture library It can be simply derived from a set Implementation, which includes Each model The first texture. For the first... When acquiring textures for a model, a random index is selected. And return the corresponding Similarly, when the first When a model needs to update its texture, the new texture will overwrite the existing texture at another random index.

[0029] (2) In this invention, the texture library Initialize directly based on the natural colors of the corresponding devices for each model. For example, use gray for hardware devices, cyan for the insulator discs of glass insulators, and gray for the metal caps. If a device has multiple materials and colors, such as white ceramic and brown ceramic, then initialize the corresponding texture library with multiple colors uniformly and randomly.

[0030] Step 4: Execute the unsupervised detection training framework The unsupervised 3D detection training framework proposed in this invention is as follows: Figure 3 As shown, it mainly includes a renderer. 3D detector and texture library Detector and texture library As already explained, the renderer Perspective rendering is achieved based on physical rules, requiring no training and containing no learnable parameters. The proposed framework operates in a loop, with the loop index denoted as . Each cycle consists of the following steps: (1) in the At the start of each loop, the first step is to process each unlabeled image of the distribution network equipment. Randomly generated containing Random device 3D pose labels , The representative will A random type of device is placed at any position in the 3D camera coordinate system.

[0031] (2) Then Input to renderer According to the model set Triangular rigid body mesh in The texture rendering in the process yields the composite image. This invention uses parallel light sources and the Phong lighting model for rendering. To obtain richer rendering results and increase the network's generalization ability, this invention enhances the rendered foreground from both color and spatial perspectives, as shown in Figure 4. Regarding color, the acquired texture... Through random deviation of the same size Interference is applied to simulate different ambient lighting and device textures. Spatially, the foreground is rendered via a mask. Random dropping is used because real electrical equipment is always obstructed by other equipment, towers, or power lines. To ensure... The continuity and randomness, From a smaller random matrix The result is obtained by interpolation and binarization.

[0032] in The size is random feature maps, This is the scaling factor. It is a bilinear interpolation function. For threshold-based The binarized function. In the above process, It can control the continuity of random occlusion and simulate occlusion of objects of different scales. This allows for control over the occlusion ratio. The present invention addresses... and There are no mandatory requirements for the choice of , and an appropriate value can generally be selected based on the actual data distribution to enhance the network's generalization ability.

[0033] To fully represent device details, the number of model vertices and number of triangular faces Typically, rendering on multiple devices simultaneously in the thousands of pixels can lead to a significant burden on video memory and rendering latency. To address this, this invention utilizes Level of Detail (LOD) technology to accelerate the overall rendering process. The core idea of ​​LOD is that when rendering the same model at a fixed image resolution, the greater the distance, the weaker the representation of model details. Therefore, models with different levels of detail can be used based on different rendering distances, avoiding performance waste.

[0034] Given a set of triangular rigid body mesh models This invention first performs model collapse processing to generate multiple simplified models, each model having half the number of vertices of the previous model, such as... Figure 5 As shown, the model collapse algorithm continuously merges the closest vertex pairs with the least local information in the model. Therefore, while reducing the number of vertices, it only slightly affects the overall representation of the model. Figure 5 In models (a)-(d), the number of vertices and faces gradually decreases from (a) to (d), and the resources required for rendering also gradually decrease. Given 3D annotations... Subsequently, the present invention is based on By selecting the most appropriate model and doubling the distance between the model and the camera origin, the number of vertices in the corresponding model can be reduced by half. Experiments show that this method can double the rendering speed.

[0035] (3) After obtaining the rendered image, the original image Previous round of pseudo-labels ( )and ) merged into training set Specifically: in Indicates will Paste to superior. This indicates that two labels will be combined and merged.

[0036] Based on training set and 3D enhancement methods Training detector : in It is a loss function. It is a function value. The specific form of the loss function corresponds to four prediction branches: the classification branch uses cross-entropy for calculation, while the 3D translation, quaternion, and 2D bounding box regression branches use distributed focus loss for calculation.

[0037] Due to the similarity between the rendered objects and real objects, the first round A preliminary system can be established using only rendering tags. The 3D detection capability, in subsequent rounds The detection capability is enhanced based on pseudo-labels and rendered labels.

[0038] This invention is a 3D augmentation method for constructing 3D annotation geometric transformations, because 3D pose labels... Defined in camera coordinate system rather than image coordinate system Therefore, traditional two-dimensional spatial augmentation methods cannot be used for [the following]. The transformation is performed. Two-dimensional spatial enhancement can be uniformly represented as a homography transformation matrix. This matrix will take the input image Convert to enhanced image This invention constructs a suitable three-dimensional transformation. This transformation can Convert to And maintain consistency between the 3D labels and image content before and after the transformation.

[0039] Given image transformation corresponding Assuming a two-dimensional point It is a three-dimensional point The projection, pass Convert to Then we have: Combining the above equation with the projection formula, we get: The above formula shows that, exist The projection on corresponds to exist The projection onto the matrix. It is a two-dimensional transformation matrix The corresponding three-dimensional form. However, if directly through Transformation This may disrupt the object's inherent shape. Because Not always orthogonal, especially when When including translation, shearing, and perspective components. For example... Figure 6 As shown, the image illustrates other 3D perspectives to represent the relative positions of objects (red) and cameras (yellow) in space. The 3D annotations show images captured by that camera. The model is achieved through... The transformation will result in significant distortion and shrinkage. Therefore, this invention firstly... Normalize the determinant and then apply the orthogonalization method. Obtain an approximate orthogonal matrix : use right Perform the transformation: in This represents the operations between matrices and quaternions.

[0040] Under the constraint of invariant model shape and size, In essence, translation or rotation transformations on an image are equivalent to translation or rotation transformations of the model in 3D camera space. The difference is that scaling transformations on an image correspond to scaling transformations of the model along... Translation of the axis, i.e., changing the distance from the camera, reflects the perspective rule of near objects appearing larger and distant objects appearing smaller in reality, which is beneficial for the detector to learn scale invariance.

[0041] (4) Detector After training, predictions are made on unlabeled images, and a filtering algorithm is used. Process the prediction results to obtain the current round pseudo-label. : For a given network prediction result, the filtering algorithm retains only the image with the highest confidence score for each image. The number of objects. To further balance the predicted categories, this invention also sets a quantity threshold for each category, namely, the number of objects. Before each category is retained One prediction result. Based on the number of images The first of each image One device and constant ratio Confirmed. Prior. It can be obtained through empirical estimation or statistical analysis of small batches of data.

[0042] Finally, according to Sample new texture And update This is to ensure that the rendered samples in the next loop are more realistic. It is expected that with... The increase, , and They will be updated sequentially and mutually reinforcing. When No longer changing or reaching the maximum number of cycles At that point, the entire framework terminates.

[0043] Example verification and effect analysis 1 Distribution Network Dataset and Model Construction To verify the effectiveness of the proposed method in practical applications, this invention collected 160,761 images of distribution network equipment with a resolution of 640 dpi, divided into training, validation, and test sets. The training set was unlabeled, while the validation and test sets were manually labeled with two-dimensional bounding boxes (horizontal bounding boxes) for eight types of power equipment, such as... Figure 7 As shown, different devices are labeled with different colors. Since the images are acquired from different cameras and may have been cropped or scaled, accurate 3D annotations cannot be obtained. Therefore, this invention uses 2D bounding box detection metrics to characterize detector performance.

[0044] The detector is trained on an unlabeled training set and a validation set with labels removed. Then, its bounding box average precision (AP) is evaluated on the validation and test sets, respectively, to characterize its performance on seen and unseen images.

[0045] Table 1 shows the quantity distribution, mean area, and standard deviation of each category in the distribution network inspection dataset. Overall, their quantity distribution is consistent with the actual distribution network, with significant differences in quantity among different equipment types. The area distribution of equipment is mainly affected by the inherent size of the equipment and the shooting distance; therefore, there are also large intra-class differences in equipment area.

[0046] Table 1. Statistics on the annotation of data sets for power distribution network inspection For a certain type of equipment, there can be significant differences in appearance due to different specifications. Therefore, this invention constructs 31 models to cover the common styles of 8 types of equipment, as shown in Figure 8. The 8 types of equipment include: (a) ceramic crossarm insulator, (b) ceramic post insulator, (c) ceramic suspension insulator, (d) composite post insulator, (e) composite crossarm insulator, (f) composite tie rod insulator, (g) vibration damper, and (h) wire clamp. Figure 8 (i) in the figure illustrates various initial uniform textures for ceramic suspension insulators. The models in the figure are all built based on dimensions published by power equipment manufacturers, ensuring consistency with the actual equipment shape. Even with discrepancies, due to physical constraints, equipment at specific voltage levels will have similar shapes and fixed size ranges, and approximate sizes are sufficient to provide a 3D prior. All these models are learned as independent categories during training and remapped to their respective equipment categories during evaluation.

[0047] 2. Evaluation Indicators Average precision (AP) is a key metric for measuring the overall performance of a detector across all confidence thresholds.

[0048] After matching a certain number of bounding boxes and predicted boxes using a specific algorithm, the following cases can be identified: (1) True Positive (TP): Samples that are actually positive are correctly predicted as positive, and the labeled objects are detected and the labeled boxes match the predicted boxes.

[0049] (2) True Negative (TN): Samples that are actually negative are correctly predicted as negative, and the labeled background area has no prediction box.

[0050] (3) False Positive (FP): Samples that are actually negative are incorrectly predicted as positive, and a prediction box appears in the labeled background area.

[0051] (4) False Negative (FN): Samples that are actually positive are incorrectly predicted as negative. Therefore, precision and recall are defined as follows: When the detector's confidence threshold changes, some low-confidence predicted boxes are discarded, thus affecting the overall precision and recall. A precision-recall curve can be plotted by iterating through all thresholds, using precision and recall as coordinates. The area under this curve represents the detector's overall performance, i.e., the average precision (AP). Since calculating the area under the curve using integral methods is too time-consuming, current research typically uses fixed sampling points to estimate the area.

[0052] In the above process, the matching of the labeled bounding boxes and the predicted bounding boxes is determined by the Intersection over Union (IoU) threshold. IoU is defined as the area of ​​intersection of two bounding boxes divided by their area of ​​union; only bounding box pairs with an IoU greater than the threshold are considered a match. Different IoU thresholds result in different AP calculations; commonly used thresholds are 0.5 and 0.75, corresponding to... and Deep learning typically uses This refers to the average AP value as the IoU threshold changes from 0.5 to 0.95 in steps of 0.05, hereinafter referred to as AP. Additionally, it is also commonly used... (Target ≥ 962) (322≤Target<962) and (Small target < 322) represents APs of different sizes.

[0053] 3 Training Details This invention was implemented using PyTorch 2.1 and CUDA 12.1 on four NVIDIA RTX 4090 GPUs. The proposed unsupervised 3D detection framework has a total of There are 3 iterations, each containing 3 training rounds. The SGD optimizer is used, with weight decay at a certain value. The batch size is 48. During training, the learning rate decays from 0.01 to 0 using a cosine function.

[0054] This invention selects suitable hyperparameters for an unsupervised 3D inspection framework through statistical manual annotation. Based on the UAV inspection distance, 3D model size, and annotation horizontal border size, the focal length is... Set to 5000. Considering that each image has approximately 2.7 horizontal borders, the generated object ratio... Set to 6. Similarly, the... Class restrictions Set to 0.9, limit per image Set it to 20.

[0055] 4. Unsupervised detection performance on the distribution network dataset Without loss of generality, this invention selects an FPN with a ResNet-50 backbone as the baseline detector and replaces its head for unsupervised detection training. Metrics measured on the test set in each training epoch are as follows: and metrics measured on the validation set like Figure 9 As shown. The best detection metrics for all device categories on the test set ( Figure 7 The 20th training round is shown in Table 2.

[0056] Table 2. Best performance of unsupervised training based on FPN-50 on the test set. Figure 9 shows that the detector's performance trends on the test and validation sets are similar, indicating that the detector can automatically generalize based on unlabeled images. Because a certain proportion of rendering devices and corresponding annotations ensure the correct optimization direction, a large number of real devices can be detected as false labels in the early stages of training, resulting in rapid AP growth. However, as the detection of salient device targets saturates and false detections accumulate, the detector's performance remains unchanged or slightly decreases in the last few training epochs.

[0057] Table 2 shows that the detector performs significantly worse on clamps or vibration dampers than on insulators. This is partly due to the relatively smaller number of vibration dampers and the inherently smaller size of clamps, as shown in Table 1. Another reason is that metal fittings are more susceptible to background interference from the same metal material of the tower and line. For the same reason, the detector fits ceramic crossarm insulators best, due to their larger size and distinctive appearance.

[0058] Figure 10 The 2D and 3D prediction results include some example images. The yellow lines represent the camera's field of view, and there is another perspective with 3D labels to better show the relative position of the devices. Figure 10 This demonstrates that the network can estimate the 3D position and orientation of devices while performing routine 2D detection tasks, indicating its potential applications in depth estimation, spatial perception, and visual navigation. The detector performs well in predicting the position and orientation of large-scale devices, but may overlook or misjudge the position and orientation of small devices. For example, the predicted depth of the composite post insulator in Figure 10(a) is incorrect, thus failing to align with other ceramic post insulators. This is because the features of small targets are easily confused with complex backgrounds, hindering detector learning.

[0059] (4) Overall effect evaluation: Verification using actual distribution network inspection datasets demonstrates that the unsupervised inspection method for distribution network equipment based on a 3D model can achieve 3D equipment inspection without manual annotation, validating the effectiveness and practicality of the technical solution. This method can be widely applied to various power inspection scenarios, including UAV inspections of distribution networks, identification of multiple equipment types, cross-voltage level detection, and pre-diagnosis of defects, providing a complete, efficient, and reliable technical solution for the intelligent and low-cost implementation of power inspection projects.

[0060] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for unsupervised detection of distribution network equipment based on a 3D model, characterized in that, Includes the following steps: Step 1: Establish a two-dimensional image coordinate system and a three-dimensional camera coordinate system and determine the intrinsic parameter matrix. Based on the public dimensions of the power distribution equipment, construct a set of triangular rigid body mesh models and define three-dimensional pose labels and two-dimensional bounding box labels. Step 2: Construct a 3D detection network for predicting 3D pose and 2D bounding boxes; Step 3: Build and initialize the texture library. The texture library stores multiple textures of the triangular rigid body mesh model. It is obtained or updated through random indexing. Initialization is completed based on the natural material color of the device. For multi-material devices, multiple colors are used for uniform random initialization. Step 4: Based on the 3D detection network from Step 2, the texture library from Step 3, and the renderer, construct an unsupervised 3D detection training network and perform unsupervised iterative training. During the cyclic training process, 3D labels are randomly generated for each unlabeled image and the synthesized image is rendered. A training set is constructed based on the synthesized image, the previous round of pseudo-labels, and the synthesized labels. After 3D enhancement, the 3D detection training network is trained. The prediction results are filtered to generate the current round of pseudo-labels and the texture library is updated. The cycle is iterated until convergence. In the iterative training process described in step four, each iteration includes the following steps: 41) No. In each loop, a random number is generated for each unlabeled distribution network image. 3D pose label Through the renderer Synthetic image , This is a set of rigid body mesh models with triangular faces. For texture library; Random masks are used during image synthesis. The foreground elements are randomly discarded to simulate situations where the device is occluded. Random color perturbation is used to enhance the diversity of the synthesized samples, and level detail technology is used to accelerate the overall rendering process; 42) Merge the composite image with the original image ,Right now ,in Indicates will Paste to Above, attach the label from step 41). Compared to the previous round of pseudo-labels ,Right now , This means concatenating and merging the two types of labels; merging the image and labels to form the training set. ; 43) Based on the training set and 3D enhancement methods Training a 3D detector : ； in It is a loss function. This is a 3D enhancement method. In the 3D enhancement method, the label 3D transformation matrix corresponding to the 2D transformation of the image is constructed through intrinsic parameter matrix and matrix orthogonalization method to ensure the consistency between the label and the image content before and after augmentation. 44) Filter the prediction results of the 3D detector and retain the image with the highest confidence level. Each prediction and the top of each category One prediction, generating pseudo-labels. ,according to Sample new texture And update the texture library Repeat the training loop until... Convergence or reaching the maximum number of iterations ; Finally, the trained unsupervised 3D detection network is used to detect images of distribution network equipment.

2. The unsupervised detection method for distribution network equipment based on a three-dimensional model as described in claim 1, characterized in that, Step one, establishing the two-dimensional image coordinate system and the three-dimensional camera coordinate system and determining the intrinsic parameter matrix, includes: Given dimensions Images of distribution network equipment The origin is the top left corner of the image. The right direction is the horizontal axis. The downward direction is the vertical axis. Establish a two-dimensional image coordinate system Establish a 3D camera coordinate system The horizontal and vertical axes are aligned with the coordinate system of the two-dimensional image, i.e. , vertical axis Pointing inside the image plane; camera origin The vertical projection onto the image is the image center. And the projection distance is the focal length. Fixed image size , and focal length The intrinsic parameter matrix is ​​obtained as follows: 。 3. The unsupervised detection method for distribution network equipment based on a three-dimensional model as described in claim 1, characterized in that, Step one, which involves constructing a set of triangular rigid body mesh models based on the publicly available dimensions of the distribution network equipment, includes: constructing a set of triangular rigid body mesh models based on the publicly available dimensions of the distribution network equipment from the power equipment manufacturer. Each model ,in For vertex set, For triangle face indexing, For vertex textures.

4. The unsupervised detection method for distribution network equipment based on a three-dimensional model as described in claim 3, characterized in that, The definition of the 3D pose label and the 2D bounding box label in step one includes: Triangular Rigid Body Mesh Model Set In the 3D camera coordinate system Rigid body motion in [the context] is equivalent to [the motion of] the set of vertices. The transformations corresponding to rigid body motion are determined by orthogonal rotation matrices. Translation vector Sure; for Each point It can be transformed into the following ways : ； Define the 3D pose label of an object ,in, It is a quaternion, where For the real part, It is the imaginary part; Let the translation vector be defined; define the 2D bounding box label. Used to characterize the foreground region of objects in an image of power distribution network equipment. These correspond to the left, top, right, and bottom coordinates of the border, respectively.

5. The unsupervised detection method for distribution network equipment based on a three-dimensional model as described in claim 4, characterized in that, The 3D detection network in step two comprises a backbone network and a 3D detection layer. The backbone network outputs multi-scale feature maps, and the 3D detection layer has four decoupled branches: a classification branch... Translation regression branch Quaternion Regression Branch Boundary regression branch ; The classification branch activates the sigmoid function to output class probabilities; the quaternion regression branch predicts quaternions. The bounding box regression branch predicts the offset from the grid center to the 2D bounding box; the translation regression branch first predicts the translation center. In the two-dimensional image coordinate system The projection in, and then through Map the projection onto the 3D camera coordinate system ; All three regression branches employ a discrete-to-continuous D2C process, through... Regression of continuous variables is achieved by summing the probabilities of discrete points.

6. The unsupervised detection method for distribution network equipment based on a three-dimensional model as described in claim 1, characterized in that, In step three, the texture library storage Each model Each texture is obtained or updated through random indexing of the corresponding model texture; the texture library is initialized based on the natural material color of the distribution network equipment, and multiple materials of the equipment are uniformly and randomly initialized with multiple colors to avoid the network overfitting to a single texture.

Images