Domain offset mitigation method and system based on electrical equipment image

Abstract

The invention discloses a domain offset mitigation method and system based on a power equipment image, and relates to the technical field of cross-domain detection, and the method comprises the steps: constructing a generalization model based on insulator defect detection; based on an IRM risk extrapolation method, regularization items are added to extract invariant features, so that the aligned sample loss of each domain is the same; by measuring the Zensen-Shannon divergence of each domain feature of the selected layer in the generalization model, the distribution difference in the feature space is minimized, so that the similarity of the feature space is the same, and the domain offset between the source domain and the target domain is reduced. According to the method, the generalization of real data is enhanced by reducing the difference between the synthetic domain and the real domain to the greatest extent, so that the problem of domain migration is solved.

Description

Technical Field

[0001] This invention relates to the field of cross-domain detection technology, and more specifically to a domain offset mitigation method and system based on power equipment images. Background Technology

[0002] Currently, the ever-increasing demand for electricity has increased the capacity of transmission lines, thereby raising the requirements for transmission reliability. Power inspection, crucial for maintaining grid stability, has become more complex and labor-intensive. Insulators, critical components of transmission lines, are exposed to harsh outdoor conditions for extended periods, increasing the risk of defects and failures. Timely inspection is essential to ensuring the safe operation of insulators, reducing potential safety hazards, and improving the reliability of the power system. In recent years, with unmanned aerial vehicles (UAVs) gradually replacing manual inspection, computer vision technology has been widely applied to insulator monitoring. However, domain bias between datasets significantly impacts insulator monitoring, affecting model performance and generalization ability.

[0003] In power grid defect detection, domain bias is often overlooked, as previous methods assumed that training and testing data came from the same domain. In reality, current detection methods face the challenge of domain bias due to insufficient data, failing to cover most detection scenarios.

[0004] Therefore, how to provide a domain offset mitigation method and system based on power equipment images to accurately and timely detect insulator defects and improve the safety and stability of power systems is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a domain offset mitigation method and system based on power equipment images, which enhances the generalization of real data by minimizing the difference between the synthetic domain and the real domain, thereby solving the domain offset problem.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a domain offset mitigation method based on power equipment images, comprising:

[0007] Construct a generalized model based on insulator defect detection;

[0008] Based on the IRM risk extrapolation method, a regularization term is added to extract invariant features, so that the loss of aligned samples in each domain is the same.

[0009] By measuring the Jensen-Shannon divergence of each domain feature in a selected layer of the generalization model, the distributional differences in the feature space are minimized, making the feature spaces similar and reducing the domain offset between the source and target domains.

[0010] Preferably, before adding a regularization term to extract invariant features based on the IRM risk extrapolation method, the method further includes: synthesizing insulator defect data in three-dimensional space to obtain a three-dimensional synthetic dataset;

[0011] Style transfer is performed on the three-dimensional synthetic dataset to extend a synthetic defect image from a single-source domain to a multi-source domain.

[0012] Preferably, the IRM risk extrapolation method is expressed as follows:

[0013]

[0014] in, Let K represent institutional risk management data in domain K, e be the domain (environment) index, θ be the learning model that minimizes the equation, λ be the hyperparameter that needs to be adjusted, and Var(R) be the hyperparameter that needs to be adjusted. e (ω,Φ)) is the risk variance for each domain.

[0015] Preferably, the penalty loss during training for the IRM risk extrapolation method is set as follows:

[0016] L IRM =λ IRM Var(L 0 (θ),L 1 (θ),...,L K (θ));

[0017] Where L(·) represents the loss function in the generalization model.

[0018] Preferably, minimizing the distributional differences in the feature space by measuring the Jensen-Shannon divergence of features in selected layers of the generalization model includes:

[0019] Given the feature outputs of the selected n layers, the corresponding feature map for each source domain in each layer is obtained;

[0020] Convert the feature divergence into probabilities that sum to 1;

[0021] Calculate the average feature map of all source domains, denoted as Q;

[0022] Calculate the feature map P for each source domain i The Jansen-Shannon divergence between Q and Q.

[0023] Preferably, the feature map P of each source domain is calculated. i The Jensen-Shannon divergence between the average feature map Q across multiple domains is as follows:

[0024]

[0025]

[0026] Where KL(·) is the KL divergence, JS(·) is the Jensen-Shannon divergence, N is the length of the eigenvector, and p m Let q be the m-th value in feature P. m Let m be the m-th value in feature Q.

[0027] Preferably, the loss component l of each selected layer i Defined as the variance of the source domain output divergence, the distribution loss of each domain in each layer is multiplied by a hyperparameter λ. FDM This is combined to form another regularization component, resulting in the distribution difference loss;

[0028] The final loss function of the model is obtained based on the distribution difference loss.

[0029] Preferably, a domain offset mitigation system based on power equipment images includes:

[0030] The model building module is used to build a generalized model based on insulator defect detection;

[0031] The invariant feature extraction module is used to extract invariant features by adding a regularization term based on the IRM risk extrapolation method, so that the loss of aligned samples in each domain is the same.

[0032] The minimization module is used to minimize the distributional differences in the feature space by measuring the Jensen-Shannon divergence of each domain feature in a selected layer of the generalization model, thereby making the feature spaces similar and reducing the domain offset between the source and target domains.

[0033] As can be seen from the above technical solution, compared with the prior art, this invention discloses a domain offset mitigation method based on power equipment images, including: constructing a generalization model based on insulator defect detection; extracting invariant features by adding a regularization term based on the IRM risk extrapolation method, so that the aligned sample loss of each domain is the same; minimizing the distribution difference in the feature space by measuring the Jensen-Shannon divergence of each domain feature in the selected layer of the generalization model, so that the similarity of the feature space is the same, and reducing the domain offset between the source domain and the target domain. By minimizing the difference between the synthetic domain and the real domain, the generalization to real data is enhanced, thereby solving the domain offset problem. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0035] Figure 1This is a schematic diagram of a domain offset mitigation method based on power equipment images provided in an embodiment of the present invention.

[0036] Figure 2 This is a schematic diagram illustrating the data distribution changes between the source and target domains, provided in an embodiment of the present invention.

[0037] Figure 3 mAP provided in the embodiments of the present invention 50 Visualization diagram of indicator results. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] This invention discloses a domain offset mitigation method based on power equipment images, such as... Figure 1 As shown, it includes:

[0040] Construct a generalized model based on insulator defect detection (breakage and spontaneous explosion);

[0041] Based on the IRM risk extrapolation method, a regularization term is added to extract invariant features, so that the loss of aligned samples in each domain is the same.

[0042] By measuring the Jensen-Shannon divergence of each domain feature in a selected layer of the generalization model, the distributional differences in the feature space are minimized, making the feature spaces similar and reducing the domain offset between the source and target domains.

[0043] Domain offset refers to the change in data distribution between the training dataset (source domain) and the test data (target domain). In insulator defect detection, four main factors contribute to this change: camera viewpoint (distance and orientation), environmental conditions, insulator material, and defect type. Figure 2 As shown, the model exhibits acceptable performance when the training and test data come from the same domain. However, performance drops significantly when the distribution of the test data differs.

[0044] Specifically, before adding regularization terms to extract invariant features based on the IRM risk extrapolation method, the process also includes: synthesizing insulator defect data in three-dimensional space to obtain a high-fidelity three-dimensional synthetic dataset;

[0045] Style transfer is performed on the 3D synthetic dataset to extend a synthetic defect image from a single-source domain to a multi-source domain. The 3D synthetic dataset includes synthetic defect images.

[0046] This invention introduces domain randomization (DR) into insulator defect detection in the power industry for the first time, utilizing the 3D rendering software Blender to generate synthetic data in 3D space. Furthermore, the two-stage method proposed in this invention improves the performance of the generated synthetic data in real-world defect detection. In this work, this invention proposes a general framework for insulator defect detection that can synthesize high-fidelity synthetic data and mitigate the domain offset between the synthetic domain (source domain) and the real-world domain (target domain). This invention introduces a domain randomization (DR)-based data synthesis method (referred to as DR-SYN) to create synthetic datasets. Subsequently, this invention introduces an instance-weighted image quality assessment (IR-IQA) domain expansion method, which can increase domain variability and be used for invariant feature extraction. During training, this invention employs a difference-constrained invariant learning (DCIL) model, which simultaneously considers feature loss and output loss. Similarity of invariants extracted from multiple expanded domains provides features robust to domain variations.

[0047] This invention applies a weighted content loss to foreground insulator objects, particularly in defect regions, to maintain the quality of stylized defect images. The DCIL model introduces a Feature Difference Minimization (FDM) method and employs Invariant Risk Minimization (IRM) to extract robust invariant features across the extended domain. Jensen-Shannon divergence is used to measure feature divergence and is incorporated as a loss term. In the output space, a regularization term based on the variance of the loss across different domains is added to the total loss.

[0048] This invention proposes a DR-Syn method for automatically generating synchronous defect insulators. The IR-IQA domain extension method and the DCIL model reduce the domain offset between the synthesized domain and the target domain by extracting invariant features.

[0049] The generalization framework of this invention utilizes domain randomization to generate high-fidelity 3D synthetic data and enhances generalization to real-world data by minimizing the difference between the synthetic and real domains, thereby solving the domain offset problem. Notably, no real-world defective images are used during generation and training.

[0050] Traditional machine learning models employ Empirical Risk Minimization (ERM) techniques, which minimize the loss function L for each sample in the training dataset. e The method learns the model by summing the losses; it uses data from the domain D to calculate empirical risk and minimizes training risk; if the training and test data have the same distribution and no domain bias, the method can achieve good performance. However, this assumption does not hold because there is a domain bias between the source and target domain data.

[0051] To address this issue, this invention introduces an Invariant Risk Minimization (IRM) method and improves it by adding a Feature Difference Minimization (FDM) method to learn invariant representations with better generalization ability. Unlike Empirical Risk Minimization (ERM), IRM extracts invariant features by adding a regularization term, thus considering domain shifts and causal relationships. In this invention, IRM Risk Extrapolation (IRM REx) is used; if the learned representations have the same style but the same defects, then the model's predictions for different domains should be consistent, leading to the same loss. This ensures that the learned representations are unaffected by domain changes, thereby minimizing risk differences in different environments.

[0052] Specifically, the IRM risk extrapolation method is expressed as follows:

[0053]

[0054] in, Let K represent institutional risk management data in domain K, e be the domain (environment) index, θ be the learning model that minimizes the equation, λ be the hyperparameter that needs to be adjusted, and Var(R) be the hyperparameter that needs to be adjusted. e (ω,Φ)) is the risk variance for each domain.

[0055] Specifically, the penalty loss in training for the IRM risk extrapolation method is set as follows:

[0056] L IRM =λ IRM Var(L 0 (θ),L 1 (θ),...,L K (θ));

[0057] Where L(·) represents the loss function in the generalization model.

[0058] Feature Dissimilarity Minimization (FDM), besides IRM, aims to minimize distributional differences in the feature space by ensuring that aligned samples in each domain have the same loss and that the feature space has the same similarity. This invention minimizes these distributional differences by measuring the Jensen-Shannon divergence of features in each domain across certain layers of the network model. The best-performing distribution measurement function was selected from L2 regularization, Kullback-Leibler (KL) divergence, and Jenson-Shannon (Js) divergence. This regularization term is stricter than the IRM penalty loss term because identical feature outputs in the final layer will result in the same loss, while identical loss values ​​may not guarantee identical features.

[0059] Specifically, the distributional differences in the feature space are minimized by measuring the Jensen-Shannon divergence of features in selected layers of the generalization model, including:

[0060] Given the feature output of the selected n layers, the corresponding feature map of each source domain in each layer is obtained;

[0061] The softmax function is used to convert the feature divergence into probabilities that sum to 1;

[0062] Calculate the average feature map of all source domains, denoted as Q;

[0063] Calculate the feature map P for each source domain i The Jansen-Shannon divergence between Q and Q.

[0064] Specifically, calculate the feature map P for each source domain. i The Jensen-Shannon divergence between the average feature map Q across multiple domains is as follows:

[0065]

[0066]

[0067] Where KL(·) is the KL divergence, JS(·) is the Jensen-Shannon divergence, N is the length of the eigenvector, and p m Let q be the m-th value in feature P. m Let P be the m-th value in feature Q. P is a feature of a certain source domain, and Q is the average feature map of multiple domains, serving as two input terms for the KL divergence.

[0068] Specifically, each domain outputs a divergence, and these divergences are then used to calculate the variance; the loss components of each selected layer are then... i Defined as the variance of the source domain output divergence, the distribution loss of each domain in each layer is multiplied by a hyperparameter λ. FDM This is combined to form another regularization component, resulting in the distribution difference loss;

[0069] The final loss function of the model is obtained based on the distribution difference loss.

[0070] Distribution difference loss L FDM Represented as:

[0071]

[0072] The final loss of this model will include the default classification loss L in the YOLO object detection model. cls Bounding box coordinate loss L box And confidence loss L obj In addition to the IRM penalty and the difference loss from the invariant representation learning part, the final loss of the model is expressed as:

[0073]

[0074] In a specific embodiment of the present invention, the method described in the embodiment of the present invention is verified by experiments, including: focusing on processing the "damage" and "spontaneous explosion" defects of insulators. For each defect type, 1500 virtual samples with annotations in the generative model are used as data A and B, and 761 insulator damage defect data images from dataset [2], 372 glass insulator spontaneous explosion defect data provided by the power grid company, and 247 synthetic insulator spontaneous explosion defect data from CPLID dataset [3] are collected as data C, D, and E for training and testing. See data samples. Figure 2 The details of each dataset are shown in Table 1. Each dataset is divided into training and test sets in a 7:3 ratio.

[0075] The data generation model used Blender 3.4 and its Python API. Rendering was set to visible light images using the Cycles engine with 512 sampling points, and indexed images using 128 sampling points. Training was performed on a local PC equipped with an Intel Core i7-13700KF, RTX 4080Ti GPU, and CUDA 12.2. The YOLOv5-M pre-trained model provided by Ultralytics was used, trained for 100 durations with default hyperparameters and a batch size of 8. The input image size was resized to 640×640, and the IOU threshold was set to 0.60. YOLO applied data augmentation by default.

[0076] The PSNR thresholds used were 15, 20, and 25, with ω0-ω2 values ​​of 1, 1.5, and 2 for background, "no-problem" CDs, and defective CDs, respectively. The weights α and β in the experiment were 10000 and 5, respectively. The layer used to measure distribution similarity was the YOLO "concat" layer. The hyperparameter λ for each training epoch... IRM and λ FDM Set the exponent value to 100 × 1.05 epoch and 10 × 1.05 epoch .

[0077] Table 1 Dataset used in the experiment

[0078]

[0079] This set of experiments aims to investigate how domain bias between different domains affects the performance of trained models when tested on data from different domains. Acceptable results can be achieved when neither the training nor the test data has domain bias. The synthetic data in this embodiment of the invention generalizes well to the target domain, and when tested in the same domain, its performance is superior to training using datasets B and C.

[0080] This invention employs multiple experiments to evaluate the performance of the method described in this embodiment. Figure 3 It is the mAP of each model 50 A bar chart visualizing the metrics. This embodiment of the invention uses the same network structure in all model training. The DR-Syn model and domain expansion operations are performed only at the data end; DCIL does not involve modifications to the network structure. Figure 3 mAP for each experiment 50 The visualization of the metrics results shows the model ID on the left side of each bar chart and the performance of each model on the right side. Performance improvements are displayed in different colors as the added value increases. The blue and green sections represent performance improvements based on IR-IQA domain extension and DCIL.

[0081] Evaluation of synthetic data domain expansion for indifference constraint invariant learning: Comparing modes 6 and 15, and modes 7 and 18, the result of domain expansion using more domains and instances is that, in the detection of defects in the real-world domain of the target, precision decreased by 1.2%, but recall increased by 0.6%, and mAP... 50 Improved by 0.1%. In terms of self-destruction defect detection, accuracy improved by 0.9%, recall by 0.2%, and mAP50 by 0.2%. Domain extension without DCIR also slightly improved performance.

[0082] Evaluation of Difference Constraint Invariant Learning: By comparing models 15, 16 and 18, 19, it can be concluded that the DCIL method helps improve model performance. The model trained using the method of this embodiment outperforms models 14 and 17, which involve mixing real-world non-domain transfer data, the synthetic domain, and the high-dimensional space of the target real-world domain data, thereby improving performance.

[0083] The synthetic data obtained from the generalization model of this invention was applied to state-of-the-art detection models for insulator defect detection. These models included YOLOv5-M, YOLOv5-L, YOLOv8-M, YOLOv8-L, RetinaNet, and Faster R-CNN. Results showed that the synthetic data from this invention outperformed models trained on real-world data in the same domain. Among all models, YOLOv5-L and YOLOv5-M, trained with synthetic data, performed best. This invention selected YOLOv5-M as the optimal training model.

[0084] This invention simulates a virtual scene to replicate a real-world setting and generates defects using the DR-Syn method. A dataset of 400 virtual images was created for two defect types by randomly adjusting lighting, insulator positions, defect structures, and camera parameters. The real-world dataset was used as the test set, while the virtual dataset was used for training. Compared to datasets C and D, the data A and B generated by DR-Syn showed better generalization performance. The synthetic data based on the digital twin model outperformed other data due to the tight alignment of scene components and minimal domain offset.

[0085] In one specific embodiment of the present invention, a domain offset mitigation system based on power equipment images includes:

[0086] The model building module is used to build a generalized model based on insulator defect detection;

[0087] The invariant feature extraction module is used to extract invariant features by adding a regularization term based on the IRM risk extrapolation method, so that the loss of aligned samples in each domain is the same.

[0088] The minimization module is used to minimize the distributional differences in the feature space by measuring the Jensen-Shannon divergence of each domain feature in a selected layer of the generalization model, thereby making the feature spaces similar and reducing the domain offset between the source and target domains.

[0089] This invention addresses the key challenge of domain offset in insulator defect detection by proposing a general framework to effectively mitigate the domain offset problem. The invention employs the proposed IR-IQA domain extension and DCIL methods to achieve mAP 50 The performance of the metrics in detecting fracture and spontaneous breakage defects was improved by 4.2% and 5.3%, respectively. Real-world datasets are difficult to generalize across domains due to domain bias. In contrast, the embodiments of this invention can generate domain-specific synthetic data at low cost, thereby enhancing generalization ability. It can serve as an effective solution to the problems of insufficient data and domain bias in insulator defect detection.

[0090] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0091] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A domain offset mitigation method based on power equipment images, characterized in that, include: Construct a generalized model based on insulator defect detection; Based on the IRM risk extrapolation method, a regularization term is added to extract invariant features, so that the loss of aligned samples in each domain is the same. By measuring the Jensen-Shannon divergence of each domain feature in a selected layer of the generalization model, the distributional differences in the feature space are minimized, making the feature spaces similar and reducing the domain offset between the source and target domains.

2. The domain offset mitigation method based on power equipment images according to claim 1, characterized in that, Before adding a regularization term to extract invariant features based on the IRM risk extrapolation method, the process also includes: synthesizing insulator defect data in three-dimensional space to obtain a three-dimensional synthetic dataset; Style transfer is performed on the three-dimensional synthetic dataset to extend a synthetic defect image from a single-source domain to a multi-source domain.

3. The domain offset mitigation method based on power equipment images according to claim 1, characterized in that, The IRM risk extrapolation method is expressed as follows: in, Let represent the experience loss risk in the K domain, e be the domain index, θ be the learning model that minimizes the equation, λ be the hyperparameter that needs to be tuned, and Var(R) be the hyperparameter. e (ω,Φ)) is the risk variance for each domain.

4. The domain offset mitigation method based on power equipment images according to claim 3, characterized in that, The penalty loss for training in the IRM risk extrapolation method is set as follows: L IRM =λ IRM Var(L 0 (θ),L 1 (θ),...,L K (i)); Where L(·) represents the loss function in the generalization model.

5. The domain offset mitigation method based on power equipment images according to claim 1, characterized in that, Minimize the distributional differences in the feature space by measuring the Jensen-Shannon divergence of features in selected layers of the generalization model, including: Given the feature outputs of the selected n layers, the corresponding feature map for each source domain in each layer is obtained; Convert the feature divergence into probabilities that sum to 1; Calculate the average feature map of all source domains, denoted as Q; Calculate the feature map P for each source domain i The Jansen-Shannon divergence between Q and Q.

6. The domain offset mitigation method based on power equipment images according to claim 5, characterized in that, Calculate the feature map P for each source domain i The Jensen-Shannon divergence between the average feature map Q across multiple domains is as follows: Where KL(·) is the KL divergence, JS(·) is the Jensen-Shannon divergence, N is the length of the eigenvector, and p m Let q be the m-th value in feature P. m Let m be the m-th value in feature Q.

7. A domain offset mitigation method based on power equipment images according to claim 6, characterized in that, The loss component of each selected layer l i Defined as the variance of the source domain output divergence, the distribution loss of each domain in each layer is multiplied by a hyperparameter λ. FDM This is combined to form another regularization component, resulting in the distribution difference loss; The final loss function of the model is obtained based on the distribution difference loss.

8. A domain offset mitigation system based on power equipment images, characterized in that, include: The model building module is used to build a generalized model based on insulator defect detection; The invariant feature extraction module is used to extract invariant features by adding a regularization term based on the IRM risk extrapolation method, so that the loss of aligned samples in each domain is the same. The minimization module is used to minimize the distributional differences in the feature space by measuring the Jensen-Shannon divergence of each domain feature in a selected layer of the generalization model, thereby making the feature spaces similar and reducing the domain offset between the source and target domains.

Images