A power transmission inspection defect identification model construction method and device and a storage medium
By classifying, labeling, and training a dataset of defects from power transmission inspections, a single model is constructed to identify multiple defects, solving the problem of wasted computing resources and time in existing technologies and achieving efficient defect identification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SGCC GENERAL AVIATION
- Filing Date
- 2022-11-17
- Publication Date
- 2026-07-31
AI Technical Summary
In existing technologies, using multiple models to sequentially identify the same image requires significantly more computing resources and inference time.
By acquiring multiple power transmission inspection defect datasets, labeling each defect dataset with defect categories, training a defect identification sub-model separately, and obtaining the final defect identification model through total loss training, replacing the batch normalization layer with a layer normalization layer, and using algorithms such as ScaledYolov4-P6, a single model can identify multiple defects.
It enables the identification of multiple defects in a single image in a single model without reducing recognition accuracy, thus solving the problem of wasted computing resources and inference time.
Smart Images
Figure CN115908316B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of power transmission line defect identification technology, specifically a method, device, and storage medium for constructing a power transmission inspection defect identification model. Background Technology
[0002] Defects and hidden dangers in power transmission lines can lead to power grid operation accidents, causing power outages, further equipment damage, and other problems. Therefore, regular inspections of power transmission lines are necessary to promptly identify and address defects / potential hazards. Currently, the process for power transmission line inspections using drones and helicopters is as follows: First, the aircraft inspects the target line and towers along a pre-determined route. During this process, the drone pilot or helicopter inspector takes photos of every potentially defective piece of equipment on the line and towers. Then, all the photos are uploaded to an inspection image analysis system, where an intelligent model identifies potential defects. Next, the defects identified by the model are manually reviewed, and the remaining images are randomly checked to find all defects in the images. A defect report is then created (each line on the defect report describes a defect, including the tower, location, specific details, and severity level). Finally, the maintenance unit accepts the defect report and arranges a repair plan to eliminate the defects.
[0003] The accuracy of intelligent models in recognizing images directly impacts the workload of manual review and the defect detection rate. To train high-precision intelligent models, defect images need to be labeled, and corresponding defect models need to be trained using image target detection algorithms. However, transmission lines can have numerous types of defects, and a single labeler typically cannot simultaneously master all types of defects and label them in images. Therefore, a common industry practice is to divide defects into multiple groups based on their location or shape, complete sample labeling and model training for each group, and then use multiple models to sequentially identify the same image, merging the results to achieve identification of defect categories for all groups. The problem with this approach is that using multiple models to sequentially identify the same image requires significantly more computational resources and inference time. Summary of the Invention
[0004] In view of this, the purpose of the present invention is to provide a method, device and storage medium for constructing a defect identification model for power transmission inspection, so as to solve the problem in the prior art that using multiple models to identify the same image in sequence requires many times more computing resources and inference time.
[0005] According to a first aspect of the present invention, a method for constructing a power transmission inspection defect identification model is provided, comprising:
[0006] Multiple power transmission inspection defect datasets were obtained, and each defect dataset was labeled with a defect category. Different defect datasets were labeled with different defect categories.
[0007] Each defect dataset is input into a pre-built defect recognition model for training, and a defect recognition sub-model corresponding to each defect dataset is obtained. The defect recognition sub-model corresponding to each defect dataset outputs the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset for each defect dataset.
[0008] For each power transmission inspection defect dataset, preprocessing was performed and images were extracted to obtain training samples;
[0009] The training samples are input into the pre-built defect recognition model, and the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset are output for each sample. The prediction loss of the corresponding sample is calculated based on the output results.
[0010] Based on the target foreground probability, target box coordinates, and classification probability of each type of defect labeled in the defect dataset for each defect dataset output by the defect identification sub-model, the total loss is obtained based on the classification probability of each type of defect and the prediction loss of the corresponding sample.
[0011] The pre-built defect identification model is trained based on the total loss to obtain the final defect identification model.
[0012] Preferably,
[0013] Replace the batch normalization layer in the pre-built defect identification model with a layer normalization layer or a group normalization layer:
[0014] The replacement includes:
[0015] Replace CSPDarknet in the feature extraction module of the defect identification model with another network model that does not contain a batch normalization layer.
[0016] Replace the batch normalization layer in the detection head module of the defect identification model with a layer normalization layer.
[0017] Preferably,
[0018] The process of preprocessing each power transmission inspection defect dataset and extracting images to obtain training samples includes:
[0019] Before each round of training begins, class balance weights are generated based on the number of samples for each defect category and the recognition accuracy of the model's output from the previous round of training.
[0020] Based on the number of defects of each category contained in each image, the corresponding category balance weights are added together to obtain the sampling weight of each image;
[0021] The sampling weight of each image is reweighted according to the defect dataset to which it belongs, so that the probability of extracting an image from each defect dataset meets the preset value.
[0022] Training samples are obtained by extracting images from the defect dataset according to sampling weights.
[0023] Preferably,
[0024] The training samples obtained by extracting images from the defect dataset according to sampling weights include:
[0025] Extract an image from any defect dataset. Based on the sampling weight of the image, extract multiple images belonging to the same defect dataset and preprocess them to obtain an image of a specified resolution, along with the defect dataset number to which the image belongs. Repeat the above steps to obtain multiple images and the defect dataset number to which the images belong, forming a training sample.
[0026] Preferably,
[0027] The calculation of the prediction loss for the corresponding sample based on the output results includes:
[0028] For samples belonging to the same batch, the target box regression loss and classification loss of different samples are weighted and averaged according to the number of samples. The weighted average target box regression loss, the weighted average target classification loss and the foreground prediction loss are then summed to obtain the prediction loss of the batch of samples.
[0029] Preferably,
[0030] The training method for the pre-built defect identification model adopts the ScaledYolov4-P6 algorithm, ScaledYolov4-P5 algorithm, or ScaledYolov4-P7 algorithm.
[0031] Preferably,
[0032] The process of obtaining the total loss based on the classification probability of each type of defect and the prediction loss of the corresponding sample includes:
[0033] The classification probability of each type of defect is compared with the prediction result of the pre-trained defect recognition model corresponding to the defect dataset to obtain the distillation loss based on the average variance.
[0034] The total loss is obtained by weighting the prediction loss and distillation loss according to specific weights.
[0035] According to a second aspect of the present invention, a power transmission inspection defect identification model construction device is provided, the device comprising:
[0036] Defect labeling module: Used to acquire multiple power transmission inspection defect datasets, label each defect dataset with defect categories, and label different defect datasets with different defect categories;
[0037] Defect recognition sub-model acquisition module: used to input each defect dataset into the pre-built defect recognition model for training, and obtain the defect recognition sub-model corresponding to each defect dataset. The defect recognition sub-model corresponding to each defect dataset outputs the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset for each defect dataset.
[0038] Training sample acquisition module: used to preprocess each power transmission inspection defect dataset and extract images to obtain training samples;
[0039] Prediction loss acquisition module: It is used to input training samples into the pre-built defect recognition model, output the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset for each sample, and calculate the prediction loss of the corresponding sample based on the output results.
[0040] Distillation loss acquisition module: It is used to obtain the total loss based on the target foreground probability, the coordinates of the target box and the classification probability of each type of defect labeled in the defect dataset for each defect dataset output by the defect recognition sub-model, and the classification probability of each type of defect.
[0041] Final Defect Recognition Model Acquisition Module: Used to train the pre-built defect recognition model based on the total loss to obtain the final defect recognition model.
[0042] According to a third aspect of the present invention, a storage medium is provided, the storage medium storing a computer program, which, when executed by a processor, implements the various steps in the method for constructing a power transmission inspection defect identification model as described in any of the preceding claims.
[0043] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:
[0044] This application labels defect types on multiple defect datasets and trains a defect recognition model on each individual defect dataset to obtain a defect recognition sub-model that can be used to identify defects in a single defect dataset. Then, training samples are obtained from multiple defect datasets and input into the pre-built defect recognition model to obtain the prediction loss for the corresponding samples. The total loss is obtained based on the prediction loss of each type of sample and the output of the corresponding defect recognition sub-model for that defect dataset. The pre-built defect recognition model is trained based on the total loss to obtain the final defect recognition model. Through the above scheme, a single model can identify multiple defect analogies in an image without reducing the recognition accuracy compared to a single model. This solves the problem in the prior art that using multiple models to identify the same image sequentially requires many times more computing resources and inference time.
[0045] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description
[0046] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0047] Figure 1 This is a flowchart illustrating a method for constructing a defect identification model for power transmission inspection, according to an exemplary embodiment.
[0048] Figure 2 This is a system schematic diagram of a power transmission inspection defect identification model construction device according to another exemplary embodiment;
[0049] In the attached diagram: 1-Defect annotation module, 2-Defect identification sub-model acquisition module, 3-Training sample acquisition module, 4-Prediction loss acquisition module, 5-Distillation loss acquisition module, 6-Final defect identification model acquisition module. Detailed Implementation
[0050] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.
[0051] Example 1
[0052] Figure 1This is a flowchart illustrating a method for constructing a defect identification model for power transmission inspection, according to an exemplary embodiment. Figure 1 As shown, it includes:
[0053] S1. Obtain multiple power transmission inspection defect datasets, label each defect dataset with defect categories, and label different defect datasets with different defect categories;
[0054] S2, each defect dataset is input into a pre-built defect recognition model for training, and a defect recognition sub-model corresponding to each defect dataset is obtained. The defect recognition sub-model corresponding to each defect dataset outputs the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset for each defect dataset.
[0055] S3, preprocess each power transmission inspection defect dataset and extract images to obtain training samples;
[0056] S4. Input the training samples into the pre-built defect recognition model, output the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset for each sample, and calculate the prediction loss of the corresponding sample based on the output results.
[0057] S5. Based on the target foreground probability, target box coordinates, and classification probability of each type of defect labeled in the defect dataset output by the defect identification sub-model, the total loss is obtained based on the classification probability of each type of defect and the prediction loss of the corresponding sample.
[0058] S6. Train the pre-built defect identification model based on the total loss to obtain the final defect identification model;
[0059] First, it should be noted that the model training method described in this invention is based on ScaledYolo-v4 (see paper [2011.08036] Scaled-YOLOv4: Scaling Cross Stage Partial Network (arxiv.org)). Assuming there are P datasets, each dataset has labeled defect categories n1, n2, ..., n_p, this method has five improvements compared to the original method, and these five improvements are all reflected in the following scheme:
[0060] (1) Modify the model network structure to predict the foreground probability of the target belonging to that dataset, the xc, yc, w, h coordinates of the target box, and the classification probability of each class for each feature map at each resolution, for each anchor, and for each dataset. Therefore, the output dimension of the last layer is... Where N is the number of anchors at each resolution.
[0061] (2) Modify the loss function. For samples belonging to each dataset, extract the feature dimension corresponding to that dataset in the last layer, and calculate the loss of the corresponding sample based on the predicted foreground probability of the target, the xc, yc, w, h coordinates of the target box, and the classification probability of each class. For samples belonging to the same batch, perform a weighted average of the target box regression loss and classification loss of different samples according to the number of foreground samples.
[0062] (3) The preprocessing loader for training samples is modified so that when generating each sample, a dataset is first randomly specified, and then samples are extracted from that dataset according to the probability corresponding to the class balance weights of the corresponding dataset. These samples are used for Mosaic (see paper [2004.10934] YOLOv4: Optimal Speed and Accuracy of Object Detection (arxiv.org)) and Mix-up (see paper [1710.09412] mixup: Beyond Empirical RiskMinimization (arxiv.org)) sample augmentation. Among them, the class balance weights are generated based on the number of samples and recognition accuracy of each class, as well as the number of objects of each class contained in the current image.
[0063] (4) Modify the model's feature extraction network (backbone). Replace the original CSPDarknet (see paper [2004.10934] YOLOv4: Optimal Speed and Accuracy of Object Detection (arxiv.org)) with other networks that do not include batch normalization, such as ViT or SWIN (see paper [2103.14030] SwinTransformer: Hierarchical Vision Transformer using Shifted Windows (arxiv.org)), or replace BN in CSPDarknet with LN. We found that when training the model across datasets, the feature distribution of the model is inconsistent across different layers of the network, resulting in poor performance of BN. Therefore, replacing BN with LN can improve the overall recognition accuracy of the model.
[0064] (5) To further improve model accuracy, we first use the original ScaledYolov4-P6 algorithm to train a single model for each dataset that can only identify the defect category corresponding to that dataset. When training the multi-dataset model using the modified methods described in (1)-(4), we first load P single-dataset models. Then, according to the dataset number (assumed to be i) corresponding to each sample, we select the corresponding single-dataset model, use the model to predict the sample, and obtain the output at each resolution (the feature dimension of each feature point output is ni+5); we compare the output of the single model for the sample with the output of the corresponding dimension obtained by the multi-dataset model, calculate its mean squared error loss, and use this loss as a weighted distillation loss to be included in the total training loss.
[0065] Understandably, this application acquires multiple power transmission inspection defect datasets, labels each dataset with defect categories, and assigns different defect categories to different datasets. It's worth noting that each defect dataset can contain multiple defect categories, but different datasets will not label the same defect category. A single inspection image may contain defects of multiple categories; that is, an image may appear in multiple defect datasets, but each dataset labels the image with a different defect category. After labeling multiple defect datasets, each dataset is input into a defect recognition model, and the model is then used to perform defect recognition based on the defect datasets. Training is performed to obtain a defect recognition sub-model capable of identifying defects in the defect dataset. In other words, one defect dataset corresponds to one defect recognition sub-model; this is the method currently used for defect recognition in images. Then, each power transmission inspection defect dataset is preprocessed, and images are extracted to obtain training samples. This means that the training samples may contain images from multiple defect datasets. The training samples are input into the pre-built defect recognition model, which outputs the feature dimensions corresponding to each defect dataset. These dimensions include the target foreground probability, the coordinates of the bounding box (xc, yc, w, h), and the classification probability of each defect class labeled in the dataset. Based on the output results, the pre-classification of the corresponding samples is calculated. Regarding loss measurement, it's worth noting that the model's output feature dimensions may include feature dimensions from multiple defect datasets. The order of these feature dimensions is related to the input order of the samples. That is, if defect dataset A has 5 labeled defect classes and defect dataset B has 4 labeled defect classes, and the input order is A first, then B, and the model outputs 19 feature dimensions, then the first 10 are the feature dimensions corresponding to defect dataset A, including one foreground probability dimension, four bounding box coordinate dimensions, and five classification probability dimensions for each defect class. The last 9 are the feature dimensions of the samples in defect dataset B. The loss is calculated based on the features output from each defect dataset. The prediction loss is calculated by comparing the feature dimensions of each defect recognition sub-model obtained earlier with the output of the defect recognition model to obtain the distillation loss based on the average variance. The prediction loss and distillation loss are weighted and summed according to specific weights to obtain the total loss. The pre-built defect recognition model is then trained based on the total loss to obtain the final defect recognition model. Through the above scheme, a single model can recognize multiple defect analogies in an image without reducing the recognition accuracy compared to a single model. This solves the problem in existing technologies where multiple models are used to recognize the same image sequentially, which requires many times more computing resources and inference time.
[0066] Preferably,
[0067] Replace the batch normalization layer in the pre-built defect identification model with a layer normalization layer or a group normalization layer:
[0068] The replacement includes:
[0069] Replace CSPDarknet in the feature extraction module of the defect identification model with another network model that does not contain a batch normalization layer.
[0070] Replace the batch normalization layer in the detection head module of the defect identification model with a layer normalization layer;
[0071] Understandably, the original CSPDarknet in the feature extraction module is replaced with other networks that do not include batch normalization layers, such as the ViT or SWIN model, and the batch normalization network layer in the detection head module is replaced with a layer normalization network layer. We found that when training the model across datasets, the feature distribution of samples from different datasets is inconsistent across different layers of the network, resulting in poor performance when using batch normalization layers. Therefore, replacing the batch normalization layer with a layer normalization layer or a group normalization layer can improve the overall recognition accuracy of the model.
[0072] Preferably,
[0073] The process of preprocessing each power transmission inspection defect dataset and extracting images to obtain training samples includes:
[0074] Before each round of training begins, class balance weights are generated based on the number of samples for each defect category and the recognition accuracy of the model's output from the previous round of training.
[0075] Based on the number of defects of each category contained in each image, the corresponding category balance weights are added together to obtain the sampling weight of each image;
[0076] The sampling weight of each image is reweighted according to the defect dataset to which it belongs, so that the probability of extracting an image from each defect dataset meets the preset value.
[0077] Training samples are obtained by extracting images from the defect dataset according to sampling weights;
[0078] Understandably, before each training round, class balance weights are generated based on the number of samples in each class and the recognition accuracy of the model's output from the previous training round. Based on the number of targets in each class contained in each image, the corresponding class balance weights are added together to obtain the sampling weight of each image. Then, the image sampling weights are reweighted according to their respective datasets to ensure that the probability of extracting images from each dataset is the same. The preprocessing loader for training samples is modified so that when generating each sample, a dataset is first randomly specified, and then samples are extracted from that dataset according to the probability corresponding to the class balance weight of the corresponding dataset for Mosaic and Mix-up sample expansion.
[0079] Preferably,
[0080] The training samples obtained by extracting images from the defect dataset according to sampling weights include:
[0081] Extract an image from any defect dataset. Based on the sampling weight of the image, extract multiple images belonging to the same defect dataset and preprocess them to obtain an image of a specified resolution and the defect dataset number to which the image belongs. Repeat the above steps to obtain multiple images and the defect dataset number to which the images belong, and form a training sample.
[0082] Understandably, during training, based on the reweighted sampling weights of each image, one image is selected. Then, multiple images belonging to the same dataset are selected according to their sampling weights. After sample augmentation using random color dithering, Mosaic, and Mix-up, an image of a specified resolution and its corresponding dataset number are obtained. This process is repeated multiple times to obtain a specified number of images and their corresponding dataset numbers, forming a batch of training samples for subsequent model training.
[0083] Preferably,
[0084] The calculation of the prediction loss for the corresponding sample based on the output results includes:
[0085] For samples belonging to the same batch, the target box regression loss and classification loss of different samples are weighted and averaged according to the number of samples. The weighted average target box regression loss, the weighted average target classification loss and the foreground prediction loss are then summed to obtain the prediction loss of the batch of samples.
[0086] Understandably, this approach modifies the existing loss function. For samples belonging to each dataset, it extracts the feature dimension corresponding to that dataset from the last layer and calculates the loss for the corresponding sample based on the model's predicted foreground probability, the xc, yc, w, and h coordinates of the bounding box, and the classification probability of each class. For samples belonging to the same batch, it performs a weighted average of the bounding box regression loss and classification loss of different samples according to the number of foreground samples. Then, it sums the weighted average bounding box regression loss, the weighted average target classification loss, and the foreground prediction loss to obtain the prediction loss for that batch of samples.
[0087] Preferably,
[0088] The training method for the pre-built defect identification model adopts the ScaledYolov4-P6 algorithm, ScaledYolov4-P5 algorithm, or ScaledYolov4-P7 algorithm.
[0089] Understandably, in order to further improve the accuracy of the model, the pre-built defect identification model is trained using the ScaledYolov4-P6 algorithm (the P5 or P7 algorithm can also be used).
[0090] Preferably,
[0091] The process of obtaining the total loss based on the classification probability of each type of defect and the prediction loss of the corresponding sample includes:
[0092] The classification probability of each type of defect is compared with the prediction result of the pre-trained defect recognition model corresponding to the defect dataset to obtain the distillation loss based on the average variance.
[0093] The total loss is obtained by weighting the prediction loss and distillation loss according to specific weights.
[0094] To verify the above solution, the following examples demonstrate that the defect recognition model of this solution can identify all defects in an image at once, and that the existing method of inputting the image into multiple defect recognition models can achieve the same effect, as shown below:
[0095] The defect dataset includes: (1) Grounding body exposure dataset, containing a single defect category: grounding body exposure, with a total of 261 images; (2) Typical targets and defects dataset of line equipment, containing 9 categories of equipment and defects such as vibration dampers, insulators, and foreign objects, with a total of 2376 images; (3) Burn and corrosion dataset, containing 12 categories of defects such as corrosion, burns, and dirt of line equipment, with a total of 6035 images. We divided the training and validation sets according to a 15% ratio. The model accuracy based on a single dataset and the accuracy of the model after applying different improved algorithms on each dataset (mAP@0.2) are shown in the following table:
[0096]
[0097] As shown in the table above, training the model by simply mixing the three datasets resulted in a significant drop in model accuracy. After completing the image labels in the mixed dataset and those outside the single dataset using the single dataset model, the model accuracy recovered somewhat, but still lagged significantly behind the single-model accuracy. The accuracy difference was most pronounced in dataset 1 due to its smaller sample size. Applying improvements 1-3 significantly improved the model accuracy. Replacing the batch normalization layer (BN) with a layer normalization layer (LN) and adding the distillation loss from the single model further improved the model accuracy to 0.949, 0.894, and 0.680, respectively, matching the accuracy of the single model. Therefore, the model trained using the method described in this study meets practical requirements.
[0098] Example 2
[0099] This embodiment also discloses a power transmission inspection defect identification model construction device, as shown in the attached figure. Figure 2 As shown, it includes:
[0100] Defect labeling module 1: Used to acquire multiple power transmission inspection defect datasets, label each defect dataset with defect categories, and label different defect datasets with different defect categories;
[0101] Defect recognition sub-model acquisition module 2: It is used to input each defect dataset into the pre-built defect recognition model for training, and obtain the defect recognition sub-model corresponding to each defect dataset. The defect recognition sub-model corresponding to each defect dataset outputs the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset for each defect dataset.
[0102] Training sample acquisition module 3: used to preprocess each power transmission inspection defect dataset and extract images to obtain training samples;
[0103] Prediction Loss Acquisition Module 4: This module is used to input training samples into the pre-built defect recognition model, output the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset for each sample, and calculate the prediction loss of the corresponding sample based on the output results.
[0104] Distillation loss acquisition module 5: It is used to obtain the total loss based on the target foreground probability, the coordinates of the target box and the classification probability of each type of defect labeled in the defect dataset for each defect dataset output by the defect identification sub-model, and the classification probability of each type of defect.
[0105] Final Defect Recognition Model Acquisition Module 6: Used to train the pre-built defect recognition model based on the total loss to obtain the final defect recognition model;
[0106] Understandably, this application acquires multiple power transmission inspection defect datasets through defect annotation module 1, annotates each defect dataset with defect categories, and the defect categories annotated for different defect datasets are different; through defect recognition sub-model acquisition module 2, each defect dataset is input into a pre-built defect recognition model for training, obtaining a defect recognition sub-model corresponding to each defect dataset. The defect recognition sub-model corresponding to each defect dataset outputs the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect annotated for each defect dataset; through training sample acquisition module 3, each power transmission inspection defect dataset is preprocessed and images are extracted to obtain training samples; through prediction loss acquisition module 4, the training samples are input into the pre-built defect recognition model, outputting the target foreground probability, the coordinates of the target box, and the number of defects for each sample corresponding to each defect dataset. The classification probability of each type of defect in the data set is used to calculate the prediction loss of the corresponding sample based on the output results. The distillation loss acquisition module 5 compares the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect in the defect dataset with the prediction loss to obtain the distillation loss based on the average variance. The final defect recognition model acquisition module 6 calculates the weighted sum of the prediction loss and the distillation loss according to specific weights to obtain the total loss. The pre-built defect recognition model is trained based on the total loss to obtain the final defect recognition model. Through the above scheme, a single model can recognize multiple defect analogies in an image without reducing the recognition accuracy compared to a single model. This solves the problem in the existing technology that using multiple models to recognize the same image sequentially requires many times more computing resources and inference time.
[0107] Example 3:
[0108] This embodiment also discloses a storage medium storing a computer program. When the computer program is executed by a processor, it implements the various steps in the method for constructing a power transmission inspection defect identification model as described in any of the above embodiments.
[0109] It is understood that the storage media mentioned above can be read-only memory, disk, or optical disc, etc.
[0110] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.
[0111] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.
[0112] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.
[0113] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0114] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
[0115] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0116] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.
[0117] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0118] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A method for constructing a power transmission inspection defect identification model, characterized in that, The method includes: Multiple power transmission inspection defect datasets were obtained, and each defect dataset was labeled with a defect category. Different defect datasets were labeled with different defect categories. Each defect dataset is input into a pre-built defect recognition model for training, and a defect recognition sub-model corresponding to each defect dataset is obtained. The defect recognition sub-model corresponding to each defect dataset outputs the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset for each defect dataset. Replace the batch normalization layer in the pre-built defect identification model with a layer normalization layer or a group normalization layer: The replacement includes: Replace CSPDarknet in the feature extraction module of the defect identification model with another network model that does not contain a batch normalization layer. Replace the batch normalization layer in the detection head module of the defect identification model with a layer normalization layer; For each power transmission inspection defect dataset, preprocessing was performed and images were extracted to obtain training samples; The process of preprocessing each power transmission inspection defect dataset and extracting images to obtain training samples includes: Before each round of training begins, class balance weights are generated based on the number of samples for each defect category and the recognition accuracy of the model's output from the previous round of training. Based on the number of defects of each category contained in each image, the corresponding category balance weights are added together to obtain the sampling weight of each image; The sampling weight of each image is reweighted according to the defect dataset to which it belongs, so that the probability of extracting an image from each defect dataset meets the preset value. Training samples are obtained by extracting images from the defect dataset according to sampling weights; The training samples are input into the pre-built defect recognition model, and the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset are output for each sample. The prediction loss of the corresponding sample is calculated based on the output results. Based on the target foreground probability, target box coordinates, and classification probability of each type of defect labeled in the defect dataset for each defect dataset output by the defect identification sub-model, the total loss is obtained based on the classification probability of each type of defect and the prediction loss of the corresponding sample. The process of obtaining the total loss based on the classification probability of each type of defect and the prediction loss of the corresponding sample includes: The classification probability of each type of defect is compared with the prediction result of the pre-trained defect recognition model corresponding to the defect dataset to obtain the distillation loss based on the average variance. The total loss is obtained by weighting the predicted loss and the distillation loss according to specific weights. The pre-built defect identification model is trained based on the total loss to obtain the final defect identification model.
2. The method according to claim 1, characterized in that, The training samples obtained by extracting images from the defect dataset according to sampling weights include: Extract an image from any defect dataset. Based on the sampling weight of the image, extract multiple images belonging to the same defect dataset and preprocess them to obtain an image of a specified resolution, along with the defect dataset number to which the image belongs. Repeat the above steps to obtain multiple images and the defect dataset number to which the images belong, forming a training sample.
3. The method according to claim 2, characterized in that, The calculation of the prediction loss for the corresponding sample based on the output results includes: For samples belonging to the same batch, the target box regression loss and classification loss of different samples are weighted and averaged according to the number of samples. The weighted average target box regression loss, the weighted average target classification loss and the foreground prediction loss are then summed to obtain the prediction loss of the batch of samples.
4. The method according to claim 1, characterized in that, The training method for the pre-built defect identification model adopts the ScaledYolov4-P6 algorithm, ScaledYolov4-P5 algorithm, or ScaledYolov4-P7 algorithm.
5. A power transmission inspection defect identification model construction apparatus characterized by comprising: The device includes: Defect labeling module: Used to acquire multiple power transmission inspection defect datasets, label each defect dataset with defect categories, and label different defect datasets with different defect categories; Defect recognition sub-model acquisition module: used to input each defect dataset into the pre-built defect recognition model for training, and obtain the defect recognition sub-model corresponding to each defect dataset. The defect recognition sub-model corresponding to each defect dataset outputs the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset for each defect dataset. Replace the batch normalization layer in the pre-built defect identification model with a layer normalization layer or a group normalization layer: The replacement includes: Replace CSPDarknet in the feature extraction module of the defect identification model with another network model that does not contain a batch normalization layer. Replace the batch normalization layer in the detection head module of the defect identification model with a layer normalization layer; Training sample acquisition module: used to preprocess each power transmission inspection defect dataset and extract images to obtain training samples; The process of preprocessing each power transmission inspection defect dataset and extracting images to obtain training samples includes: Before each round of training begins, class balance weights are generated based on the number of samples for each defect category and the recognition accuracy of the model's output from the previous round of training. Based on the number of defects of each category contained in each image, the corresponding category balance weights are added together to obtain the sampling weight of each image; The sampling weight of each image is reweighted according to the defect dataset to which it belongs, so that the probability of extracting an image from each defect dataset meets the preset value. Training samples are obtained by extracting images from the defect dataset according to sampling weights; Prediction loss acquisition module: It is used to input training samples into the pre-built defect recognition model, output the target foreground probability, the coordinates of the target box, and the classification probability of each type of defect labeled in the defect dataset for each sample, and calculate the prediction loss of the corresponding sample based on the output results. Distillation loss acquisition module: It is used to obtain the total loss based on the target foreground probability, the coordinates of the target box and the classification probability of each type of defect labeled in the defect dataset for each defect dataset output by the defect recognition sub-model, and the classification probability of each type of defect. The process of obtaining the total loss based on the classification probability of each type of defect and the prediction loss of the corresponding sample includes: The classification probability of each type of defect is compared with the prediction result of the pre-trained defect recognition model corresponding to the defect dataset to obtain the distillation loss based on the average variance. The total loss is obtained by weighting the predicted loss and the distillation loss according to specific weights. Final Defect Recognition Model Acquisition Module: Used to train the pre-built defect recognition model based on the total loss to obtain the final defect recognition model.
6. A storage medium, characterized by The storage medium stores a computer program, which, when executed by a processor, implements the various steps in the method for constructing a power transmission inspection defect identification model as described in any one of claims 1-4.