Defect identification model construction method and device based on electric power inspection small sample, and storage medium

Abstract

The invention discloses a defect identification model construction method and device based on electric power inspection small samples, a storage medium and computer equipment, and the method comprises the steps: obtaining an electric power inspection picture sample set, and obtaining a defect identification model based on the defect tag carrying condition of each electric power inspection picture sample; dividing the electric power inspection picture sample set into a labeled subset and a non-labeled subset, wherein the number of samples of the labeled subset is significantly smaller than the number of samples of the non-labeled subset; performing fine tuning on the teacher model pre-trained based on the general image data set through the labeled subset to obtain a fine-tuned teacher model; based on the non-label subset, through the fine-tuned teacher model, outputting defect category probability distribution of each electric power inspection picture sample in the non-label subset, and generating a soft label of the electric power inspection picture sample based on the defect category probability distribution; and performing model training on the initial recognition model based on the labeled subset, the unlabeled subset after the soft label is generated and the target loss function to obtain a defect recognition model.

Description

Technical Field

[0001] This application relates to the field of power system operation and maintenance technology, and in particular to a method and device for constructing a defect identification model based on a small sample of power inspection, as well as a storage medium and computer equipment. Background Technology

[0002] In power systems, power grid inspection is a core component ensuring the safe and stable operation of the power grid. With the continuous expansion of the power grid and the increasing complexity of power equipment, traditional manual inspection methods, due to their low efficiency, high cost, and susceptibility to subjective factors, are no longer sufficient to meet the real-time and accuracy requirements of intelligent inspection systems. Therefore, artificial intelligence-based power grid inspection technology has emerged. Through image recognition, target detection, and other technologies, it achieves intelligent monitoring and fault diagnosis of equipment status, significantly improving inspection efficiency and accuracy. However, the scarcity and uneven distribution of power equipment fault samples have become a key bottleneck restricting the practical application of this technology.

[0003] To alleviate the small sample size problem, researchers have employed various techniques in the past. Early approaches primarily involved data augmentation to expand datasets, such as rotating, cropping, and scaling images. However, the resulting data had limited diversity and was prone to noise, offering little improvement to model performance. In recent years, semi-supervised learning and transfer learning have been introduced, attempting to reduce reliance on labeled data by transferring models pre-trained on large-scale general datasets to power line inspection tasks. However, power sector data is unique, with significant differences in distribution between general datasets and power line inspection data. This results in slow model convergence in small sample scenarios, making it difficult to accurately capture fault characteristics of power equipment and ultimately failing to meet actual inspection needs. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, storage medium, and computer equipment for constructing a defect identification model based on small samples from power inspections. The method divides the power inspection image sample set into a significantly small number of labeled subsets and a large number of unlabeled subsets. It then uses the labeled subsets to fine-tune a general pre-trained teacher model, and generates high-quality soft labels for the unlabeled subsets based on the fine-tuned teacher model, thus providing valuable supervisory information to the unlabeled samples. Finally, it combines the labeled data, the unlabeled data after generating soft labels, and the target loss function to train the initial identification model. This method fully utilizes the domain characteristics of power inspection data, effectively alleviates the data scarcity problem in small sample scenarios, and avoids the impact of distribution differences caused by the migration of general datasets. Compared to training only with labeled samples, this method can significantly improve the model's generalization ability and recognition accuracy, reduce the dependence on a large number of manually labeled samples, and save manpower and time costs.

[0005] According to one aspect of this application, a method for constructing a defect identification model based on a small sample of power line inspection is provided, comprising: A power inspection image sample set is obtained, and based on the defect label carrying status of each power inspection image sample in the power inspection image sample set, the power inspection image sample set is divided into a labeled subset and an unlabeled subset, wherein the number of samples in the labeled subset is significantly smaller than the number of samples in the unlabeled subset. The teacher model pre-trained on a general image dataset is fine-tuned using the labeled subset to obtain the fine-tuned teacher model. Based on the unlabeled subset, the teacher model, after fine-tuning, outputs the probability distribution of defect categories for each power inspection image sample in the unlabeled subset, and generates soft labels for the corresponding power inspection image samples based on the probability distribution of defect categories. Based on the labeled subset, the unlabeled subset after generating soft labels, and the target loss function, the initial identification model is trained to obtain the defect identification model.

[0006] According to another aspect of this application, a defect identification model construction device based on small samples of power line inspection is provided, comprising: The sample set partitioning module is used to acquire a power inspection image sample set and, based on the defect label carrying status of each power inspection image sample in the power inspection image sample set, divide the power inspection image sample set into a labeled subset and an unlabeled subset, wherein the number of samples in the labeled subset is significantly smaller than the number of samples in the unlabeled subset. The model fine-tuning module is used to fine-tune the teacher model pre-trained on a general image dataset using the labeled subset, so as to obtain the fine-tuned teacher model. The tag generation module is used to output the defect category probability distribution corresponding to each power inspection image sample in the unlabeled subset through a fine-tuned teacher model, and generate soft tags for the corresponding power inspection image samples based on the defect category probability distribution. The model training module is used to train the initial recognition model based on the labeled subset, the unlabeled subset after generating soft labels, and the target loss function to obtain the defect recognition model.

[0007] According to another aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described method for constructing a defect identification model based on a small sample of power line inspection.

[0008] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described method for constructing a defect identification model based on a small sample of power inspection.

[0009] By employing the above technical solutions, this application provides a method, apparatus, storage medium, and computer device for constructing a defect identification model based on small samples from power inspections. This method divides the power inspection image sample set into a significantly small number of labeled subsets and a large number of unlabeled subsets. It then uses the labeled subsets to fine-tune a general pre-trained teacher model, and generates high-quality soft labels for the unlabeled subsets based on the fine-tuned teacher model. This provides valuable supervisory information to the unlabeled samples. Finally, it combines labeled data, unlabeled data with generated soft labels, and the target loss function to train the initial identification model. This fully utilizes the domain characteristics of power inspection data, effectively alleviating the data scarcity problem in small sample scenarios, while avoiding the impact of distribution differences caused by the migration of general datasets. Compared to training only with labeled samples, this method significantly improves the model's generalization ability and recognition accuracy, reduces reliance on a large number of manually labeled samples, and saves labor and time costs.

[0010] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0011] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 The illustration shows a flowchart of a defect identification model construction method based on a small sample of power inspection provided in an embodiment of this application; Figure 2 This illustration shows a schematic diagram of a third loss value calculation process provided in an embodiment of this application; Figure 3 This illustration shows a structural schematic diagram of a defect identification model construction device based on a small sample of power inspection provided in an embodiment of this application; Figure 4 A schematic diagram of the device structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation

[0012] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0013] This embodiment provides a method for constructing a defect identification model based on a small sample of power line inspection data, such as... Figure 1 As shown, the method includes: Step 101: Obtain a power inspection image sample set, and based on the defect label carrying status of each power inspection image sample in the power inspection image sample set, divide the power inspection image sample set into a labeled subset and an unlabeled subset, wherein the number of samples in the labeled subset is significantly smaller than the number of samples in the unlabeled subset.

[0014] Step 102: Fine-tune the teacher model pre-trained on the general image dataset using the labeled subset to obtain the fine-tuned teacher model.

[0015] Step 103: Based on the unlabeled subset, the defect category probability distribution corresponding to each power inspection image sample in the unlabeled subset is output through the fine-tuned teacher model, and soft labels for the corresponding power inspection image samples are generated based on the defect category probability distribution.

[0016] Step 104: Based on the labeled subset, the unlabeled subset after generating soft labels, and the target loss function, train the initial recognition model to obtain the defect recognition model.

[0017] This application provides a method for constructing a defect identification model based on a small sample of power inspection, which aims to solve the problem that it is difficult to effectively construct a defect identification model in power inspection due to the limited number of samples (the number of labeled samples is significantly less than the number of unlabeled samples).

[0018] First, a power line inspection image sample set is obtained. Based on whether each power line inspection image sample carries a defect label, it is divided into labeled and unlabeled subsets. In actual power line inspection scenarios, due to the high cost and difficulty of manual labeling, the number of labeled samples is often far less than the number of unlabeled samples. This division method lays the foundation for subsequent processing of different types of power line inspection image samples using different methods. In a specific embodiment, the labeled subset can account for 10% of the samples, and the unlabeled subset can account for 90%. The labeled subset can include both faulty and normal samples, while the unlabeled subset only contains normal samples or low-confidence samples.

[0019] Subsequently, the pre-trained teacher model based on a general image dataset was fine-tuned using a labeled subset. The general image dataset possesses rich image feature information, and the pre-trained teacher model has already learned a large amount of general image feature representation capabilities. By using labeled samples from the power inspection field for fine-tuning, the teacher model can better adapt to the characteristics of power inspection images, focusing on feature learning related to power equipment defects, thus obtaining a fine-tuned teacher model that performs better on the defect recognition task. Here, defects can be such as "insulator cracks" or "tower corrosion."

[0020] Furthermore, for the unlabeled subset, a fine-tuned teacher model is used to output the defect category probability distribution for each unlabeled power inspection image sample. This probability distribution reflects the fine-tuned teacher model's judgment on the likelihood that the power inspection image sample belongs to different defect categories. Based on these defect category probability distributions, a soft label is generated for each unlabeled sample. Unlike traditional hard labels (which explicitly specify that a sample belongs to a certain category), soft labels provide richer information, reflecting the uncertainty of power inspection image samples across different categories. This helps the subsequent model training to better capture the relationship between sample features and categories.

[0021] Finally, the labeled subset, the unlabeled subset after generating soft labels, and the objective loss function are combined to train the initial identification model. The objective loss function comprehensively considers the supervision information of labeled samples (calculating the loss using hard labels and model prediction results) and the soft supervision information of unlabeled samples (calculating the loss using soft labels and model prediction results). By continuously optimizing the objective loss function and adjusting the model parameters of the initial identification model, the initial identification model gradually learns accurate power inspection defect identification capabilities under the combined effect of labeled and unlabeled samples, ultimately yielding the defect identification model.

[0022] By applying the technical solution of this embodiment, a power inspection image sample set is first obtained. Based on whether each power inspection image sample carries a defect label, it is divided into labeled and unlabeled subsets. Then, the labeled subset is used to fine-tune a pre-trained teacher model based on a general image dataset. Further, for the unlabeled subset, the fine-tuned teacher model is used to output the defect category probability distribution corresponding to each unlabeled power inspection image sample. Based on these defect category probability distributions, a soft label is generated for each unlabeled sample. Finally, the labeled subset, the unlabeled subset after generating soft labels, and the target loss function are combined to train the initial recognition model, ultimately obtaining the defect recognition model. This application's embodiments divide the power inspection image sample set into a significantly small number of labeled subsets and a large number of unlabeled subsets. The labeled subsets are used to fine-tune a general pre-trained teacher model. Then, based on the fine-tuned teacher model, high-quality soft labels are generated for the unlabeled subsets, providing valuable supervisory information to the unlabeled samples. Finally, the initial recognition model is trained by combining the labeled data, the unlabeled data after generating soft labels, and the target loss function. This fully utilizes the domain characteristics of power inspection data, effectively alleviating the data scarcity problem in small sample scenarios, while avoiding the impact of distribution differences caused by general dataset migration. Compared to training only with labeled samples, this method significantly improves the model's generalization ability and recognition accuracy, reduces reliance on a large number of manually labeled samples, and saves manpower and time costs.

[0023] In this embodiment, optionally, the target loss function includes a preset cross-entropy loss function and a preset KL divergence loss function; step 104 includes: inputting each power inspection image sample in the labeled subset into the initial recognition model to obtain a first predicted label corresponding to each power inspection image sample; inputting each power inspection image sample in the unlabeled subset into the initial recognition model to obtain a second predicted label corresponding to each power inspection image sample; calculating a first loss value based on each first predicted label and the real label corresponding to each first predicted label using the preset cross-entropy loss function; calculating a second loss value based on each second predicted label and the soft label corresponding to each second predicted label using the preset KL divergence loss function; calculating a target loss value based on the first loss value and the second loss value, and adjusting the model parameters of the initial recognition model when the target loss value is greater than a preset loss threshold, and calculating the target loss value again based on the initial recognition model with adjusted model parameters, until the preset condition is met, thus obtaining a defect recognition model.

[0024] In this embodiment, firstly, each power inspection image sample in the labeled subset is input into the initial recognition model. The initial recognition model can perform feature extraction and classification prediction on these power inspection image samples, thereby obtaining the first predicted label corresponding to each labeled power inspection image sample.

[0025] Each power inspection image sample in the unlabeled subset can also be input into the initial recognition model. Similarly, the initial recognition model can process these unlabeled power inspection image samples to obtain a second predicted label for each sample. Since unlabeled samples lack explicit category labeling, the labels predicted by the initial recognition model are inferences about the sample category based on the features it has learned. Soft labels will be used to measure the accuracy of the predictions later.

[0026] Next, based on each first predicted label and the corresponding true label (i.e., the original category label of the power inspection image samples in the labeled subset), a first loss value is calculated using a preset cross-entropy loss function. The cross-entropy loss function is often used to measure the degree of difference between the prediction result of the classification model and the true label. In supervised learning scenarios, it can effectively evaluate the classification performance of the model on labeled samples. The smaller the loss value, the more accurate the model's classification of labeled samples.

[0027] Simultaneously, based on each second predicted label and the corresponding soft label (the label generated by the fine-tuned teacher model for the unlabeled power inspection image samples, representing the probability distributions of different categories), a pre-defined KL divergence loss function is used to calculate the second loss value. KL divergence measures the difference between two probability distributions; here, it assesses the degree of difference between the probability distribution of the unlabeled sample categories predicted by the initial recognition model (the distribution corresponding to the second predicted labels) and the probability distribution represented by the soft labels generated by the teacher model. A smaller loss value indicates that the probability distribution predicted by the initial recognition model is closer to the distribution represented by the soft labels.

[0028] Finally, based on the first and second loss values ​​calculated earlier, the target loss value is calculated using a certain method (e.g., weighted summation). When the target loss value is greater than the preset loss threshold, it indicates that the model's current performance has not yet reached an ideal state, and the model parameters of the initial recognition model need to be adjusted. After adjusting the parameters, the labeled and unlabeled subsets are input into the parameter-adjusted initial recognition model again, and the above steps of calculating predicted labels and loss values ​​are repeated until the target loss value meets the preset conditions (e.g., less than or equal to the preset loss threshold or reaching the preset number of iterations). Training is then stopped, and the final model obtained is the defect recognition model.

[0029] This application embodiment fully utilizes the information from both labeled and unlabeled samples by combining a preset cross-entropy loss function and a preset KL divergence loss function. The preset cross-entropy loss function ensures the model's accurate classification ability under supervised information (labeled samples), while the preset KL divergence loss function mines the supervised information contained in the soft labels generated by the teacher model in unlabeled samples. This allows the model to better learn the features of defects in power line inspection images even with a small number of samples, improving the model's generalization ability and recognition accuracy, and effectively solving the problem of model training difficulties caused by the small number of labeled samples.

[0030] Optionally, in this embodiment, the preset cross-entropy loss function is as follows: ; in, This represents the first loss value. This represents the number of samples in the labeled subset. Indicates the tagged subset of the Sample images of power line inspection The true label, This represents the natural logarithm function, used to calculate the difference between the predicted and true labels. This indicates that the current initial recognition model is applied to the first element in the labeled subset. Sample images of power line inspection The first predicted label; The preset KL divergence loss function is as follows: ; in, This represents the second loss value. This represents the number of samples in the unlabeled subset. This indicates that the fine-tuned teacher model applies to the first unlabeled subset. i Sample images of power line inspection The generated soft tags, This indicates that the current initial recognition model is applied to the first unlabeled subset. i Sample images of power line inspection The second predicted label, KL divergence is used to measure the difference in output between the fine-tuned teacher model and the current initial recognition model; The target loss value is calculated based on the following method: ; in, , , This represents the current number of training steps. This is the preset number of steps.

[0031] In this embodiment, when calculating the target loss value, the weights corresponding to the first loss value and the second loss value are constantly changing. The weight corresponding to the first loss value decreases continuously as the number of training steps increases, while the weight corresponding to the second loss value increases continuously as the number of training steps increases. That is, in the early stages of model training, the focus is on learning from labeled data, while in the later stages, the focus gradually shifts to soft-labeled data. It should be noted that the preset number of steps can be determined according to actual needs.

[0032] In this embodiment, optionally, the target loss function further includes a self-distillation loss function, the initial recognition model includes an encoder, a decoder, and a detection head, and the decoder includes multiple decoder layers connected in sequence; the third loss value corresponding to the self-distillation loss function is calculated in the following manner: the power inspection image sample is input into the encoder of the initial recognition model, the encoder extracts image features from the power inspection image sample, and generates a global feature representation based on the feature extraction results; the global feature representation is input into each decoder layer, and the target feature representation is output through each decoder layer, wherein, for each decoder layer other than the first decoder layer, the input of the decoder layer includes the global feature representation and the previous one of the decoder layers. The target feature representations output by adjacent decoder layers are input into the detection head to obtain the target detection result corresponding to each decoder layer. For each decoder layer except the last decoder layer, the third loss value of the decoder layer is calculated based on the target feature representation corresponding to the decoder layer and the target feature representation corresponding to the next adjacent decoder layer through the self-distillation loss function. Accordingly, the target loss value also includes the sum of the third loss values ​​of each decoder layer. The "adjusting the model parameters of the initial recognition model" includes: for each decoder layer except the last decoder layer, the target detection result corresponding to the last decoder layer is used as a supervision signal to adjust the model parameters of each of the remaining decoder layers.

[0033] In this embodiment, the target loss function may include a self-distillation loss function in addition to the preset cross-entropy loss function and the preset KL divergence loss function. Here, the initial recognition model includes an encoder, a decoder (composed of multiple sequentially connected decoder layers), and a detection head. The third loss value corresponding to the self-distillation loss function is calculated as follows: First, power line inspection image samples are input into the encoder of the initial recognition model. The encoder's main function is to extract image features from the input power line inspection image samples. Through a series of convolution and pooling operations, representative feature information is mined from the original image, and a global feature representation is generated based on these feature extraction results. This global feature representation summarizes and integrates the features of the entire input image, providing a foundation for subsequent decoding layer processing.

[0034] Next, the global feature representation generated by the encoder is input into each decoder layer. For the first decoder layer, it processes only the global feature representation as input; while for each of the remaining decoder layers, the input includes not only the global feature representation but also the target feature representation output by the preceding adjacent decoder layer. In this way, each decoder layer can combine the processing results of previous layers, progressively refining and adjusting the features, ultimately allowing each decoder layer to output the target feature representation.

[0035] Subsequently, the target feature representations output from each decoder layer are input into the detection head. The detection head performs classification and localization operations based on the input target feature representations, thereby obtaining the target detection results for each decoder layer. These detection results reflect the model's ability to recognize targets in the image at different decoder layers.

[0036] Furthermore, for each decoder layer except the last one, a third loss value is calculated for that decoder layer based on the target feature representation corresponding to that decoder layer and the target feature representation corresponding to the next adjacent decoder layer, using a self-distillation loss function. The self-distillation loss function measures the difference between the output feature representations of adjacent decoder layers, enabling shallow decoder layers to learn the feature representation information of deep decoder layers, realizing the transfer of knowledge from deep to shallow layers, and enhancing the collaborative working ability between the layers of the model.

[0037] Then, based on the original first and second loss values, the third loss values ​​of each decoder layer are summed and incorporated into the target loss value. In this way, the target loss value comprehensively considers the supervision information of labeled samples, the soft supervision information of unlabeled samples, and the feature transfer information between layers within the model, enabling a more comprehensive evaluation of the model's performance.

[0038] When adjusting the model parameters of the initial recognition model, for each decoder layer except the last one, the target detection result corresponding to the last decoder layer is used as a supervision signal. This is because the last decoder layer can usually integrate the information from the previous layers to obtain a relatively more accurate target detection result. Using it as a supervision signal can guide other decoder layers to adjust their parameters in a more accurate direction, thereby optimizing the performance of the entire model.

[0039] In a specific embodiment, the calculation process of the third loss value is as follows: Figure 2 As shown. The decoder has three layers, designated decoder layer 1 to decoder layer 3. For each power inspection image sample, it first passes through the encoder, which extracts image features to generate a global feature representation. Then, all feature representations are input into each decoder layer of the decoder. Wherein, the first... The target features corresponding to each decoder layer are represented as follows: The corresponding target detection result is For the first layer( <3), during each forward propagation, the first Layer target feature representation Will be the next layer This provides more reliable initial values ​​for the deep decoder layers; simultaneously, it uses... Target detection results of layer 3 (deepest layer) As a monitoring signal, guiding the first The learning of each decoder layer is achieved through iterative training to form a closed-loop feedback loop. It is important to note that... Figure 2 In the diagram, the two small boxes corresponding to each decoder layer represent the same target feature representation. For clarity of internal connections, each decoder layer is shown with two small boxes representing the target feature representation.

[0040] This application's embodiments achieve knowledge transfer between different decoder layers within the model by introducing a self-distillation loss function: the reliable features output by the deep decoder layer supervise the training of the shallow decoder layer, and the output of the shallow decoder layer provides initial values ​​for the deep decoder layer. By iteratively optimizing rare class detection, the model convergence is accelerated. At the same time, the third loss value of each decoder layer is incorporated into the target loss value, and the detection result of the last decoder layer is used as a supervision signal to adjust the parameters, which can further improve the model's generalization ability and recognition accuracy.

[0041] Optionally, in this embodiment of the application, the self-distillation loss function is as follows: ; in, Used to measure the m The target feature representation corresponding to the decoder layer is the same as the first decoder layer. m +1 third loss value between the target feature representations corresponding to the decoder layers. Indicates the first m The target feature representation corresponding to each decoder layer Indicates the first m+1 decoder layer corresponding to the target feature representation This indicates blocking gradient backpropagation. The target feature is represented by a dimension of N×d, where N represents the number of queries and d represents the feature dimension.

[0042] In this embodiment, This operation, which blocks gradient backpropagation, prevents gradients from propagating back through the manipulated variables during gradient calculation and parameter updates during model training. This avoids undesirable effects on parameter updates in certain areas, ensuring the stability and correctness of model training. The embodiments in this application, by defining a self-distillation loss function, effectively utilize information from different decoder layers within the model. By measuring the differences in target feature representations between adjacent decoder layers and blocking partial gradient backpropagation, the model can better learn feature representations during training, improving its ability to extract and utilize features, and enhancing its stability and generalization performance.

[0043] Optionally, in this embodiment, the target loss function further includes a contrastive loss function; the fourth loss value corresponding to the contrastive loss function is calculated based on the following method: Based on the labeled subset and the unlabeled subset, construct power inspection image sample pairs; The fourth loss value for the power inspection image sample pair is calculated using the following comparative loss function: ; in, This represents the fourth loss value. H This indicates the number of power line inspection image sample pairs. This indicates a category indicator function, if the power inspection image samples are... If they belong to the same defect category, then If the power inspection image sample is correct If they do not belong to the same defect category, then , This indicates the power inspection image sample pair. The Euclidean distance between them is a hyperparameter representing the minimum Euclidean distance between negative sample pairs; Accordingly, the target loss value also includes the fourth loss value.

[0044] In this embodiment, the target loss function may include a contrastive loss function in addition to the preset cross-entropy loss function and the preset KL divergence loss function. In power line inspection defect detection, to address the problem of ambiguous class confidence in small samples, introducing a contrastive loss function can maximize inter-class distance and minimize intra-class distance, strengthening the ability to distinguish class boundaries and optimizing the separability of the feature space under small sample conditions. The fourth loss value corresponding to the contrastive loss function is calculated based on the following method: First, power line inspection image sample pairs are constructed based on existing labeled and unlabeled subsets. This step prepares for the subsequent calculation of the contrastive loss function. By constructing appropriate sample pairs, useful information can be extracted by fully utilizing the relationships between different samples, providing richer supervision signals for model training. The two power line inspection image samples in a sample pair can both come from labeled subsets, both from unlabeled subsets, or one from a labeled subset and the other from an unlabeled subset; no specific limitation is imposed here.

[0045] Next, the total fourth loss value for each power inspection image sample pair is calculated by comparing the loss function. Among them, Used to quantify the degree of difference between power line inspection image sample pairs in the feature space; is a hyperparameter representing the minimum Euclidean distance between negative sample pairs. It plays a limiting and regulating role in the loss calculation for dissimilar sample pairs in the formula, ensuring that the loss value reflects the differences between sample pairs within a reasonable range. In a specific embodiment, the Euclidean distance can be calculated based on the target feature representation.

[0046] Then, based on the original first and second loss values, a fourth loss value is incorporated into the target loss value. In this way, the target loss value comprehensively considers the supervision information of labeled samples, the soft supervision information of unlabeled samples, and the relationship information between sample pairs, enabling a more comprehensive evaluation of the model's performance.

[0047] This application's embodiments utilize a contrastive loss function to calculate a fourth loss value, fully leveraging the information about the relationships between sample pairs. For similar sample pairs, it encourages them to be closer together in the feature space; for dissimilar sample pairs, the hyperparameter margin controls the distance between them, enabling the model to better learn the distinguishing features between different defect categories. By maximizing inter-class distributional distance and minimizing intra-class differences, it enhances the discriminative power of small sample defect features and dynamically assigns higher weights to rare categories, thereby improving detection accuracy.

[0048] In a specific embodiment, the target loss function may also include a preset cross-entropy loss function, a preset KL divergence loss function, a self-distillation loss function, and a contrastive loss function. Accordingly, the target loss value can be calculated based on the first loss value, the second loss value, the third loss value, and the fourth loss value, and the calculation formula can be as follows: ; in, , For weight parameters, The weight coefficients for the third loss value corresponding to each decoder layer are used to adjust the contribution of the third loss value corresponding to each decoder layer to the total loss.

[0049] Finally, backpropagation is performed based on the target loss value to update the model parameters. The above forward propagation, loss calculation, and backpropagation process is repeated until the model converges, resulting in the defect identification model. The model training method combining the four loss values ​​aims to reduce annotation costs through semi-supervised learning, improve detection performance through a self-distillation training strategy, and enhance classification robustness through contrastive learning. These three methods work together to solve the problems of small sample size, high noise, and class imbalance in power line inspection scenarios, achieving efficient, accurate, and lightweight end-to-end target detection.

[0050] In this embodiment of the application, optionally, the teacher model is a model based on the Vision Transformer architecture, and the initial recognition model is a target detection model based on the DETR architecture with a lightweight convolutional neural network as the backbone network.

[0051] In this embodiment, the teacher model uses a Vision Transformer (ViT) architecture. The ViT architecture segments the image into a series of image patches, then linearly embeds these patches into a sequence, adds positional encoding, and finally inputs it into a Transformer encoder for processing. This architecture fully utilizes the Transformer's self-attention mechanism to capture long-range dependencies in the image and has good extraction capabilities for global image features. In power inspection scenarios, this can help the teacher model better understand the overall features of power equipment images, providing accurate supervisory information for subsequent fine-tuning of the initial recognition model.

[0052] The initial recognition model is based on the DETR (DEtection Transformer) architecture. DETR is an innovative architecture that applies the Transformer to object detection tasks, treating object detection as a set prediction problem. It transforms image feature maps into a fixed-size set element representation, and then uses the Transformer to predict the object's class and bounding box. This architecture simplifies the complex process of traditional object detection, avoiding manual anchor box design and enabling more direct object detection in images.

[0053] The backbone network of the initial recognition model adopts a lightweight convolutional neural network. Lightweight convolutional neural networks, while maintaining a certain feature extraction capability, have the advantages of fewer model parameters and lower computational cost. In power line inspection tasks, which require processing large amounts of image data, a lightweight backbone network can reduce the demand for computing resources, improve the model's inference speed, and enable the model to run efficiently even in resource-constrained environments (such as embedded devices).

[0054] This application combines the global feature extraction advantages of the Vision Transformer architecture with the concise and efficient target detection features of the DETR architecture, while utilizing a lightweight convolutional neural network as the backbone network to ensure the model's operational efficiency. The teacher model can provide accurate supervision information to the initial recognition model, helping the initial recognition model to better learn the feature representations of power line inspection defects. This ensures the model's detection accuracy while improving inference speed and reducing computational resource requirements.

[0055] Furthermore, as Figure 1 In terms of specific implementation, this application provides a defect identification model construction device based on a small sample of power line inspection, such as... Figure 3 As shown, the device includes: The sample set partitioning module is used to acquire a power inspection image sample set and, based on the defect label carrying status of each power inspection image sample in the power inspection image sample set, divide the power inspection image sample set into a labeled subset and an unlabeled subset, wherein the number of samples in the labeled subset is significantly smaller than the number of samples in the unlabeled subset. The model fine-tuning module is used to fine-tune the teacher model pre-trained on a general image dataset using the labeled subset, so as to obtain the fine-tuned teacher model. The tag generation module is used to output the defect category probability distribution corresponding to each power inspection image sample in the unlabeled subset through a fine-tuned teacher model, and generate soft tags for the corresponding power inspection image samples based on the defect category probability distribution. The model training module is used to train the initial recognition model based on the labeled subset, the unlabeled subset after generating soft labels, and the target loss function to obtain the defect recognition model.

[0056] Optionally, the target loss function includes a preset cross-entropy loss function and a preset KL divergence loss function; the model training module is used for: Each power inspection image sample in the labeled subset is input into the initial recognition model to obtain the first predicted label corresponding to each power inspection image sample; Each power inspection image sample in the unlabeled subset is input into the initial recognition model to obtain the second predicted label corresponding to each power inspection image sample; Based on each first predicted label and the corresponding real label, the first loss value is calculated using a preset cross-entropy loss function. Based on each second predicted label and the soft label corresponding to each second predicted label, the second loss value is calculated using a preset KL divergence loss function; Based on the first loss value and the second loss value, a target loss value is calculated. If the target loss value is greater than a preset loss threshold, the model parameters of the initial identification model are adjusted. Based on the initial identification model with adjusted model parameters, the target loss value is calculated again until the preset conditions are met, thus obtaining the defect identification model.

[0057] Optionally, the preset cross-entropy loss function is as follows: ; in, This represents the first loss value. This represents the number of samples in the labeled subset. Indicates the tagged subset of the Sample images of power line inspection The true label, This represents the natural logarithm function, used to calculate the difference between the predicted and true labels. This indicates that the current initial recognition model is applied to the first element in the labeled subset. Sample images of power line inspection The first predicted label; The preset KL divergence loss function is as follows: ; in, This represents the second loss value. This represents the number of samples in the unlabeled subset. This indicates that the fine-tuned teacher model applies to the first unlabeled subset. i Sample images of power line inspection The generated soft tags, This indicates that the current initial recognition model is applied to the first unlabeled subset. i Sample images of power line inspection The second predicted label, KL divergence is used to measure the difference in output between the fine-tuned teacher model and the current initial recognition model; The target loss value is calculated based on the following method: ; in, , , This represents the current number of training steps. This is the preset number of steps.

[0058] Optionally, the target loss function further includes a self-distillation loss function, the initial recognition model includes an encoder, a decoder, and a detection head, and the decoder includes multiple decoder layers connected in sequence; the third loss value corresponding to the self-distillation loss function is calculated based on the following method: The power inspection image samples are input into the encoder of the initial recognition model. The encoder extracts image features from the power inspection image samples and generates a global feature representation based on the feature extraction results. The global feature representation is input to each decoder layer, and the target feature representation is output through each decoder layer. For each decoder layer other than the first decoder layer, the input of the decoder layer includes the global feature representation and the target feature representation output by the previous adjacent decoder layer. The target feature representations output by each decoder layer are respectively input into the detection head to obtain the target detection result corresponding to each decoder layer; For each decoder layer except the last decoder layer, the third loss value of the decoder layer is calculated using the self-distillation loss function based on the target feature representation corresponding to the decoder layer and the target feature representation corresponding to the next adjacent decoder layer. Accordingly, the target loss value also includes the sum of the third loss values ​​of each decoder layer; The model training module is used for: For each decoder layer except the last one, the target detection result corresponding to the last decoder layer is used as a supervision signal to adjust the model parameters of each of the remaining decoder layers.

[0059] Optionally, the self-distillation loss function is as follows: ; in, Used to measure the m The target feature representation corresponding to the decoder layer is the same as the first decoder layer. m +1 third loss value between the target feature representations corresponding to the decoder layers. Indicates the first m The target feature representation corresponding to each decoder layer Indicates the first m +1 decoder layer corresponding to the target feature representation This indicates blocking gradient backpropagation. The target feature is represented by a dimension of N×d, where N represents the number of queries and d represents the feature dimension.

[0060] Optionally, the target loss function further includes a contrastive loss function; the fourth loss value corresponding to the contrastive loss function is calculated based on the following method: Based on the labeled subset and the unlabeled subset, construct power inspection image sample pairs; The fourth loss value for the power inspection image sample pair is calculated using the following comparative loss function: ; in, This represents the fourth loss value. H This indicates the number of power line inspection image sample pairs. This indicates a category indicator function, if the power inspection image samples are... If they belong to the same defect category, then If the power inspection image sample is correct If they do not belong to the same defect category, then , This indicates the power inspection image sample pair. The Euclidean distance between them is a hyperparameter representing the minimum Euclidean distance between negative sample pairs; Accordingly, the target loss value also includes the fourth loss value.

[0061] Optionally, the teacher model is a model based on the Vision Transformer architecture, and the initial recognition model is a target detection model based on the DETR architecture with a lightweight convolutional neural network as the backbone network.

[0062] It should be noted that other corresponding descriptions of the functional units involved in the defect identification model construction device based on small samples of power inspection provided in this application embodiment can be found in the following references. Figures 1 to 2 The corresponding descriptions in the method will not be repeated here.

[0063] This application also provides a computer device, which may specifically be a personal computer, a server, a network device, etc. Figure 4As shown, the computer device includes a bus, a processor, memory, and a communication interface, and may also include an input / output interface and a display device. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores location information. The network interface allows communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0064] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0065] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, having stored thereon a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0066] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0067] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0068] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0069] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0070] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for constructing a defect identification model based on a small sample of power line inspection, characterized in that, include: A power inspection image sample set is obtained, and based on the defect label carrying status of each power inspection image sample in the power inspection image sample set, the power inspection image sample set is divided into a labeled subset and an unlabeled subset, wherein the number of samples in the labeled subset is significantly smaller than the number of samples in the unlabeled subset. The teacher model pre-trained on a general image dataset is fine-tuned using the labeled subset to obtain the fine-tuned teacher model. Based on the unlabeled subset, the teacher model, after fine-tuning, outputs the probability distribution of defect categories for each power inspection image sample in the unlabeled subset, and generates soft labels for the corresponding power inspection image samples based on the probability distribution of defect categories. Based on the labeled subset, the unlabeled subset after generating soft labels, and the target loss function, the initial identification model is trained to obtain the defect identification model.

2. The method according to claim 1, characterized in that, The target loss function includes a preset cross-entropy loss function and a preset KL divergence loss function; the step of training the initial identification model based on the labeled subset, the unlabeled subset after generating soft labels, and the target loss function to obtain a defect identification model includes: Each power inspection image sample in the labeled subset is input into the initial recognition model to obtain the first predicted label corresponding to each power inspection image sample; Each power inspection image sample in the unlabeled subset is input into the initial recognition model to obtain the second predicted label corresponding to each power inspection image sample; Based on each first predicted label and the corresponding real label, the first loss value is calculated using a preset cross-entropy loss function. Based on each second predicted label and the soft label corresponding to each second predicted label, the second loss value is calculated using a preset KL divergence loss function; Based on the first loss value and the second loss value, a target loss value is calculated. If the target loss value is greater than a preset loss threshold, the model parameters of the initial identification model are adjusted. Based on the initial identification model with adjusted model parameters, the target loss value is calculated again until the preset conditions are met, thus obtaining the defect identification model.

3. The method according to claim 2, characterized in that, The preset cross-entropy loss function is as follows: ； in, This represents the first loss value. This indicates the number of samples in the labeled subset. Indicates the tagged subset of the Sample images of power line inspection The true label, This represents the natural logarithm function, used to calculate the difference between the predicted and true labels. This indicates that the current initial recognition model is applied to the first element in the labeled subset. Sample images of power line inspection The first predicted label; The preset KL divergence loss function is as follows: ； in, This represents the second loss value. This represents the number of samples in the unlabeled subset. This indicates that the fine-tuned teacher model applies to the first unlabeled subset. i Sample images of power line inspection The generated soft tags, This indicates that the current initial recognition model is applied to the first unlabeled subset. i Sample images of power line inspection The second predicted label, KL divergence is used to measure the difference in output between the fine-tuned teacher model and the current initial recognition model; The target loss value is calculated based on the following method: ； in, , , This represents the current number of training steps. This is the preset number of steps.

4. The method according to claim 2, characterized in that, The target loss function further includes a self-distillation loss function. The initial recognition model includes an encoder, a decoder, and a detection head. The decoder includes multiple decoder layers connected in sequence. The third loss value corresponding to the self-distillation loss function is calculated based on the following method: The power inspection image samples are input into the encoder of the initial recognition model. The encoder extracts image features from the power inspection image samples and generates a global feature representation based on the feature extraction results. The global feature representation is input to each decoder layer, and the target feature representation is output through each decoder layer. For each decoder layer other than the first decoder layer, the input of the decoder layer includes the global feature representation and the target feature representation output by the previous adjacent decoder layer. The target feature representations output by each decoder layer are respectively input into the detection head to obtain the target detection result corresponding to each decoder layer; For each decoder layer except the last decoder layer, the third loss value of the decoder layer is calculated using the self-distillation loss function based on the target feature representation corresponding to the decoder layer and the target feature representation corresponding to the next adjacent decoder layer. Accordingly, the target loss value also includes the sum of the third loss values ​​of each decoder layer; The adjustment of the model parameters of the initial recognition model includes: For each decoder layer except the last one, the target detection result corresponding to the last decoder layer is used as a supervision signal to adjust the model parameters of each of the remaining decoder layers.

5. The method according to claim 4, characterized in that, The self-distillation loss function is as follows: ； in, Used to measure the m The target feature representation corresponding to the decoder layer is the same as the first decoder layer. m +1 third loss value between the target feature representations corresponding to the decoder layers. Indicates the first m The target feature representation corresponding to each decoder layer Indicates the first m +1 decoder layer corresponding to the target feature representation This indicates blocking gradient backpropagation. The target feature is represented by a dimension of N×d, where N represents the number of queries and d represents the feature dimension.

6. The method according to claim 2, characterized in that, The target loss function further includes a contrastive loss function; the fourth loss value corresponding to the contrastive loss function is calculated based on the following method: Based on the labeled subset and the unlabeled subset, construct power inspection image sample pairs; The fourth loss value for the power inspection image sample pair is calculated using the following comparative loss function: ； in, This represents the fourth loss value. H This indicates the number of power line inspection image sample pairs. This indicates a category indicator function, if the power inspection image samples are... If they belong to the same defect category, then If the power inspection image sample is correct If they do not belong to the same defect category, then , This indicates the power inspection image sample pair. The Euclidean distance between them is a hyperparameter representing the minimum Euclidean distance between negative sample pairs; Accordingly, the target loss value also includes the fourth loss value.

7. The method according to claim 1, characterized in that, The teacher model is based on the Vision Transformer architecture, and the initial recognition model is a target detection model based on the DETR architecture with a lightweight convolutional neural network as the backbone network.

8. A defect identification model construction device based on small samples of power line inspection, characterized in that, include: The sample set partitioning module is used to acquire a power inspection image sample set and, based on the defect label carrying status of each power inspection image sample in the power inspection image sample set, divide the power inspection image sample set into a labeled subset and an unlabeled subset, wherein the number of samples in the labeled subset is significantly smaller than the number of samples in the unlabeled subset. The model fine-tuning module is used to fine-tune the teacher model pre-trained on a general image dataset using the labeled subset, so as to obtain the fine-tuned teacher model. The tag generation module is used to output the defect category probability distribution corresponding to each power inspection image sample in the unlabeled subset through a fine-tuned teacher model, and generate soft tags for the corresponding power inspection image samples based on the defect category probability distribution. The model training module is used to train the initial recognition model based on the labeled subset, the unlabeled subset after generating soft labels, and the target loss function to obtain the defect recognition model.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images