Electric power inspection image recognition method and device, and storage medium
By fine-tuning the basic model on the power inspection image data set, and combining data augmentation and attention mechanisms to extract and fuse multi-scale features, the problem of insufficient positive samples in power inspection is solved, and the accuracy of identification of power defects and abnormalities is improved.
Patent Information
- Application Number
- CN202411626178.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-14
- Publication Date
- 2025-03-07
AI Technical Summary
During power inspection, the image data of the positive sample is insufficient, and the image model has poor recognition accuracy for power defects and abnormalities.
Feature fusion and image recognition are achieved by using large-scale image datasets to train the basic model and fine-tune its specific layers, combining data augmentation technology and attention mechanisms, and extracting multi-scale low-level and high-level features.
On the small sample power inspection image dataset, the model's recognition accuracy of power defects and anomalies is improved, adapted to the problem of data scarcity, and captured key information in the image through multi-scale feature fusion.
Smart Images
Figure CN119580090A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of computer vision technology and power equipment technology, and in particular to a power inspection image recognition method, device, and storage medium. Background Art
[0002] Foreign research institutions and companies are also very active in the research of large model technology in power image analysis applications. In the field of computer vision, deep learning and large model technology are used for image analysis. Their research results have been published at world-class computer vision conferences, and they are actively cooperating with the industry to promote the application of technology.
[0003] Other domestic research units have also carried out a number of important studies in the field of large models and image data analysis. They focus on building a reliable and efficient power image analysis system to meet the actual needs of power inspection, fault detection, etc., thereby providing strong support for the intelligent transformation of the power industry. The joint efforts of these research units have promoted the rapid development of large models and image data analysis in China, and promoted the continuous advancement and application of technology.
[0004] With the continuous development of large model technology, future research trends will mainly focus on the following directions: (1) How to further improve the performance and robustness of the model so that it can adapt to more complex power image scenarios; (2) How to improve the small sample learning algorithm, reduce the dependence on labeled data, and improve the generalization ability of the model; (3) How to combine other technologies, such as reinforcement learning, transfer learning, etc., to further expand the scope of power image analysis applications. Summary of the invention
[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a power inspection image recognition method, device, and storage medium to solve or partially solve the problems of insufficient positive sample image data in power inspection and unsatisfactory recognition accuracy of image models for power defects and anomalies.
[0006] The purpose of the present invention can be achieved by the following technical solutions:
[0007] One aspect of the present invention provides a method for power inspection image recognition, comprising the following steps:
[0008] Use large-scale image datasets to train basic models;
[0009] Using the data-augmented power inspection image dataset, we fine-tune specific layers of the trained base model.
[0010] Obtain the power inspection image to be identified, and use the fine-tuned basic model to extract multi-scale low-level features and high-level features in a residual connection manner;
[0011] For the multi-scale low-level features and high-level features, multi-scale feature fusion is implemented based on a self-attention mechanism and a channel attention mechanism to obtain fused features;
[0012] Based on the fusion features, predicted labels are obtained through a classifier to achieve power inspection image recognition.
[0013] As a preferred technical solution, the goal is to minimize the loss function to achieve fine-tuning of a specific layer of the trained basic model, wherein the loss function is:
[0014]
[0015] In the formula, y i is the true label, f(x i ; θ) is the output of the model, θ is the parameter of the trained base model, and L(θ) is the loss function.
[0016] As a preferred technical solution, the self-attention mechanism and channel attention mechanism are respectively:
[0017]
[0018] s=σ(W2·ReLU(W1·F avg ))
[0019] Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the feature dimension, F avg It is the feature after global average pooling, W1 and W2 are learnable parameters, ReLU() is the activation function, and σ() is the activation function Sigmoid.
[0020] As a preferred technical solution, the loss function of the classifier is:
[0021]
[0022] Among them, L is the loss function, y i is the true label, p i is the predicted probability of the model and C is the number of categories.
[0023] As a preferred technical solution, pyramid pooling or feature pyramid network is used to achieve multi-scale feature fusion.
[0024] As a preferred technical solution, the low-level features include edge features and texture features, and the high-level features include shape features and object part features. The low-level features and high-level features are obtained by the following formula:
[0025]
[0026] y=F(x,{W i})+x (2)
[0027] In formula (1), f k (x, y) is the result of the convolution operation, W k is the convolution kernel, I(x,y) is the input image, m and n are the height and width of the convolution kernel, respectively, which are usually odd numbers. In formula (2), F(x,{W i}) is the result of the convolution operation, W i is a trainable convolution kernel, x is the input image, and y is the output low-level features and high-level features.
[0028] As a preferred technical solution, the data enhancement processing includes one or more of random cropping, rotation, and flipping.
[0029] As a preferred technical solution, the basic model is ResNet, VGG or EfficientNet.
[0030] As a preferred technical solution, during the basic model training and / or fine-tuning process, the model parameter adjustment is achieved by dynamically updating the learning rate.
[0031] Another aspect of the present invention provides an electronic device, comprising: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the aforementioned power inspection image recognition method.
[0032] Another aspect of the present invention provides a computer-readable storage medium, comprising one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the aforementioned power inspection image recognition method.
[0033] Compared with the prior art, the present invention has at least one of the following beneficial effects:
[0034] (1) Adaptable to scenarios with small power inspection image data sets: To address the data scarcity problem in power inspection, the present invention uses a large deep learning model for small sample learning, which can extract effective features from limited sample data.
[0035] (2) Good effect of feature fusion and multi-scale processing: The present invention combines convolution, residual connection and attention mechanism to fuse features of different scales, so as to capture the key information in the power inspection image to the greatest extent.
[0036] (3) Accurate image recognition and classification: The present invention establishes an accurate image classification model based on the extracted information. Through technologies such as transfer learning and pre-trained models, high-accuracy image classification can be achieved in small sample conditions, thereby achieving accurate recognition of defects and anomalies in power inspection images. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 Flow chart of the force inspection image recognition method in the embodiment. DETAILED DESCRIPTION
[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0039] Example 1
[0040] In order to solve the problem of defect and abnormality recognition in power inspection images, this embodiment provides a power inspection image recognition method. Figure 1 , the method mainly includes the following steps:
[0041] Step S1: transfer learning and pre-training.
[0042] The trained convolutional neural network (CNN) is used as the basic model and fine-tuned through transfer learning to adapt to the small sample dataset of power inspection.
[0043] Specifically, step S1 may include the following sub-steps:
[0044] Step S101: Select a pre-trained model, such as ResNet, VGG or EfficientNet, which has been trained on large-scale image datasets (such as ImageNet). The initial weights of the model come from these large-scale datasets, thus containing rich low-level and high-level image features.
[0045] Step S102, apply the pre-trained model to the power inspection image dataset and fine-tune the last few layers. By fine-tuning specific layers (such as fully connected layers or higher convolutional layers), the model can better adapt to the characteristics of small sample power images. The goal of fine-tuning is to minimize the target loss function, which can be achieved by using the cross entropy loss function:
[0046]
[0047] Among them, y i is the true label, f(xi ; θ) is the output of the model and θ is the model parameter.
[0048] Preferably, in step S102, data enhancement techniques (such as random cropping, rotation, flipping, etc.) are used in advance on small sample data to expand the data set to reduce the risk of model overfitting.
[0049] Step S2: feature fusion and multi-scale processing.
[0050] In order to capture the key information in power inspection images to the maximum extent, a variety of feature extraction methods are studied, including convolution, residual connection and multi-scale feature fusion technology.
[0051] Specifically, step S2 may include the following sub-steps:
[0052] Step S201, convolution feature extraction: extract multi-level features of the image through CNN, the early convolution layer extracts low-level features (such as edges, textures), and the later convolution layer extracts high-level features (such as shapes, object parts). The convolution operation can be expressed as:
[0053]
[0054] Among them, W k is the convolution kernel, and I(x,y) is the input image.
[0055] Step S202, residual connection: short-circuit connections are established between different layers through residual blocks (ResNet) to alleviate the gradient vanishing problem and improve the convergence speed and feature extraction ability of the model. The formula for residual connection is:
[0056] y=F(x,{W i})+x
[0057] Where x is the input, (x, {W i}) is the convolution operation, W i is a trainable convolution kernel.
[0058] Step S203, multi-scale feature fusion: features from different scales are fused to capture global and local information in the image. For example, pyramid pooling (SPP) or FPN (Feature Pyramid Networks) is used to achieve feature fusion of different scales. Multi-scale pooling in SPP can be expressed as: SPP =[f k1 ,f k2 ,…,f kn ], where f k1 ,f k2 ,…,f kn They represent features of different scales.
[0059] Step S3, feature fusion is realized based on the attention mechanism.
[0060] In the process of feature fusion, attention mechanism is added to enhance the model's attention to key areas and suppress the interference of irrelevant information. Commonly used attention mechanisms include self-attention mechanism (Self-Attention) and channel attention mechanism (SE module).
[0061] (1) Self-attention mechanism: Capture long-distance dependencies by calculating the autocorrelation of feature maps. The specific formula is: Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the characteristic dimension.
[0062] (2) Channel attention mechanism: By adaptively assigning weights to different channels, the response of important features is enhanced. The specific process is: s = σ(W2·ReLU(W1·F avg ), where F avg is the feature after global average pooling, W1 and W2 are learnable parameters.
[0063] Step S4: Accurate identification of defects and anomalies.
[0064] The information extracted by fusion of multiple features and attention mechanism is combined with classifiers (such as fully connected layers or Softmax classifiers) to achieve accurate identification of defects and anomalies in power inspection images.
[0065] Classification: After feature extraction and attention mechanism processing, the feature vector is input to the fully connected layer, and the classification result is finally output. The loss function of the classifier can use cross-entropy loss (Cross-Entropy Loss): Among them, y i is the true label, p i is the predicted probability of the model and C is the number of categories.
[0066] Step S5: model optimization and evaluation.
[0067] Optimize and evaluate model performance in the following ways:
[0068] Model optimization: Use Adam or SGD optimizer to adjust model parameters and dynamically update the learning rate to accelerate convergence.
[0069] Model evaluation: The recognition effect of the model on the test set is evaluated through evaluation indicators such as accuracy, precision, recall, and F1-score.
[0070] In order to more clearly demonstrate the specific implementation of the above solution, the following is an actual experimental case of power inspection image recognition to illustrate the specific steps and experimental results.
[0071] (1) Dataset Description
[0072] The power inspection image dataset used in this experiment contains two types of images: normal (Non-defect) and abnormal (Defect), 200 images in each category, a total of 400 images, and the resolution of each image is 224x224 pixels.
[0073] Category 1: 200 normal images. The images mainly contain power equipment in normal operation, such as transformers, transmission lines, disconnectors, etc., with no obvious damage or abnormalities on the surface of the equipment.
[0074] Category 2: 200 abnormal images. The images contain various abnormal conditions of the equipment, such as equipment damage, line aging, component failure, etc. These abnormal conditions are sometimes difficult to quickly detect through simple manual visual inspection.
[0075] Dataset division: The dataset is divided into a training set (320 images) and a test set (80 images) in a ratio of 8:2 to ensure the stability and generalization ability of the training model.
[0076] Due to the small size of the dataset, it is difficult to achieve good results by directly using a convolutional neural network (CNN) to train from scratch. To address this problem, transfer learning is used to fine-tune the pre-trained model to make full use of the existing large-scale image features and effectively solve the small sample problem.
[0077] (2) Detailed design of the solution.
[0078] The solution of this experiment is divided into three key steps: transfer learning and pre-trained model fine-tuning, feature fusion and multi-scale processing, and the introduction of attention mechanism. Ultimately, by integrating these technologies, high-precision recognition of defects and anomalies in power inspection images can be achieved.
[0079] 1) Transfer learning and fine-tuning of pre-trained models.
[0080] In deep learning, transfer learning is an effective way to solve the small sample problem. By using a model trained on a large-scale dataset (such as ImageNet), the general image features in the pre-trained model can be transferred to the power inspection image. This process is divided into the following steps:
[0081] Pre-trained model selection: ResNet50 was selected as the basic model in the experiment. ResNet is a convolutional neural network widely used in image classification tasks. It solves the gradient vanishing problem in deep networks by introducing residual connections. ResNet50 contains 50 convolutional layers, which can effectively extract deep features in images. The initial weights of the model come from pre-training on the ImageNet dataset, which contains more than 1 million images and 1,000 categories. The model has accumulated rich feature representation capabilities for this task.
[0082] Transfer learning strategy: For the power inspection image dataset, due to the small amount of data, directly training the entire model may lead to overfitting. Therefore, this experiment only fine-tunes the last few layers of ResNet50, including the last few convolutional layers and the fully connected layers. This can adapt to the specific needs of power inspection tasks while retaining the low-level feature extraction capabilities. The fine-tuning process uses the Adam optimizer, the learning rate is 0.0001, and the loss function selects the cross entropy loss function:
[0083]
[0084] Among them, y i is the true label, f(x i ; θ) is the output of the model and θ is the model parameter.
[0085] Data enhancement: In order to avoid model overfitting, the experiment also used a series of data enhancement techniques, including random rotation, horizontal flipping, random cropping, etc. These techniques can effectively expand the diversity of the data set and improve the generalization ability of the model.
[0086] 2) Feature fusion and multi-scale processing.
[0087] In image recognition tasks, convolutional layers at different levels can extract image features of different scales. In order to more comprehensively capture the key information in power inspection images, this experiment uses multi-scale feature fusion technology and combines residual connections to maintain feature transferability.
[0088] Convolution and residual connection: The ResNet model directly jumps and transfers features at different levels through residual connection to slow down the gradient disappearance and improve the efficiency of feature extraction. In the residual connection, the feature map not only depends on the extraction results of the deep convolution layer, but also includes the shallow features of the lower convolution layer. This method can help the model retain more detailed information when processing complex images.
[0089] y=F(x,{W i})+x
[0090] Where x is the input, (x, {W i}) is the convolution operation, W i is a trainable convolution kernel.
[0091] Multi-scale feature fusion: In order to further enhance the model's ability to capture features at different scales, the experiment used the spatial pyramid pooling (SPP) technique. SPP generates feature representations of different scales by applying pooling operations of different scales (such as 1x1, 2x2, 4x4, etc.) on the feature map. These features are then concatenated or weighted to form a global feature vector that can capture both local details and global structural information of the image. The output feature of SPP is represented as: f SPP =[f k1 ,f k2 ,…,f kn ], where f k1 ,f k2 ,…,f kn They represent features of different scales.
[0092] 3) Introduction of attention mechanism.
[0093] In order to improve the accuracy of the model in identifying anomalies and defects, the experiment introduced a channel attention mechanism, which can enhance the model's attention to key areas and suppress irrelevant or noise information by adaptively adjusting the weights of different channel features.
[0094] Channel attention mechanism (SE module): The SE module generates a global context vector by globally averaging the features of each channel, and then calculates the weight distribution of each channel through the fully connected layer. Ultimately, the model will enhance important feature channels according to the weights, so as to more accurately focus on abnormal areas in the power inspection image. The specific calculation formula of the SE module is: s = σ(W2·ReLU(W1·F avg ), where F avg is the feature after global average pooling, W1 and W2 are learnable parameters, and σ is the Sigmoid activation function used to assign weights to each channel.
[0095] 4) Final classification and identification.
[0096] After feature extraction, feature fusion and attention enhancement, the image features are input into the fully connected classification layer. The fully connected layer uses the Softmax activation function to map the features to the probability distribution of two categories (normal and abnormal). The specific formula is: where z c is category z c The score of , C is the number of categories.
[0097] (3) Experimental results
[0098] The experiment was trained for 50 rounds, using the Adam optimizer, a learning rate of 0.0001, and a batch size of 32. The final performance on the test set was as follows: the test set accuracy was 92.5%, the test set precision was 93%, the test set recall was 91%, and the F1-score was 92%.
[0099] In summary, this method first uses transfer learning and pre-training technology, takes the trained convolutional neural network as the basis, and fine-tunes it on a small sample power inspection image dataset, so that the large model can learn richer and more specific power image features; secondly, in terms of feature fusion and multi-scale processing, a variety of feature extraction methods are studied, including convolution and residual connection techniques, and the attention mechanism is used to fuse features of different scales to maximize the capture of key information in power inspection images; finally, a variety of different information is used to achieve accurate identification of defects and anomalies in power inspection images.
[0100] This method aims to address the problem of insufficient positive sample image data in power inspection and improve the recognition accuracy of image models for power defects and anomalies. In view of the data scarcity problem in power inspection, a large deep learning model is used for small sample learning, which can extract effective features from limited sample data. In terms of feature fusion and multi-scale processing, a variety of feature extraction methods are studied, such as convolution and residual connection techniques, and the attention mechanism is combined to fuse features of different scales, so as to capture the key information in the power inspection image to the greatest extent. Finally, an accurate image classification model is established based on the extracted information. Through technologies such as transfer learning and pre-trained models, high-accuracy image classification can be achieved in small sample conditions, and accurate recognition of defects and anomalies in power inspection images can be achieved.
[0101] Example 2
[0102] This embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the power inspection image recognition method as described in Example 1.
[0103] Example 3
[0104] This embodiment provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the power inspection image recognition method as described in Suitable for Gift 1.
[0105] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A method for recognizing power inspection images, characterized in that: The steps include: Use large-scale image datasets to train basic models; Using the data-augmented power inspection image dataset, we fine-tune specific layers of the trained base model. Obtain the power inspection image to be identified, and use the fine-tuned basic model to extract multi-scale low-level features and high-level features in a residual connection manner; For the multi-scale low-level features and high-level features, multi-scale feature fusion is implemented based on a self-attention mechanism and a channel attention mechanism to obtain fused features; Based on the fusion features, predicted labels are obtained through a classifier to achieve power inspection image recognition.
2. A method for electric power inspection image recognition according to claim 1, characterized in that: Fine-tune a specific layer of the trained base model with the goal of minimizing the loss function: In the formula, y i is the true label, f(x i ; θ) is the output of the model, θ is the parameter of the trained base model, and L(θ) is the loss function.
3. The method for power inspection image recognition according to claim 1, characterized in that: The self-attention mechanism and channel attention mechanism are: s=σ(W2·ReLU(W1·F avg )) Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the feature dimension, F avg It is the feature after global average pooling, W1 and W2 are learnable parameters, ReLU() is the activation function, and σ() is the activation function Sigmoid.
4. The method for electric power inspection image recognition according to claim 1, characterized in that: The loss function of the classifier is: Among them, L is the loss function, y i is the true label, p i is the predicted probability of the model and C is the number of categories.
5. The method for electric power inspection image recognition according to claim 1, characterized in that: Multi-scale feature fusion is achieved using pyramid pooling or feature pyramid network.
6. The method for electric power inspection image recognition according to claim 1, characterized in that: The low-level features include edge features and texture features, and the high-level features include shape features and object part features. The low-level features and high-level features are obtained by the following formula: y=F(x,{W i })+x (2) In formula (1), f k (x, y) is the result of the convolution operation, W k is the convolution kernel, I(x,y) is the input image, m and n are the height and width of the convolution kernel, respectively, which are usually odd numbers. In formula (2), F(x,{W i }) is the result of the convolution operation, W i is a trainable convolution kernel, x is the input image, and y is the output low-level features and high-level features.
7. The method for recognizing power inspection images according to claim 1, characterized in that: The data enhancement processing includes one or more of random cropping, rotation, and flipping, and the basic model is ResNet, VGG or EfficientNet.
8. The method for recognizing power inspection images according to claim 1, characterized in that: During basic model training and / or fine-tuning, model parameter adjustment is achieved by dynamically updating the learning rate.
9. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the power inspection image recognition method as described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that: It comprises one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs comprise instructions for executing the power inspection image recognition method as claimed in any one of claims 1 to 8.