Small sample electric power inspection image classification method and system based on multi-scale fusion
Through multi-scale fusion and adaptive weighting strategies, the problem of overfitting deep learning models in power inspection is solved, and the identification ability and classification accuracy of power equipment defects are improved, which is suitable for inspection of diverse power equipment.
Patent Information
- Application Number
- CN202411248288.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-06
- Publication Date
- 2025-01-03
AI Technical Summary
Deep learning models are prone to overfitting problems during power inspections, resulting in reduced detection accuracy, especially in the case of limited samples and diverse power equipment.
Using a small sample power inspection image classification method based on multi-scale fusion and adaptive weights, the backbone network is extracted through ResNet-12 feature, combined with pyramid strategy and convolutional attention module, the feature weight is dynamically adjusted, and feature fusion and weight allocation are optimized. Reduce overfitting and improve model generalization capabilities.
Effectively utilize limited sample feature information, improve the model's ability to identify power equipment defects, enhance classification accuracy, reduce dependence on large-scale data sets, and improve detection performance on diverse power equipment.
Smart Images
Figure CN119251549A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of electric power inspection, and in particular to a small sample electric power inspection classification method based on multi-scale fusion and adaptive weighting. Background Art
[0002] Power line inspections are a critical task in the power industry, ensuring the stable operation of power transmission lines and related equipment. Traditional manual inspections are limited by efficiency, cost, and accuracy, while drone inspections have become the preferred method due to their efficiency, flexibility, and low cost. Computer vision technology, particularly deep learning, plays a key role in drone inspections, especially in object detection and defect identification. However, faced with limited training samples and the diversity of power equipment, deep learning models such as convolutional neural networks (CNNs) are prone to overfitting, affecting detection accuracy. Summary of the Invention
[0003] In view of the limited training samples and the diversity of power equipment, deep learning models such as convolutional neural networks (CNNs) are prone to overfitting, which affects detection accuracy. This paper proposes a small-sample power inspection image classification method based on multi-scale fusion and adaptive weighting. The method includes: S1: Collect different types of power inspection images and pre-process the power inspection images to obtain a power inspection equipment defect image dataset; S2: Construct an image classification model based on multi-scale fusion and adaptive weights, input the power inspection equipment defect image dataset into the image classification model based on multi-scale fusion and adaptive weights for training, and obtain the optimal small sample classification model; S3: Input the training set into the optimal small sample classification model for training, and after training, select the parameter model with the smallest loss function during the training process; S4: Input the test set into the trained parameter model for small sample classification to obtain the classification results of the defect images of the power inspection equipment.
[0004] Furthermore, a preferred embodiment is proposed, wherein step S1 includes: The sizes of different categories of power inspection images are unified into 84×84 pixels, and a power inspection equipment defect image dataset is obtained. The images in the power inspection equipment defect image dataset are divided into a training set, a validation set, and a test set with non-overlapping categories according to the ratio of 4:1:1.
[0005] Furthermore, a preferred embodiment is proposed, wherein step S2 includes: S21, using ResNet-12 as the feature extraction backbone network, removing the fully connected layer related to the classification task; S22, feature extraction includes the second residual block conv2 and the fourth residual block conv4; S23, using the pyramid strategy to fuse the shallow conv2 and deep conv4 features, and adjust the conv4 features to the same size as conv2 through deconvolution; S24, generate feature vectors through global average pooling and global maximum pooling, and obtain channel attention through a multi-layer perceptron with shared weights; S25, perform global pooling operation on the channel dimension, and generate a spatial attention vector through convolution operation and Sigmoid activation; S26. Introduce a local multi-path adaptive weighting strategy to update and optimize the weights in real time during the training process to obtain the optimal small sample classification model.
[0006] Furthermore, a preferred embodiment is proposed, wherein step S23 includes:
[0007]
[0008]
[0009] in, is the residual operation of the a-th residual block, a=1, 2, 3, 4; The deconvolution operation is performed with a kernel size of 3, a kernel number of 128, a stride of 4, and a padding of 1. is the input of the embedding module; is the output feature of the second residual block, and the feature map size is 21×21×128; is the output feature of the fourth residual block, and the feature map size is 6×6×512; Represents the features after deconvolution operation, and the feature map size is 21×21×128.
[0010] Furthermore, a preferred embodiment is proposed, wherein step S24 includes:
[0011] in, I is the input feature of the channel attention module, is the average pooling operation, is the maximum pooling operation, is a multi-layer perceptron, is the Sigmoid activation function.
[0012] Furthermore, a preferred embodiment is proposed, wherein step S25 includes:
[0013]
[0014] in, is an element-wise multiplication operation, is the input feature The feature obtained by multiplying the channel attention is also the input feature of the spatial attention module. is a convolution operation with a convolution kernel size of 7×7, is the spatial attention vector.
[0015] Furthermore, a preferred embodiment is proposed, wherein step S26 includes:
[0016] in, To fuse the output features of the second residual block and the fourth residual block, , is the adaptive feature weight, It is an element-by-element addition operation.
[0017] Based on the same inventive concept, the present invention also proposes a small sample power inspection image classification system based on multi-scale fusion and adaptive weighting, the system comprising: An image acquisition unit is used to acquire different types of power inspection images and pre-process the power inspection images to obtain a data set of defect images of power inspection equipment; An optimal small sample classification model acquisition unit is used to construct an image classification model based on multi-scale fusion and adaptive weights, input the power inspection equipment defect image dataset into the image classification model based on multi-scale fusion and adaptive weights for training, and obtain the optimal small sample classification model; The training unit is used to input the training set into the optimal small sample classification model for training, and after training, select the parameter model with the smallest loss function during the training process; The classification unit is used to input the test set into the trained parameter model for small sample classification to obtain the classification results of the defect images of the power inspection equipment.
[0018] Based on the same inventive concept, the present invention also proposes a computer device, including a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a small sample power inspection image classification method based on multi-scale fusion and adaptive weights according to any one of the above.
[0019] Based on the same inventive concept, the present invention also proposes a computer-readable storage medium, which stores a computer program. When the computer program is run by a processor, it executes the steps of a small sample power inspection image classification method based on multi-scale fusion and adaptive weights as described in any of the above items.
[0020] The present invention is beneficial in that: In the inspection of power equipment, it is difficult to obtain enough sample images. Limited training samples can easily lead to overfitting of deep learning models, that is, the model performs well on the training data, but performs poorly on unseen data. In the small-sample power inspection image classification method based on multi-scale fusion and adaptive weights proposed in the present invention, multi-scale feature fusion and adaptive weight strategies are introduced, which can effectively utilize the feature information of limited samples and improve the generalization ability of the model by integrating multi-scale features. Specifically, multi-scale feature fusion can extract information from different levels, while the adaptive weight strategy can dynamically adjust the importance of different features to alleviate the overfitting problem.
[0021] Power equipment is diverse, with varying defect types and manifestations. This diversity makes it difficult to fully capture the full range of defect characteristics using features at a single scale. The method proposed in this paper utilizes multi-scale fusion technology to allow the model to extract features from different levels and scales, thereby more comprehensively capturing the full range of power equipment defects. This allows the model to better handle the diversity of power equipment and improve classification accuracy.
[0022] During feature fusion, features at different levels may have varying importance for the classification task. Improper weight distribution during feature fusion can lead to decreased model performance. The method proposed in this paper employs an adaptive weighting strategy, enabling the model to adjust the weights of each feature branch in real time during training, thereby better focusing on the features most important for the task at hand. This approach can dynamically optimize the weight distribution for feature fusion and improve model performance.
[0023] The present invention proposes a small sample power inspection image classification method based on multi-scale fusion and adaptive weighting. Through multi-scale feature fusion, the model can extract rich feature information from multiple levels and enhance the ability to identify equipment defects. In small sample scenarios, this multi-level information fusion can effectively reduce the overfitting phenomenon of the model, thereby improving the classification accuracy. The adaptive weight strategy can dynamically adjust the feature weights according to the feedback during the training process, so that the model can flexibly adapt when facing power equipment of different types and under different conditions, thereby improving its generalization ability in practical applications. Taking into account the diversity and defect characteristics of the equipment, through multi-scale fusion and adaptive weighting, the model can more comprehensively capture the key information in the image, thereby improving the classification accuracy of defects in power inspection equipment. In small sample scenarios, by optimizing the feature extraction and feature fusion methods, it is possible to make more full use of limited training data and reduce dependence on large-scale data sets.
[0024] The present invention is applied to the field of drone inspection. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 This is a flow chart of a small sample power inspection image classification method based on multi-scale fusion and adaptive weighting according to the first embodiment; Figure 2 This is a structural diagram of the image classification model based on multi-scale fusion and adaptive weights according to the eleventh embodiment; Figure 3 This is a structural diagram of the multi-scale fusion and adaptive weight feature extraction network according to embodiment 11, wherein Residule Block 64 filters is a residual block containing 64 filters, Residule Block 128 filters is a residual block containing 128 filters, Residule Block 256 filters is a residual block containing 256 filters, and Residule Block 512 filters is a residual block containing 512 filters; Figure 4 This is a structural diagram of each residual block of the backbone network ResNet-12 in the feature extraction network described in embodiment 11; Figure 5 This is a structural diagram of the convolutional attention mechanism described in embodiment 11; Figure 6 This is a structural diagram of the channel attention in the convolutional attention mechanism described in embodiment 11; Figure 7 This is a structural diagram of the spatial attention in the convolutional attention mechanism described in embodiment 11. DETAILED DESCRIPTION
[0026] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in combination with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.
[0027] Implementation method 1, see Figure 1 This embodiment describes a small sample power inspection image classification method based on multi-scale fusion and adaptive weighting, the method comprising: S1: Collect different types of power inspection images and pre-process the power inspection images to obtain a power inspection equipment defect image dataset; S2: Construct an image classification model based on multi-scale fusion and adaptive weights, input the power inspection equipment defect image dataset into the image classification model based on multi-scale fusion and adaptive weights for training, and obtain the optimal small sample classification model; S3: Input the training set into the optimal small sample classification model for training, and after training, select the parameter model with the smallest loss function during the training process; S4: Input the test set into the trained parameter model for small sample classification to obtain the classification results of the defect images of the power inspection equipment.
[0028] In this embodiment, drones and identification devices are used to inspect photovoltaic power stations and collect defect images of different types of power inspection equipment.
[0029] During power equipment inspections, it is difficult to obtain enough sample images. Limited training samples can easily lead to overfitting of deep learning models, meaning that the model performs well on the training data but poorly on unseen data. The method proposed in this embodiment introduces multi-scale feature fusion and an adaptive weighting strategy, which can effectively utilize the feature information of limited samples and improve the generalization ability of the model by integrating multi-scale features. Specifically, multi-scale feature fusion can extract information from different levels, while the adaptive weighting strategy can dynamically adjust the importance of different features to alleviate the overfitting problem.
[0030] Power equipment is diverse, with varying defect types and manifestations. This diversity makes it difficult to fully capture the full range of defect characteristics using features at a single scale. The method proposed in this embodiment utilizes multi-scale fusion technology to allow the model to extract features from different levels and scales, thereby more comprehensively capturing the full range of power equipment defects. This allows the model to better handle the diversity of power equipment and improve classification accuracy.
[0031] During feature fusion, features at different levels may have varying importance for the classification task. Improper weight distribution during feature fusion can lead to decreased model performance. This implementation utilizes an adaptive weighting strategy, enabling the model to adjust the weights of each feature branch in real time during training, thereby better focusing on the features most important for the task at hand. This approach dynamically optimizes feature fusion weight distribution and improves model performance.
[0032] The method proposed in this embodiment, through multi-scale feature fusion, the model can extract rich feature information from multiple levels and enhance the ability to identify equipment defects. In small sample scenarios, this multi-level information fusion can effectively reduce the overfitting phenomenon of the model, thereby improving classification accuracy. The adaptive weight strategy can dynamically adjust the feature weights according to the feedback during the training process, so that the model can flexibly adapt when facing power equipment of different types and under different conditions, thereby improving its generalization ability in practical applications. Taking into account the diversity and defect characteristics of the equipment, through multi-scale fusion and adaptive weighting, the model can more comprehensively capture the key information in the image, thereby improving the classification accuracy of defects in power inspection equipment. In small sample scenarios, by optimizing the feature extraction and feature fusion methods, it is possible to make more full use of limited training data and reduce dependence on large-scale data sets.
[0033] The image classification model based on multi-scale fusion and adaptive weighting (MSAW-Net) described in this implementation uses multi-scale fusion technology to capture different features in an image and optimizes the model's processing of these features using an adaptive weighting strategy to improve classification performance with limited samples. The introduction of MSAW-Net provides new insights for the intelligent follow-up of power inspections, enabling rapid and accurate identification of power equipment and potential defects in resource-limited environments, thus providing strong support for the safe and stable operation of power systems.
[0034] Implementation 2: This implementation further limits the small sample power inspection image classification method based on multi-scale fusion and adaptive weighting described in Implementation 1. Step S1 includes: The sizes of different categories of power inspection images are unified into 84×84 pixels, and a power inspection equipment defect image dataset is obtained. The images in the power inspection equipment defect image dataset are divided into a training set, a validation set, and a test set with non-overlapping categories according to the ratio of 4:1:1.
[0035] This implementation standardizes the size of different types of power inspection images into 84×84 pixel images. This design ensures consistency and stability when processing input data, avoiding issues such as increased preprocessing complexity or decreased model performance due to varying image sizes. Furthermore, the 84×84 pixel image size requires relatively few computing resources (such as memory and computing time) to process each image, helping to accelerate model training and inference. By setting the image size to 84×84, it ensures that subsequent algorithms and models involved in the comparison are tested under the same conditions, resulting in more objective and reliable experimental results.
[0036] In this implementation, the images are divided into a training set, a validation set, and a test set with mutually exclusive categories in a 4:1:1 ratio. This ensures that the majority of the data (i.e., 4 / 6) is used to train the model, fully learning the data's characteristics and patterns. At the same time, sufficient data (i.e., 1 / 6 for validation and 1 / 6 for testing) is retained to evaluate the model's performance and generalization capabilities.
[0037] There are a total of 18 categories in this embodiment, including 12 training sets, including: insulator defects, hardware defects, tower defects, conductor defects, cable defects, transformer defects, switchgear defects, capacitor defects, lightning arrester defects, instrument defects, auxiliary equipment defects, and bird nests; 3 validation sets, including: missing or blurred signboards, poor contact, and line icing; and 3 test sets, including: external force damage, oil leakage, and busbar deformation.
[0038] Implementation method 3: This implementation method further limits the small sample power inspection image classification method based on multi-scale fusion and adaptive weighting described in implementation method 1. Step S2 includes: S21. Use ResNet-12 as the feature extraction backbone network and remove the fully connected layer that is highly relevant to the classification task; S22, feature extraction includes the second residual block conv2 and the fourth residual block conv4; S23, using the pyramid strategy to fuse the shallow conv2 and deep conv4 features, and adjust the conv4 features to the same size as conv2 through deconvolution; S24, generate feature vectors through global average pooling and global maximum pooling, and obtain channel attention through a multi-layer perceptron with shared weights; S25, perform global pooling operation on the channel dimension, and generate a spatial attention vector through convolution operation and Sigmoid activation; S26. Introduce a local multi-path adaptive weighting strategy to update and optimize the weights in real time during the training process to obtain the optimal small sample classification model.
[0039] In this embodiment, ResNet-12 is used as the feature extraction network, and the fully connected layer is removed, which helps to reduce the complexity of the model. Specifically, in the overall structure of ResNet-12, it includes residual blocks and fully connected layers. The residual block is responsible for mapping the input image to the feature space and converting it into a feature vector. The fully connected layer is the last layer of the network and is used to convert the features into classification results. Since the output of the fully connected layer directly corresponds to the category prediction of the classification task, it is highly relevant to the classification task. After removing the fully connected layer, the model is equivalent to a feature extractor, and the feature vector it outputs can be used as the input of subsequent tasks.
[0040] Furthermore, by fusing shallow (conv2) and deep (conv4) features through the pyramid strategy, we can better capture information of different scales, and adjust the deep features to the size of shallow features through deconvolution, thereby enhancing the expressive power of multi-scale features.
[0041] Furthermore, this embodiment also uses the feature vectors generated by global average pooling and global maximum pooling to obtain channel attention through a multi-layer perceptron with shared weights, automatically identifying the channel features that are most important for classification. Spatial attention vectors are generated through convolution and sigmoid activation, weighting the features of important areas, further enhancing the model's focus on key areas. The introduction of a local multi-path adaptive weighting strategy can optimize weights in real time during training, adapt to data changes, and improve the model's classification performance, especially in small sample scenarios, which can effectively improve classification accuracy.
[0042] Implementation 4: This implementation further limits the small sample power inspection image classification method based on multi-scale fusion and adaptive weighting described in Implementation 3. Step S23 includes:
[0043]
[0044]
[0045] in, is the residual operation of the a-th residual block, a=1, 2, 3, 4; The deconvolution operation is performed with a kernel size of 3, a kernel number of 128, a stride of 4, and a padding of 1. is the input of the embedding module; is the output feature of the second residual block, and the feature map size is 21×21×128; is the output feature of the fourth residual block, and the feature map size is 6×6×512; Represents the features after deconvolution operation, and the feature map size is 21×21×128.
[0046] This embodiment is explained in conjunction with the second embodiment. Specifically, the 84×84 training set picture is first input into the ResNe-t12 backbone network according to the episode training strategy, and then the output features conv2 and conv4 of the second and fourth residual blocks are extracted respectively. Then, conv4 is deconvolved to achieve the same feature map size as conv2.
[0047] Implementation 5: This implementation further limits the small sample power inspection image classification method based on multi-scale fusion and adaptive weighting described in Implementation 3. Step S24 includes:
[0048] in, I is the input feature of the channel attention module, is the average pooling operation, is the maximum pooling operation, is a multi-layer perceptron, is the Sigmoid activation function.
[0049] Implementation 6: This implementation further limits the small sample power inspection image classification method based on multi-scale fusion and adaptive weighting described in Implementation 5. Step S25 includes:
[0050]
[0051] in, is an element-wise multiplication operation, is the input feature The feature obtained by multiplying the channel attention is also the input feature of the spatial attention module. is a convolution operation with a convolution kernel size of 7×7, is the spatial attention vector.
[0052] Implementation 7: This implementation further limits the small sample power inspection image classification method based on multi-scale fusion and adaptive weighting described in Implementation 5. Step S26 includes:
[0053] in, To fuse the output features of the second residual block and the fourth residual block, , is the adaptive feature weight, It is an element-by-element addition operation.
[0054] Embodiment 8: This embodiment describes a small sample power inspection image classification system based on multi-scale fusion and adaptive weighting, the system comprising: An image acquisition unit is used to acquire different types of power inspection images and pre-process the power inspection images to obtain a data set of defect images of power inspection equipment; An optimal small sample classification model acquisition unit is used to construct an image classification model based on multi-scale fusion and adaptive weights, input the power inspection equipment defect image dataset into the image classification model based on multi-scale fusion and adaptive weights for training, and obtain the optimal small sample classification model; The training unit is used to input the training set into the optimal small sample classification model for training, and after training, select the parameter model with the smallest loss function during the training process; The classification unit is used to input the test set into the trained parameter model for small sample classification to obtain the classification results of the defect images of the power inspection equipment.
[0055] Implementation method nine: A computer device described in this implementation method includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a small sample power inspection image classification method based on multi-scale fusion and adaptive weights according to any one of implementation methods one to seven.
[0056] Implementation method ten, a computer-readable storage medium described in this implementation method is characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor, the steps of a small sample power inspection image classification method based on multi-scale fusion and adaptive weights as described in any one of implementation methods one to seven are executed.
[0057] Implementation method 11, see Figures 2 to 7 This embodiment provides a specific example of the small sample power inspection image classification method based on multi-scale fusion and adaptive weighting described in embodiment 1, and is also used to explain embodiments 2 to 7. Specifically: Step 1: Use drones or manual photography to capture different types of power inspection images and perform simple preprocessing on the acquired images. The acquired power inspection equipment defect image dataset should contain a variety of different defect types: a. Collect data: Use drones and identification devices to inspect photovoltaic power plants and capture images of defects in different types of power inspection equipment; b. Data preprocessing All images in the data were resized to 84×84 to obtain a dataset of defect images of power inspection equipment. The images in the dataset were divided into a training set, a validation set, and a test set with a 4:1:1 ratio, each with its own distinct categories. There were 18 categories in total, including 12 in the training set, including insulator defects, hardware defects, tower defects, conductor defects, cable defects, transformer defects, switchgear defects, capacitor defects, lightning arrester defects, instrument defects, accessory equipment defects, and bird nests; three in the validation set, including missing or blurred signage, poor contact, and line icing; and three in the test set, including external damage, oil leakage, and busbar deformation.
[0058] Step 2: Construct an image classification model based on multi-scale fusion and adaptive weights, and use the collected pre-processed dataset to train the image classification model based on multi-scale fusion and adaptive weights to obtain the optimal small sample classification model. The specific steps are as follows: a. Based on the classic small sample classification model - relational network, a multi-scale adaptive weight embedding module is proposed. The network structure is as follows Figure 3 As shown, this module uses ResNet-12 (removing the fully connected layer that is highly relevant to the classification task) as the backbone network for feature extraction. The structure of each residual block of ResNet-12 is shown in Figure 4 As shown. Using the pyramid strategy feature fusion technology, the output results conv2 and conv4 of the second and fourth residual blocks of ResNet-12 are extracted as experimental objects, and the shallow position information is fused with the deep semantic information from top to bottom to prevent the neglect of important position and detail information. First, the 84×84 training set images are input into the ResNet-12 backbone network according to the episode training strategy, and then the output features conv2 and conv4 of the second and fourth residual blocks are extracted respectively. Then, conv4 is deconvolved to achieve the same feature map size as conv2. The specific operation is shown in the following formula:
[0059]
[0060]
[0061] in, is the residual operation of the a-th residual block; The deconvolution operation is performed with a kernel size of 3, a kernel number of 128, a stride of 4, and a padding of 1. is the input of the embedding module; is the output feature of the second residual block, and the feature map size is 21×21×128; is the output feature of the fourth residual block, and the feature map size is 6×6×512; Represents the features after deconvolution operation, and the feature map size is 21×21×128.
[0062] b. In order to make the model focus on the parts that are more relevant to the current task, the convolutional attention module CBAM is introduced to enhance the representation ability of features. The structural diagram of CBAM is as follows Figure 5 As shown in Figure 2, it includes a channel attention module (CAM) and a spatial attention module (SAM). The structure of the channel attention module is shown in Figure 2. Figure 6 As shown, the input feature map I After global average pooling and global maximum pooling, one-dimensional feature vectors are obtained respectively. The two feature vectors pass through a weight-sharing multi-layer perceptron, and then the weights are added. Finally, the channel attention mechanism is obtained through the Sigmoid activation function. , the detailed calculation formula is as follows:
[0063] in, I is the input feature of the channel attention module, and are average pooling operation and maximum pooling operation respectively, is a multi-layer perceptron, is the Sigmoid activation function.
[0064] The structure of the spatial attention module is as follows Figure 7 As shown, the input feature map After global average pooling and global maximum pooling in the channel dimension, the size and The two feature maps are the same, with the number of channels being 1. Then the two feature maps are concatenated and convolved using a 7×7 convolution kernel. Finally, the spatial attention vector is obtained by the Sigmoid activation function. , the detailed calculation formula is as follows:
[0065]
[0066] in, is an element-wise multiplication operation, is the input feature The feature obtained by multiplying the channel attention is also the input feature of the spatial attention module. It is a convolution operation with a convolution kernel size of 7×7.
[0067] The final output features of the convolutional attention module The calculation formula is:
[0068] c. To address the limitation that the weights of features in different fusion layers are the same and no appropriate weights are assigned, a local multi-path adaptive weighting strategy is introduced to achieve real-time update and optimization of weights during training, so as to dynamically adjust the weights of each branch path. The calculation formula is as follows:
[0069] in, To fuse the output features of the second residual block and the fourth residual block, , is the adaptive feature weight, It is an element-by-element addition operation.
[0070] d. The loss function uses the mean-square error function (MSE), which can be expressed as:
[0071] in, To support images With query image The relationship score between the two images calculated by the model is in the range of (0, 1) under the Sigmoid activation function; , Indicates the labels of the support image and the query image. When the two belong to the same category, ; When the two belong to different categories, .
[0072] Step 3: Input the training set into the small sample classification model constructed in step 2 for training. After the training is completed, select the model with the smallest loss function during the training process; Step 4: Input the test set into the trained parameter model for small sample classification to obtain the classification results of the power inspection equipment defect images.
[0073] The method proposed in this embodiment has the following technical effects: (1) In this embodiment, feature fusion technology is used to fuse shallow position information with deep semantic information to prevent important position and detail information from being overlooked.
[0074] (2) In this embodiment, a local multi-path adaptive weighting strategy is used to achieve real-time updating and optimization of weights during the training process, so as to dynamically adjust the weight of each branch path.
[0075] (3) In this embodiment, the convolutional attention module CBAM is used to make the model focus on the part that is more relevant to the current task, that is, the defect part.
[0076] Although the preferred embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present disclosure.
[0077] Obviously, those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.
[0078] Those skilled in the art will appreciate that embodiments of the present disclosure may be provided as methods, systems, or computer program products. Thus, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.
[0079] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure and are not intended to limit its scope of protection. Although the present disclosure has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that after reading the present disclosure, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the invention, but these changes, modifications or equivalent substitutions are all within the scope of protection of the disclosed claims.
Claims
1. A small sample power inspection image classification method based on multi-scale fusion and adaptive weight, characterized in that: The method comprises: S1: collecting different types of power inspection images, and preprocessing the power inspection images to obtain a power inspection equipment defect image dataset; S2: construct an image classification model based on multi-scale fusion and adaptive weights, input the defect image data set of power inspection equipment into the image classification model based on multi-scale fusion and adaptive weights for training, and obtain the optimal small sample classification model; S3: Input the training set into the optimal small sample classification model for training, and after training, select the parameter model with the smallest loss function during the training process; S4: Input the test set into the trained parameter model for small sample classification to obtain the classification results of the defect images of the power inspection equipment.
2. According to claim 1, a small sample power inspection image classification method based on multi-scale fusion and adaptive weighting is characterized in that: The step S1 comprises: The sizes of different categories of power inspection images are unified into 84×84 pixels, and a data set of defect images of power inspection equipment is obtained. The images in the data set of defect images of power inspection equipment are divided into training set, validation set and test set with non-overlapping categories according to the ratio of 4:1:
1.
3. According to claim 1, a small sample power inspection image classification method based on multi-scale fusion and adaptive weighting is characterized in that: The step S2 comprises: S21, using ResNet-12 as the feature extraction backbone network, removing the fully connected layer related to the classification task; S22, feature extraction includes the second residual block conv2 and the fourth residual block conv4; S23, using the pyramid strategy to fuse the shallow conv2 and deep conv4 features, and adjusting the conv4 features to the same size as conv2 through deconvolution; S24, generating feature vectors through global average pooling and global maximum pooling, and obtaining channel attention through a multi-layer perceptron with shared weights; S25, perform global pooling operation on the channel dimension, and generate a spatial attention vector through convolution operation and Sigmoid activation; S26. Introduce a local multi-path adaptive weighting strategy to update and optimize the weights in real time during the training process to obtain the optimal small sample classification model.
4. The small sample power inspection image classification method based on multi-scale fusion and adaptive weight according to claim 3 is characterized in that: The step S23 comprises: in, is the residual operation of the a-th residual block, a=1, 2, 3, 4; is the deconvolution operation; is the input of the embedding module; is the output feature of the second residual block; is the output feature of the fourth residual block; Represents the features after deconvolution operation.
5. The small sample power inspection image classification method based on multi-scale fusion and adaptive weight according to claim 3 is characterized in that: The step S24 comprises: in, I is the input feature of the channel attention module, is the average pooling operation, is the maximum pooling operation, is a multi-layer perceptron, is the Sigmoid activation function.
6. The small sample power inspection image classification method based on multi-scale fusion and adaptive weight according to claim 5 is characterized in that: The step S25 comprises: in, is an element-by-element multiplication operation, For input features The feature obtained by multiplying with the channel attention, is a convolution operation with a convolution kernel size of 7×7. is the spatial attention vector.
7. The small sample power inspection image classification method based on multi-scale fusion and adaptive weight according to claim 5 is characterized in that: The step S26 comprises: in, To fuse the output features of the second residual block and the fourth residual block, , is the adaptive feature weight, It is an element-by-element addition operation.
8. A small sample power inspection image classification system based on multi-scale fusion and adaptive weight, characterized in that: The system comprises: An image acquisition unit, used to acquire different types of power inspection images, and pre-process the power inspection images to obtain a data set of defect images of power inspection equipment; An optimal small sample classification model acquisition unit is used to construct an image classification model based on multi-scale fusion and adaptive weights, input the defect image data set of the power inspection equipment into the image classification model based on multi-scale fusion and adaptive weights for training, and obtain an optimal small sample classification model; A training unit is used to input the training set into the optimal small sample classification model for training, and after training, select the parameter model with the smallest loss function during the training process; The classification unit is used to input the test set into the trained parameter model for small sample classification to obtain the classification results of the defect images of the power inspection equipment.
9. A computer device, characterized in that: It includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes a small sample power inspection image classification method based on multi-scale fusion and adaptive weights according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of a small sample power inspection image classification method based on multi-scale fusion and adaptive weights as described in any one of claims 1 to 7.