Power operation and inspection multi-label image recognition method and system based on adaptive graph convolution

Abstract

The application discloses a power operation and inspection multi-label image recognition method and system based on adaptive graph convolution, and belongs to the technical field of power operation and inspection.The application discloses a power operation and inspection multi-label image recognition method based on adaptive graph convolution, which processes sample images by constructing a preprocessing model, a network extraction model, a semantic attention model, a static GCN network model, a dynamic GCN model and a binary classifier, and completes multi-label image recognition in a power operation and inspection scene, and the scheme is scientific, reasonable and feasible.Further, the application utilizes the adaptive graph convolution neural network combining the static GCN network model and the dynamic GCN model to learn the correlation between multi-labels, so as to improve the accuracy of multi-label image recognition in the power operation and inspection scene, improve the power operation and inspection efficiency, promote the safe and stable operation of the power system, and facilitate the popularization and use of the multi-label image recognition method in the field of power operation and inspection.

Description

Technical Field

[0001] This invention relates to a method and system for multi-label image recognition in power operation and maintenance based on adaptive graph convolution, belonging to the field of power operation and maintenance technology. Background Technology

[0002] This invention (Publication No.: CN 111931859 ​​A) discloses a multi-label image recognition method and apparatus. The apparatus includes: a semantic attention module, used to separate the feature map output by the backbone network into features of multiple categories; and a dynamic graph convolutional network module, used to model the relationships between the multiple categories of features using a dynamic graph convolutional network, which includes a static graph and a dynamic graph, wherein the static graph is used to obtain the global correlation of the image, and the dynamic graph is used to obtain the local correlation of the image. Utilizing the above invention can improve the accuracy of image recognition and has strong independence and robustness, and can be applied to image recognition in various scenarios.

[0003] However, the above-mentioned solutions are for multi-label image recognition tasks in natural scenes, and are not for specific fields, especially the multi-label recognition solutions for power scenarios. This affects the efficient and accurate recognition of power operation and maintenance images, which is not conducive to improving the efficiency of power operation and maintenance, and hinders the promotion and use of multi-label image recognition methods in the field of power operation and maintenance. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a method for multi-label power operation and maintenance image recognition based on adaptive graph convolution. This method involves constructing a preprocessing model, a network extraction model, a semantic attention model, a static GCN network model, a dynamic GCN model, and a binary classifier to preprocess sample images, obtaining multi-label power images to be recognized. The multi-label power images are then processed to obtain image features. Content-aware processing is applied to the image features to obtain a category representation V. The category representation V is used as the feature of the input node and fed into the pre-constructed static GCN network model to obtain a category representation H. The category representation H is transformed to obtain a category representation Z. The category representation Z is then used by a pre-constructed binary classifier to predict the score of each category, thus completing the multi-label image recognition in power operation and maintenance scenarios. This method is scientific, reasonable, and feasible.

[0005] The second objective of this invention is to provide an adaptive graph convolutional neural network-based multi-label image recognition method and system for power operation and maintenance. This method utilizes an adaptive graph convolutional neural network that combines static and dynamic GCN network models to learn the correlation between multiple labels, thereby improving the accuracy of multi-label image recognition in power operation and maintenance scenarios, increasing the efficiency of power operation and maintenance, promoting the safe and stable operation of the power system, and facilitating the widespread application of multi-label image recognition methods in the field of power operation and maintenance.

[0006] The third objective of this invention is to provide a method and system for identifying multi-label defect images in complex power operation and maintenance scenarios by combining static and dynamic GCN network models. This method adaptively learns the correlation between labels based on the probability of different devices and defects co-occurring, thereby improving the accuracy of image recognition in power operation and maintenance scenarios. It solves the problems of traditional image recognition algorithms in power operation and maintenance scenarios, such as difficulty in recognizing objects of different scales and low accuracy in multi-label image recognition tasks, thus improving the efficiency of power operation and maintenance defect recognition and promoting the safe, reliable and stable operation of power systems.

[0007] The fourth objective of this invention is to provide a power multi-label image recognition system based on adaptive graph convolution. This system preprocesses sample images to obtain multi-label images of power systems by constructing a preprocessing module, a network extraction module, a semantic attention module, a static GCN network module, a dynamic GCN module, and a binary classifier. The system then processes these multi-label images to obtain image features, performs content awareness on the image features to obtain a category representation V, and transforms the category representation H to obtain a category representation Z. This allows for the prediction of the score for each category, thus enabling multi-label image recognition in power operation and maintenance scenarios. The system is scientific, reasonable, and feasible.

[0008] To achieve one of the above objectives, the first technical solution of the present invention is as follows:

[0009] A method for multi-label image recognition in power operation and maintenance based on adaptive graph convolution.

[0010] Includes the following steps:

[0011] The first step is to collect sample images from power operation and maintenance scenarios;

[0012] The sample images include power equipment and / or equipment defects and / or equipment malfunctions;

[0013] The second step involves preprocessing the sample images from the first step using a pre-built preprocessing model to obtain the multi-labeled power images to be identified.

[0014] The third step involves using a pre-built network extraction model to process the multi-labeled power images from the second step to obtain image features.

[0015] The fourth step involves using a pre-built semantic attention model to perform content awareness on the image features from the third step, resulting in a category representation V.

[0016] The fifth step involves feeding the category representation V from the fourth step as a feature of the input node into a pre-built static GCN network model to obtain the category representation H.

[0017] The sixth step involves transforming the category representation H from the fifth step using the pre-built dynamic GCN model to obtain the category representation Z.

[0018] In the seventh step, the category Z from the sixth step is used to predict the score of each category through a pre-built binary classifier, thus completing the multi-label image recognition in the power operation and maintenance scenario.

[0019] Through continuous exploration and experimentation, this invention preprocesses sample images by constructing a preprocessing model, a network extraction model, a semantic attention model, a static GCN network model, a dynamic GCN model, and a binary classifier to obtain multi-labeled power images to be identified. The multi-labeled power images are then processed to obtain image features. Content-aware processing is applied to the image features to obtain a category representation V. The category representation V is used as the feature of the input node and fed into the pre-constructed static GCN network model to obtain a category representation H. The category representation H is transformed to obtain a category representation Z. Based on the category representation Z, the score of each category is predicted, thus completing the multi-labeled image recognition in power operation and maintenance scenarios. The solution is scientific, reasonable, and feasible.

[0020] Furthermore, this invention utilizes an adaptive graph convolutional neural network that combines a static GCN network model and a dynamic GCN model to learn the correlation between multiple labels, thereby improving the accuracy of multi-label image recognition in power operation and maintenance scenarios, increasing power operation and maintenance efficiency, promoting the safe and stable operation of the power system, and facilitating the widespread application of multi-label image recognition methods in the field of power operation and maintenance.

[0021] Furthermore, this invention utilizes a combination of static and dynamic GCN network models to identify multi-label defect images in complex power operation and maintenance scenarios. By adaptively learning the correlation between labels based on the probability of different devices and defects co-occurring, it improves the accuracy of image recognition in power operation and maintenance scenarios. This solves the problems of traditional image recognition algorithms in power operation and maintenance scenarios, such as difficulty in recognizing objects of different scales and low accuracy in multi-label image recognition tasks, thereby improving the efficiency of power operation and maintenance defect identification and promoting the safe, reliable, and stable operation of the power system.

[0022] As a preferred technical measure:

[0023] In the first step, the method for acquiring the sample images is as follows:

[0024] Acquire several inspection images taken by drones or by manual means.

[0025] Each inspection image contains defects in multiple devices or multiple types of defects in the same device, forming a sample image.

[0026] As a preferred technical measure:

[0027] In the second step, the method for constructing the preprocessing model is as follows:

[0028] Step 21: Scale, crop, and rotate the sample image to obtain the corrected image;

[0029] Step 22: Label the corrected image from step 21 to obtain the label image;

[0030] Step 23: Randomly divide the labeled images from Step 22 into training set, validation set and test set to form a multi-label image of electricity to be identified.

[0031] As a preferred technical measure:

[0032] The method for constructing the network extraction model in the third step is as follows:

[0033] The Res2Net-101 network architecture replaces the 3×3 convolutional kernels in the ResNet network architecture with smaller grouped convolutions; the grouped convolutions consist of three 3×3 convolutions with a residual structure and a residual edge, thereby enabling the extraction of image features from multi-labeled images.

[0034] As a preferred technical measure:

[0035] In the fourth step, the method for obtaining the category representation V is as follows:

[0036] Step 41, calculate the activation mapping for the preset categories, the expression of which is as follows:

[0037] ;

[0038] in, For each category, an activation map is defined, where H and W represent the height and width of the activation map, respectively, and C represents the category.

[0039] Step 42: Convert the activation mapping from step 41 into a feature mapping, the expression of which is as follows:

[0040] ;

[0041] Where H, W, and D' represent the height, width, and transformed depth of the feature map, respectively;

[0042] Step 43: Convert the feature mapping from step 42 into a category representation of the content, the expression of which is as follows:

[0043] ;

[0044] in, For each class, its representation is... The weighted sum, where C represents the category and D represents the depth;

[0045] Features that are selectively aggregated and related to a predefined category c are calculated using the following formula:

[0046] ;

[0047] in, Let c be the transpose of the activation map. Let the weight be represented as the weight of the c-th activation map. The eigenvector at (i, j) in the feature map.

[0048] As a preferred technical measure:

[0049] In the fifth step, the method for obtaining the category representation H is as follows:

[0050] Represent a set of categories As an input node, it is fed into the dynamic GCN network model;

[0051] Dynamic GCN network models utilize correlation matrices and state update weight matrix To update the value of V, we obtain the category representation H, which is expressed by the following formula:

[0052] ;

[0053] in, It is a predefined correlation matrix, based on The constructed graph is a static graph;

[0054] The variable values ​​are learned during training;

[0055] The activation function represents the nonlinearity of the entire operation, and the correlation matrix is... This reflects the relationship between the characteristics of each category;

[0056] During training, the correlation matrix is ​​randomly initialized using gradient descent. and state update weights ;

[0057] And Sharing with all images makes It can capture global coarse-level dependencies.

[0058] As a preferred technical measure:

[0059] In the sixth step, the method for transforming category representation H into category representation Z is as follows:

[0060] H adaptively calculates the correlation matrix based on the input category representation. ;

[0061] Correlation matrix The output category is represented by Z after passing through the dynamic GCN network.

[0062] As a preferred technical measure:

[0063] The method for obtaining the category representation Z is as follows:

[0064] The dynamic GCN model adaptively calculates the relevance matrix based on the input category representation H. ;

[0065] Correlation matrix H is dynamically constructed based on its category;

[0066] Each category represents a different correlation matrix for H. This improves the model's representativeness and reduces the risk of overfitting from static graphs.

[0067] According to the relevant matrix The output category representation of the dynamic GCN model Its expression is as follows:

[0068]

[0069] in, The LeakyReLU activation function is used. It is the Sigmoid activation function. Update the weights for the state. It is to construct a dynamic correlation matrix The weights of the conv convolutional layer, It is represented by H and global. obtained by piecing together It is obtained by concatenating global average pooling and a single convolutional layer. Represents the global vector h g The length of D1 is also the depth of the feature representation H output by the static GCN network, and D2 represents the depth of the feature representation Z output by the dynamic GCN network.

[0070] As a preferred technical measure:

[0071] In the seventh step, the method for predicting the category score is as follows:

[0072] The final classification is performed using the category representation Z, which is expressed as follows:

[0073] ;

[0074] in, The vector elements representing the category Z contain information about alignment with the preset class.

[0075] Each vector element is fed into a binary classifier to predict the class score.

[0076] To achieve one of the above objectives, the second technical solution of the present invention is as follows:

[0077] A multi-label image recognition system for power operation and maintenance based on adaptive graph convolution.

[0078] The above-mentioned method for multi-label image recognition of power operation and maintenance based on adaptive graph convolution includes a preprocessing module, a network extraction module, a semantic attention module, a static GCN network module, a dynamic GCN module, and a binary classifier.

[0079] The preprocessing module is used to preprocess the sample images to obtain the multi-label power images to be identified;

[0080] The network extraction module is used to process multi-label power images to obtain image features;

[0081] The semantic attention module is used to perform content awareness on image features to obtain a category representation V;

[0082] The static GCN network module is used to obtain the category representation H;

[0083] The dynamic GCN module is used to transform the category representation H to obtain the category representation Z;

[0084] A binary classifier is used to predict the score of each category, completing multi-label image recognition in power operation and maintenance scenarios.

[0085] Through continuous exploration and experimentation, this invention constructs a preprocessing module, a network extraction module, a semantic attention module, a static GCN network module, a dynamic GCN module, and a binary classifier to preprocess sample images, obtaining multi-labeled power images to be identified. The multi-labeled power images are then processed to obtain image features. Content-aware processing is applied to the image features to obtain a category representation V. Finally, the category representation H is transformed to obtain a category representation Z, thereby predicting the score for each category and completing multi-labeled image recognition in power operation and maintenance scenarios. The solution is scientific, reasonable, and feasible.

[0086] Furthermore, this invention utilizes an adaptive graph convolutional neural network combining static and dynamic GCN modules to learn the correlation between multiple labels, thereby improving the accuracy of multi-label image recognition in power operation and maintenance scenarios, increasing power operation and maintenance efficiency, promoting the safe and stable operation of the power system, and facilitating the widespread application of multi-label image recognition methods in the field of power operation and maintenance.

[0087] Furthermore, this invention utilizes a combination of static and dynamic GCN network modules to identify multi-label defect images in complex power operation and maintenance scenarios. By adaptively learning the correlation between labels based on the probability of different devices and defects co-occurring, it improves the accuracy of image recognition in power operation and maintenance scenarios. This solves the problems of traditional image recognition algorithms in power operation and maintenance scenarios, such as difficulty in recognizing objects of different scales and low accuracy in multi-label image recognition tasks, thereby improving the efficiency of power operation and maintenance defect identification and promoting the safe, reliable, and stable operation of the power system.

[0088] Compared with the prior art, the present invention has the following beneficial effects:

[0089] Through continuous exploration and experimentation, this invention preprocesses sample images to obtain multi-labeled power images to be identified by constructing a preprocessing model, a network extraction model, a semantic attention model, a static GCN network model, a dynamic GCN model, and a binary classifier. The multi-labeled power images are then processed to obtain image features. Content-aware processing is applied to the image features to obtain a category representation V. This category representation V is then used as the feature input to the pre-constructed static GCN network model to obtain a category representation H. The category representation H is transformed to obtain a category representation Z. The category representation Z is then used by a pre-constructed binary classifier to predict the score of each category, thus completing the multi-labeled image recognition in power operation and maintenance scenarios. The solution is scientific, reasonable, and feasible.

[0090] Furthermore, this invention utilizes an adaptive graph convolutional neural network that combines a static GCN network model and a dynamic GCN model to learn the correlation between multiple labels, thereby improving the accuracy of multi-label image recognition in power operation and maintenance scenarios, increasing power operation and maintenance efficiency, promoting the safe and stable operation of the power system, and facilitating the widespread application of multi-label image recognition methods in the field of power operation and maintenance.

[0091] Furthermore, this invention utilizes a combination of static and dynamic GCN network models to identify multi-label defect images in complex power operation and maintenance scenarios. By adaptively learning the correlation between labels based on the probability of different devices and defects co-occurring, it improves the accuracy of image recognition in power operation and maintenance scenarios. This solves the problems of traditional image recognition algorithms in power operation and maintenance scenarios, such as difficulty in recognizing objects of different scales and low accuracy in multi-label image recognition tasks, thereby improving the efficiency of power operation and maintenance defect identification and promoting the safe, reliable, and stable operation of the power system.

[0092] Furthermore, through continuous exploration and experimentation, this invention constructs a preprocessing module, a network extraction module, a semantic attention module, a static GCN network module, a dynamic GCN module, and a binary classifier to preprocess sample images, obtaining multi-labeled power images to be identified. The multi-labeled power images are then processed to obtain image features. Content-aware processing of the image features yields a category representation V. The category representation H is then transformed to obtain a category representation Z, thereby predicting the score for each category and completing multi-labeled image recognition in power operation and maintenance scenarios. The solution is scientific, reasonable, and feasible. Attached Figure Description

[0093] Figure 1 This is an overall flowchart of the power operation and maintenance multi-label image recognition method based on adaptive graph convolution of the present invention;

[0094] Figure 2 This is a structural diagram of the Res2Net model of the present invention;

[0095] Figure 3 This is a structural diagram of the semantic attention module of the present invention;

[0096] Figure 4 This is one structure of the adaptive graph convolutional neural network module of the present invention;

[0097] Figure 5 This is a diagram illustrating the recognition results of an embodiment of the present invention;

[0098] (a) shows the defect identification results of the vacuum circuit breaker, and (b) shows the defect identification results of the main transformer. Detailed Implementation

[0099] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0100] Conversely, this invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of the invention as defined in the claims. Furthermore, to provide a better understanding of the invention, certain specific details are described in detail below. However, those skilled in the art will fully understand the invention even without these detailed descriptions.

[0101] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The term "or / and" as used herein includes any and all combinations of one or more of the associated listed items.

[0102] The first specific embodiment of the power operation and maintenance multi-label image recognition method based on adaptive graph convolution of the present invention:

[0103] A multi-label image recognition method for power operation and maintenance based on adaptive graph convolution includes the following steps:

[0104] The first step is to collect sample images from power operation and maintenance scenarios;

[0105] The sample images include power equipment and / or equipment defects and / or equipment malfunctions;

[0106] The second step involves preprocessing the sample images from the first step using a pre-built preprocessing model to obtain the multi-labeled power images to be identified.

[0107] The third step involves using a pre-built network extraction model to process the multi-labeled power images from the second step to obtain image features.

[0108] The network extraction model is Res2Net-101;

[0109] The fourth step involves processing the image features from the third step using a pre-built semantic attention model to obtain the category representation V.

[0110] The fifth step involves feeding the category representation V from the fourth step as a feature of the input node into a pre-built static GCN network model to obtain the category representation H.

[0111] The sixth step involves transforming the category representation H from the fifth step using the pre-built dynamic GCN model.

[0112] The transformation method is as follows:

[0113] The correlation matrix Ad is adaptively calculated based on the input category representation H;

[0114] The correlation matrix Ad is processed by a dynamic GCN network to output the category representation Z;

[0115] In the seventh step, the category Z from the sixth step is used to predict the score of each category through a pre-built binary classifier, thus completing the multi-label image recognition in the power operation and maintenance scenario.

[0116] This invention utilizes an adaptive graph convolutional neural network that combines static and dynamic graphs to learn the correlation between multiple labels, thereby improving the accuracy of multi-label image recognition in power operation and maintenance scenarios, increasing power operation and maintenance efficiency, and promoting the safe and stable operation of the power system.

[0117] like Figures 1-4 As shown, this is the second specific embodiment of the power operation and maintenance multi-label image recognition method based on adaptive graph convolution of the present invention:

[0118] A multi-label image recognition method for power operation and maintenance based on adaptive graph convolution includes the following:

[0119] (1) Collect sample images in power operation and maintenance scenarios, including image data of power equipment, equipment defects, equipment failures, etc.;

[0120] (2) Preprocess the dataset and randomly divide it into training set, validation set and test dataset;

[0121] (3) The multi-label power image to be identified is first processed by the Res2Net-101 network to extract image features, and the extracted features are processed by the semantic attention module to obtain the category representation V;

[0122] (4) The obtained category representation V is used as the feature of the input node and fed into the static GCN network in the adaptive graph convolutional neural network module to obtain the category representation H. Then, the dynamic GCN is introduced to transform H, and the correlation matrix A is adaptively calculated according to the input feature H. d Finally, the final category representation Z is output through the dynamic GCN network;

[0123] (5) The final category representation Z is predicted by a binary classifier to determine the score for each category.

[0124] Multi-label image recognition in the power operation and maintenance scenario is completed through the above steps (1) to (5).

[0125] A specific embodiment of the data source of this invention:

[0126] The data comes from drone inspection images and images taken by manual inspections. It mainly includes defects in power equipment. Each image may contain defects in multiple devices or multiple types of defects in the same device, i.e., multiple labels.

[0127] A specific embodiment of the data preprocessing of this invention:

[0128] Data preprocessing mainly includes simple data preprocessing processes such as scaling, cropping, and rotating, as well as manual labeling of image labels. Finally, the data is randomly divided into training, validation, and test sets.

[0129] A specific embodiment of the feature extraction module of the present invention:

[0130] The feature extraction module employs a relatively novel CNN module, Res2Net-101. This module replaces the traditional 3×3 convolutional kernels in the ResNet network structure with smaller grouped convolutions, such as... Figure 2 As shown, the module consists of three 3×3 convolutions with similar residual structures and a residual edge. This structure improves the module's ability to extract multi-scale features without increasing computational cost. The Semantic Attention (SAM) module aims to obtain a set of content-aware category representations, each of which is derived from the input feature map. This describes content related to a specific tag. For example... Figure 3 As shown, SAM first computes activation maps for specific categories. Then use them to map the transformed features. Converted into category representations of perceptible content Specifically, the representation of each class Expressed as to The weighted sum is generated as shown in the following formula. Features that are associated with a specific category c can be selectively aggregated:

[0131]

[0132] in Represented as the weight of the c-th activation map, The eigenvector at (i, j) in the feature map.

[0133] A specific embodiment of the output category representation Z of this invention:

[0134] The method for representing the output category Z is as follows:

[0135] Step (3) outputs the category representation V, which is then used by the GCN network to obtain the feature representation H. The specific implementation process of the GCN network is as follows: given a set of features As an input node, the goal of GCN is to utilize the correlation matrix. and state update weight matrix To update the value of V, the updated node H can be represented using a single-layer static GCN as follows:

[0136]

[0137] in Typically, it is a predefined correlation matrix, based on The constructed graph is a static graph. It is learned during training. The activation function represents the nonlinearity of the entire operation, and the correlation matrix is... H reflects the relationship between the features of each node and can be represented as: .

[0138] During training, the correlation matrix is ​​randomly initialized using gradient descent. and state update weights ,because It is shared for all images, so the expectation is... It can capture global coarse-level dependencies.

[0139] Subsequently, a dynamic GCN is introduced to transform H, and the correlation matrix is ​​adaptively calculated based on the input features H. This differs from static GCN, where the correlation matrix is ​​fixed and shared across all input samples after training. It is dynamically constructed based on input features. Because each sample has different... This improves the model's representativeness and reduces the risk of overfitting from static graphs. Formally, the output of dynamic GCN... It can be defined as:

[0140]

[0141] in The LeakyReLU activation function is used. It is the Sigmoid activation function. Update the weights for the state. It is to construct a dynamic correlation matrix The weights of the conv convolutional layer, It is represented by H and global. obtained by piecing together It is obtained by concatenating global average pooling and a single convolutional layer. Defined as:

[0142]

[0143] A specific embodiment of the present invention for predicting category scores:

[0144] The method for predicting category scores is as follows:

[0145] The final category represents For final classification, due to each vector Each vector is aligned with its specific class and contains rich relational information with other vectors. The class score can be predicted simply by putting each class vector into a binary classifier.

[0146] This invention utilizes a combination of static and dynamic graph convolutional neural networks to identify multi-label defect images in complex power operation and maintenance scenarios. By adaptively learning the correlation between labels based on the probability of different devices and defects co-occurring, it improves the accuracy of image recognition in power operation and maintenance scenarios. This solves the problems of traditional image recognition algorithms in power operation and maintenance scenarios, such as difficulty in recognizing objects of different scales and low accuracy in multi-label image recognition tasks, thereby improving the efficiency of power operation and maintenance defect identification and promoting the safe, reliable, and stable operation of the power system.

[0147] A specific embodiment of the present invention is as follows:

[0148] The experimental scenario involved electrical equipment within a substation, including transformers, circuit breakers, and other major equipment. The proposed multi-label image recognition method based on adaptive graph convolution was used for identification, and the results are as follows: Figure 5 As shown in the figure. In (a), zkdlq represents a vacuum circuit breaker, jytg represents an insulating bushing, and jsxs represents metal corrosion; in (b), zbyq represents a main transformer.

[0149] Therefore, this invention can realize the recognition of multi-label images in power operation and maintenance scenarios, can adaptively learn the correlation between multiple labels in the image, and can provide a high-precision and robust model for power operation and maintenance defect identification tasks.

[0150] A specific embodiment of the power operation and maintenance multi-label image recognition system based on adaptive graph convolution of the present invention:

[0151] A power operation and maintenance multi-label image recognition system based on adaptive graph convolution, employing the aforementioned power operation and maintenance multi-label image recognition method based on adaptive graph convolution, includes a preprocessing module, a network extraction module, a semantic attention module, a static GCN network module, a dynamic GCN module, and a binary classifier;

[0152] The preprocessing module is used to preprocess the sample images to obtain the multi-label power images to be identified;

[0153] The network extraction module is used to process multi-label power images to obtain image features;

[0154] The semantic attention module is used to perform content awareness on image features to obtain a category representation V;

[0155] The static GCN network module is used to obtain the category representation H;

[0156] The dynamic GCN module is used to transform the category representation H to obtain the category representation Z;

[0157] A binary classifier is used to predict the score of each category, completing multi-label image recognition in power operation and maintenance scenarios.

[0158] An embodiment of a device applying the method of the present invention:

[0159] A computer device comprising:

[0160] One or more processors;

[0161] Storage device for storing one or more programs;

[0162] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-described method for multi-label image recognition of power operation and maintenance based on adaptive graph convolution.

[0163] An embodiment of a computer medium applying the method of the present invention:

[0164] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for multi-label image recognition of power operation and maintenance based on adaptive graph convolution.

[0165] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0167] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0168] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for multi-label image recognition in power operation and maintenance based on adaptive graph convolution. Its features are, Includes the following steps: The first step is to collect sample images from power operation and maintenance scenarios; The sample images include power equipment and / or equipment defects and / or equipment malfunctions; The second step involves preprocessing the sample images from the first step using a pre-built preprocessing model to obtain the multi-labeled power images to be identified. The third step involves using a pre-built network extraction model to process the multi-labeled power images from the second step to obtain image features. The fourth step involves using a pre-built semantic attention model to perform content awareness on the image features from the third step, resulting in a category representation V. The fifth step involves feeding the category representation V from the fourth step as a feature of the input node into a pre-built static GCN network model to obtain the category representation H. The sixth step involves transforming the category representation H from the fifth step using the pre-built dynamic GCN model to obtain the category representation Z. In the seventh step, the category representation Z from the sixth step is used to predict the score of each category through a pre-built binary classifier, thus completing the multi-label image recognition in the power operation and maintenance scenario. In the sixth step, the method for transforming category representation H into category representation Z is as follows: H adaptively calculates the correlation matrix based on the input category representation. ; Correlation matrix The output category representation Z is obtained after the dynamic GCN network; The method for obtaining the category representation Z is as follows: The dynamic GCN model adaptively calculates the relevance matrix based on the input category representation H. ; Correlation matrix H is dynamically constructed based on its category; Each category indicates that H has a different correlation matrix. This improves the model's representativeness and reduces the risk of overfitting from static graphs. According to the relevant matrix The output category representation of the dynamic GCN model Its expression is as follows: in, The LeakyReLU activation function is used. It is the Sigmoid activation function. Update the weights for the state. It is to construct a dynamic correlation matrix The weights of the conv convolutional layer, It is represented by H and global. obtained by piecing together It is obtained by concatenating global average pooling and a single convolutional layer. Represents the global vector h g The length of D1 is also the depth of the feature representation H output by the static GCN network, and D2 represents the depth of the feature representation Z output by the dynamic GCN network.

2. The method for multi-label image recognition in power operation and maintenance based on adaptive graph convolution as described in claim 1, characterized in that, In the first step, the method for acquiring the sample images is as follows: Acquire several inspection images taken by drones or by manual means. Each inspection image contains defects in multiple devices or multiple types of defects in the same device, forming a sample image.

3. The method for multi-label image recognition in power operation and maintenance based on adaptive graph convolution as described in claim 1, characterized in that, In the second step, the method for constructing the preprocessing model is as follows: Step 21: Scale, crop, and rotate the sample image to obtain the corrected image; Step 22: Label the corrected image from step 21 to obtain the label image; Step 23: Randomly divide the labeled images from Step 22 into training set, validation set and test set to form a multi-label image of electricity to be identified.

4. The method for multi-label image recognition in power operation and maintenance based on adaptive graph convolution as described in claim 1, characterized in that, The method for constructing the network extraction model in the third step is as follows: The Res2Net-101 network architecture replaces the 3×3 convolutional kernels in the ResNet network architecture with smaller grouped convolutions; the grouped convolutions consist of three 3×3 convolutions with a residual structure and a residual edge, thereby enabling the extraction of image features from multi-labeled images.

5. The method for multi-label image recognition in power operation and maintenance based on adaptive graph convolution as described in claim 1, characterized in that, In the fourth step, the method for obtaining the category representation V is as follows: Step 41, calculate the activation mapping of the preset category, the expression of which is as follows: ； in, For each category, an activation map is defined, where H and W represent the height and width of the activation map, respectively, and C represents the category. Step 42: Convert the activation mapping from step 41 into a feature mapping, the expression of which is as follows: ； Where H, W, and D' represent the height, width, and transformed depth of the feature map, respectively; Step 43: Convert the feature mapping from step 42 into a category representation of the content, the expression of which is as follows: ； in, For each class, its representation is... The weighted sum, where C represents the category and D represents the depth; Features that are selectively aggregated and related to a predefined category c are calculated using the following formula: ； in, Let c be the transpose of the activation map. Represented as the weight of the c-th activation map, The eigenvector at (i, j) in the feature map.

6. The method for multi-label image recognition in power operation and maintenance based on adaptive graph convolution as described in claim 5, characterized in that, In the fifth step, the method for obtaining the category representation H is as follows: Represent a set of categories As an input node, it is fed into the dynamic GCN network model; Dynamic GCN network models utilize correlation matrices and state update weight matrix To update the value of V, we obtain the category representation H, which is expressed by the following formula: ； in, It is a predefined correlation matrix, based on The constructed graph is a static graph; The variable values ​​are learned during training; The activation function represents the nonlinearity of the entire operation, and the correlation matrix is... This reflects the relationship between the characteristics of each category; During training, the correlation matrix is ​​randomly initialized using gradient descent. and state update weights ; And Sharing with all images makes It can capture global coarse-level dependencies.

7. A method for multi-label image recognition of power operation and maintenance based on adaptive graph convolution as described in any one of claims 1-6, characterized in that, In the seventh step, the method for predicting the category score is as follows: The final classification is performed using the category representation Z, which is expressed as follows: ； in, The vector elements representing the category Z contain information about alignment with the preset class. Each vector element is fed into a binary classifier to predict the class score.

8. A multi-label image recognition system for power operation and maintenance based on adaptive graph convolution, characterized in that, The method for multi-label image recognition of power operation and maintenance based on adaptive graph convolution as described in any one of claims 1-7 includes a preprocessing module, a network extraction module, a semantic attention module, a static GCN network module, a dynamic GCN module, and a binary classifier. The preprocessing module is used to preprocess the sample images to obtain the multi-label power images to be identified; The network extraction module is used to process multi-label power images to obtain image features; The semantic attention module is used to perform content awareness on image features to obtain a category representation V; The static GCN network module is used to obtain the category representation H; The dynamic GCN module is used to transform the category representation H to obtain the category representation Z; A binary classifier is used to predict the score of each category, completing multi-label image recognition in power operation and maintenance scenarios.

Images