A method and device for power image processing based on complete learning

Abstract

The present invention discloses a method and device for processing electric power images based on complete learning. The method uses a feature map Fout and a calibration value of the feature map Fout as a training set, and uses the training set to detect anomalies in the electric power image. The electric power image is input into the trained electric power image anomaly detection model, and the prediction result of anomalies of electric power equipment in the electric power image is output. The advantages of local feature extraction of the convolution process and the advantages of global feature extraction of the self-attention calculation are effectively utilized to construct an efficient image feature learning method. The feature map of hybrid learning can fully represent the feature information of the original image and effectively learn the feature information in the electric power image. The present invention improves the accuracy of recognition, reduces the cost of manual inspections, automatically detects defects on transmission lines, and ensures the safe operation of the national power system.

Description

Technical Field

[0001] The present invention relates to a method and device for processing electric power images based on complete learning, belonging to the technical field of intelligent power grid operation and maintenance. Background Art

[0002] With the continuous development of the national power grid, the number of outdoor power equipment is increasing, and as a result, the maintenance requirements for circuit equipment are also increasing. To reduce the manpower consumption during maintenance, the use of drones to automatically patrol and capture images and automatically detect outliers in these images has become the future development direction of power operation and maintenance.

[0003] In recent years, with the continuous development of machine learning and deep learning, deep learning methods have been applied across various industries. Convolutional neural networks (CNNs) have demonstrated excellent feature learning performance in tasks such as image recognition, semantic segmentation, and object detection. Transformers were first introduced in the field of natural language processing and are primarily used in tasks such as machine translation and semantic relationship recognition.

[0004] Researchers are now applying the Transformer to computer vision and discovering its potential in image generation and super-resolution. While both approaches have achieved significant success, the convolutional and self-attention modules follow fundamentally different design paradigms. While traditional convolutional networks tend to learn local image information and obtain weighted features, the Transformer can learn global image information.

[0005] Therefore, how to combine convolutional networks with Transformers to process non-power equipment objects such as trees in power images and facilitate the identification of power equipment is a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0006] Purpose: In order to overcome the shortcomings of the existing technology, the present invention provides a power image processing method and device based on full learning, which effectively utilizes the advantages of local feature extraction of the convolution process and the advantages of global feature extraction of self-attention calculation, and constructs an efficient image feature learning method. The feature map of hybrid learning can fully represent the feature information of the original image and effectively learn the feature information in the power image.

[0007] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:

[0008] In a first aspect, a power image processing method based on complete learning comprises the following steps:

[0009] The feature map F outand feature map F out The calibration values ​​are used as training sets, and the power image anomaly detection model is built using the training sets.

[0010] The power image is input into the trained power image anomaly detection model, and the abnormality prediction results of the power equipment in the power image are output.

[0011] As a preferred solution, the characteristic graph F out It is obtained by calculation using the following formula:

[0012] F out =α·F conv +β·F tran

[0013] Among them, F conv is the local feature map, F tran is the global feature map, and α and β are parameters.

[0014] As a preferred solution, the local feature map F conv The acquisition method includes the following steps:

[0015] The original image of the power image is input into an N-layer convolutional neural network, and the output of each layer of the convolutional neural network is the input of the next layer of the convolutional neural network.

[0016] After N iterative operations, the local feature map F is obtained conv .

[0017] The convolutional neural network of each layer includes at least a convolution layer, a normalization layer, an activation layer and a pooling layer.

[0018] The operation method of each layer of the convolutional neural network includes the following steps:

[0019] The image passes through the convolution filtering operation of the convolution layer to obtain the convolution layer output.

[0020] The output of the convolutional layer is normalized through the normalization operation of the normalization layer to obtain the normalized layer output.

[0021] The output of the normalized layer is passed through the activation function of the activation layer to obtain the output of the activation layer.

[0022] The output of the activation layer is compressed through the pooling layer to obtain the output of the convolutional neural network in this layer.

[0023] As a preferred solution, the global feature map F tran The acquisition method includes the following steps:

[0024] The original image of the power image is divided into K small blocks, and the value of each small block is flattened to obtain a linear projection. The linear projection plus the position information is used as the feature information of each small block. i express.

[0025] The feature information X of each small block i Input to N layers of Transformer encoders, and the output of each layer of Transformer encoder is the input of the next layer of Transformer encoder.

[0026] After N iterations, the global feature map F is obtained. tran .

[0027] The Transformer encoder of each layer includes at least a normalization layer, a self-attention calculation layer, a residual connection layer and a multi-layer perceptron module.

[0028] The operation method of the Transformer encoder at each layer includes the following steps:

[0029] The feature information X of each small block i Input to the normalization layer for linear normalization to obtain the normalized result X i ′.

[0030] Normalized result X i ′ is input to the self-attention calculation layer, and the result X of normalizing the features of each image block is obtained i ′ and W Q , W K and W V The three weight matrices are multiplied to obtain the linear projection matrix Q, K, and V. The multi-head attention MSA is calculated based on the linear projection matrices Q, K, and V. MSA = [SA1, SA2, ..., SA k ]U msa , where SAk represents the attention value of the kth attention head, U msa Represents the transformation matrix.

[0031] Input the multi-head attention MSA into the residual connection layer, and according to the normalized result X i ′ is added to the multi-head attention MSA to obtain the output of the residual connection layer.

[0032] The output of the residual connection layer is input into the multi-layer perceptron module for learning and parameter weight discarding to obtain the output of the Transformer encoder of this layer.

[0033] As a preferred solution, the calculation formula of SAk is as follows:

[0034] Where D represents the dimension of the input and softmax(*) is the activation function.

[0035] As a preferred solution, the power image anomaly value detection model includes at least a multi-layer perceptron module and a fully connected layer.

[0036] As a preferred solution, the power equipment abnormality prediction results in the power image include: the power equipment has a problem, and the power equipment has no problem.

[0037] In a second aspect, a power image processing device based on complete learning includes the following modules:

[0038] Training module, used to transform the feature map F out and feature map F out The calibration values ​​are used as training sets, and the power image anomaly detection model is built using the training sets.

[0039] The prediction module is used to input the power image into the trained power image anomaly detection model and output the abnormality prediction results of the power equipment in the power image.

[0040] Beneficial Effects: This invention provides a fully learning-based power image processing method and device, combining the advantages of convolutional neural networks and Transformers to effectively learn and represent local and global image features. This improves recognition accuracy, reduces manual inspection costs, and automatically detects defects on transmission lines, ensuring the safe operation of the nation's power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of the method of the present invention. DETAILED DESCRIPTION

[0042] The present invention will be further described below with reference to specific embodiments.

[0043] like Figure 1 As shown, a power image processing method based on complete learning includes the following steps:

[0044] The local features of the input power equipment image are extracted through an N-layer convolutional neural network to obtain a local feature map.

[0045] The global features of the input power equipment image are extracted through an N-layer Transformer encoder to obtain a global feature map.

[0046] The local feature map and the global feature map are weighted concatenated to obtain the final image feature map.

[0047] The final image feature map is passed through the power image outlier detection model to obtain the final power equipment recognition result.

[0048] The specific implementation method is:

[0049] The original image is input into an N-layer convolutional neural network to extract local features of the image. Each layer of the convolutional neural network includes a convolution layer, a normalization layer, an activation layer, and a pooling layer. The output of each layer of the convolutional neural network is the input of the next layer of the convolutional neural network. The processing method of each layer of the convolutional neural network is as follows:

[0050] The convolutional layer is the most important computational layer of the convolutional neural network. The convolutional layer contains multiple convolution kernels, and the feature map of the image can be obtained by computing and filtering the convolution kernels. The formula for this process is expressed as: P is the original image, P∈R h×w , R h×w is the pixel matrix of length h and width w of the image, W and b represent the convolutional network parameter matrix and bias value respectively. f(*) represents the convolution filtering operation. Represents the output of the i-th convolutional layer.

[0051] After the convolutional layer, the output is normalized. Batch Normalization (BN) standardizes the input values, reducing the differences between dimensions to a consistent range. The BN layer first calculates the mean and variance of each batch of data, then subtracts the mean and divides by the variance to obtain the normalized data.

[0052] After normalization, the normalized data is input into the activation layer. The activation function used in this layer is the LeakyReLU activation function, and its formula is:

[0053]

[0054] Where x represents the input data and α is a learnable parameter variable.

[0055] The result of the activation function layer is input into the maximum pooling layer to compress the data and ensure that the data is the maximum value within a certain dimension.

[0056] The above operation is the operation of a layer of convolutional neural network. The output of the i-th layer of convolutional neural network is the input of the i+1-th layer of convolutional neural network. After N iterations of this operation, the local feature map of the image can be obtained and expressed as F conv .

[0057] The original image is divided into K small blocks. This is achieved by setting the kernel_size and padding values ​​of the convolution kernel equal. Then the value of each small block is flattened to obtain a linear projection. Then the position information is added to serve as the feature information of each small block. The feature information of each image block can be expressed as X i Then, an N-layer Transformer encoder is constructed to learn the global features of the image. Each layer of the Transformer encoder contains a normalization layer, a self-attention calculation layer, a residual connection layer, and a multi-layer perceptron module. The output of each layer of the Transformer encoder is the input of the next layer of the Transformer encoder. The processing method of each layer of the Transformer encoder is as follows:

[0058] The feature information of each image block is input into the Layer Normalization (LN) layer for linear normalization. Its function is the same as Batch Normalization, which reduces the data features to a certain range.

[0059] The normalized result X i ' Perform self-attention calculation to obtain the semantic relationship between each image feature and other image features. First, calculate the normalized result X of each image feature. i ′’s linear projection matrix, respectively according to W Q , W K and W V Three weight matrices and X i 'Multiply and calculate the linear projection matrix Q, K, V. Q, K and V represent X i ′ projection matrices of different dimensions. Then the attention value is calculated based on these three linear projection matrices: Where D represents the dimension of the input and softmax(*) is the activation function. The multi-head attention mechanism is used here, so the final multi-head attention calculation result is: MSA = [SA1, SA2, ..., SA k ]U msa Here K and U msa They represent the number of attention heads and the conversion matrix respectively, and SA represents the attention calculation results of each head.

[0060] After the multi-head attention is calculated, the previous feature is added for residual connection. It can be expressed as: X i ′=X i′+MSA. A multi-layer perceptron module is set after the residual connection to further learn the features after the residual connection and discard the parameter weights. This step can effectively improve the fit of the model method.

[0061] The above operation is the operation of a layer of Transformer encoder. The output of the i-th layer encoding operation is the input of the i+1-th layer encoding operation. After N iterations of this operation, the global feature map of the image can be obtained and represented as F tran .

[0062] For the local feature map F conv and the global feature map F tran , we use two variable parameters α and β to adjust the weight ratio between the two feature maps. It can be formulated as: F out =α·F conv +β·F tran In this way, the final image feature map F can be obtained out .

[0063] For the final image feature map F out ,Set up a power image anomaly detection model, including a multi-layer perceptron module for representation learning, and then pass the final fully connected layer to obtain the final recognition result, which is used to identify abnormal situations in power images.

[0064] First, an outlier detection model was constructed through the above steps. The model input is drone-captured photos. Feature extraction in the model's intermediate layers and the final fully connected layer yield the model's output. The output is either 0 or 1, with 0 indicating no issues with the circuit device in the inspection photo and 1 indicating issues with the device, such as rust, breakage, or damage. Next, the drone-captured inspection photos are divided into training, validation, and test sets. The model is first trained using the training set photos, then validated on the validation set and fine-tuned to fine-tune the network model's hyperparameters. Finally, the model's predictive capabilities are tested on the test set. After training, validation, and testing, the final model is obtained for power transmission line defect detection. Inputting one or a set of images directly outputs one or a set of values, with values ​​of 0 and 1 representing the prediction results.

[0065] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0066] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0067] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0068] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0069] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A power image processing method based on complete learning, characterized by: The steps include: The feature map F out and feature map F out The calibration value is used as the training set, and the power image anomaly detection model is built using the training set; Input the power image into the trained power image anomaly detection model and output the abnormality prediction results of the power equipment in the power image; The feature map F out It is obtained by calculation using the following formula: F out =α·F conv +β·F tran Among them, F conv is the local feature map, F tran is the global feature map, α and β are parameters; The global feature map F tran The acquisition method includes the following steps: The original image of the power image is divided into K small blocks, and the value of each small block is flattened to obtain a linear projection. The linear projection plus the position information is used as the feature information of each small block. i express; The feature information X of each small block i Input to N layers of Transformer encoders, the output of each layer of Transformer encoder is the input of the next layer of Transformer encoder; After N iterations, the global feature map F is obtained. tran ; The Transformer encoder of each layer includes at least a normalization layer, a self-attention calculation layer, a residual connection layer and a multi-layer perceptron module; The operation method of the Transformer encoder at each layer includes the following steps: The feature information X of each small block i Input to the normalization layer for linear normalization to obtain the normalized result X i '; Normalized result X i ′ is input to the self-attention calculation layer, and the result X of normalizing the features of each image block is obtained i ′ and W Q ,W K and W V Multiply the three weight matrices to obtain the linear projection matrices Q, K, and V. Calculate the multi-head attention MSA based on the linear projection matrices Q, K, and V. MSA = [SA1,SA2,…,SA k ]U msa , among which, SA k represents the attention value of the kth attention head, U msa represents the transformation matrix; Input the multi-head attention MSA into the residual connection layer, and according to the normalized result X i ′ is added to the multi-head attention MSA to obtain the output of the residual connection layer; The output of the residual connection layer is input into the multi-layer perceptron module for learning and parameter weight discarding to obtain the output of the Transformer encoder of this layer.

2. The power image processing method based on complete learning according to claim 1, characterized in that: The local feature map F conv The acquisition method includes the following steps: The original image of the power image is input into an N-layer convolutional neural network. The output of each layer of the convolutional neural network is the input of the next layer of the convolutional neural network. After N iterative operations, the local feature map F is obtained conv ; The convolutional neural network of each layer includes at least a convolution layer, a normalization layer, an activation layer and a pooling layer; The operation method of each layer of the convolutional neural network includes the following steps: The image passes through the convolution filtering operation of the convolution layer to obtain the convolution layer output; The output of the convolution layer is normalized through the normalization layer to obtain the normalized layer output; The output of the normalized layer is passed through the activation function of the activation layer to obtain the output of the activation layer; The output of the activation layer is compressed through the pooling layer to obtain the output of the convolutional neural network in this layer.

3. The power image processing method based on complete learning according to claim 1, characterized in that: The SA k The calculation formula is as follows: Where D represents the dimension of the input and softmax(*) is the activation function.

4. The power image processing method based on complete learning according to claim 1, characterized in that: The power image outlier detection model includes at least a multi-layer perceptron module and a fully connected layer.

5. The power image processing method based on complete learning according to claim 1, characterized in that: The power equipment abnormality prediction results in the power image include: the power equipment has a problem, and the power equipment has no problem.

6. A power image processing device based on complete learning, characterized by: Includes the following modules: Training module, used to transform the feature map F out and feature map F out The calibration value is used as the training set, and the power image anomaly detection model is built using the training set; The prediction module is used to input the power image into the trained power image anomaly detection model and output the power equipment anomaly prediction result in the power image; The feature map F out It is obtained by calculation using the following formula: F out =α·F conv +β·F tran Among them, F conv is the local feature map, F tran is the global feature map, α and β are parameters; The global feature map F tran The acquisition method includes the following steps: The original image of the power image is divided into K small blocks, and the value of each small block is flattened to obtain a linear projection. The linear projection plus the position information is used as the feature information of each small block. i express; The feature information X of each small block i Input to N layers of Transformer encoders, the output of each layer of Transformer encoder is the input of the next layer of Transformer encoder; After N iterations, the global feature map F is obtained. tran ; The Transformer encoder of each layer includes at least a normalization layer, a self-attention calculation layer, a residual connection layer and a multi-layer perceptron module; The operation method of the Transformer encoder at each layer includes the following steps: The feature information X of each small block i Input to the normalization layer for linear normalization to obtain the normalized result X i '; Normalized result X i ′ is input to the self-attention calculation layer, and the result X of normalizing the features of each image block is obtained i ′ and W Q ,W K and W V Multiply the three weight matrices to obtain the linear projection matrices Q, K, and V. Calculate the multi-head attention MSA based on the linear projection matrices Q, K, and V. MSA = [SA1,SA2,…,SA k ]U msa , among which, SA k represents the attention value of the kth attention head, U msa represents the transformation matrix; Input the multi-head attention MSA into the residual connection layer, and according to the normalized result X i ′ is added to the multi-head attention MSA to obtain the output of the residual connection layer; The output of the residual connection layer is input into the multi-layer perceptron module for learning and parameter weight discarding to obtain the output of the Transformer encoder of this layer.

Images