An improved U-Net high-temperature anomaly detection method fusing multi-scale and attention mechanism

Abstract

The application relates to the technical field of image segmentation, in particular to an improved U-Net high-temperature anomaly detection method fusing a multi-scale and attention mechanism, which comprises the following steps: collecting an infrared image of power equipment and establishing a dataset sample; an improved U-Net network model fusing the multi-scale and attention mechanism is established; residual units are used to replace convolution units in the original U-Net network so as to ensure rapid convergence of the network; a Transformer model is added to the middle layer of the original U-Net network so as to realize enhancement of the global modeling capability of the network; PixelShuffle pixel reorganization is used to replace the transposed convolution adopted in the original U-Net network so as to realize recovery of more detailed information; an ASPP multi-scale feature fusion module is added before up-sampling so as to realize expansion of a receptive field; and the dataset sample is used for model training so as to realize automatic segmentation of a target high-temperature anomaly area. The algorithm can effectively realize power high-temperature anomaly detection, improve the work efficiency of a patrol personnel and provide guarantee for high-temperature anomaly detection.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to an improved U-Net high-temperature anomaly detection method that integrates multi-scale and attention mechanisms. Background Technology

[0002] my country has a vast territory with numerous power equipment distribution points. These power equipment are exposed to the elements for extended periods, making them susceptible to high temperatures, strong winds, lightning strikes, rain, snow, and wildfires, which can lead to aging and damage. Failure of any power equipment can cause widespread power outages, disrupting residential lives and industrial production. Therefore, regular inspections and timely, accurate detection of aging and faulty equipment are crucial for both daily life and production. With the development of drone-based power inspection equipment, images transmitted from various angles by drones can improve the efficiency of fault inspections for personnel. However, as my country's power grid expands, the workload of inspection personnel increases significantly. Manually comparing infrared images is no longer sufficient to meet the needs of personnel; intelligent identification technology is needed to assist manual identification and improve the detection efficiency of areas with abnormal high temperatures in power grids. Deep learning has demonstrated its strong advantages in image recognition, using semantic segmentation methods to diagnose infrared images of observed equipment, providing a new method for detecting high-temperature anomalies. Therefore, using deep learning for identifying high-temperature anomaly detection areas has become a trend, not only improving the efficiency of inspection personnel but also ensuring the detection of high-temperature anomalies. Summary of the Invention

[0003] The purpose of this invention is to address the shortcomings of existing technologies by providing an improved U-Net high-temperature anomaly detection method that integrates multi-scale and attention mechanisms, comprising:

[0004] Infrared images of power equipment are collected to establish a dataset sample, which is used to train an improved U-Net network model that integrates multi-scale and attention mechanisms;

[0005] An improved U-Net network model integrating multi-scale and attention mechanisms is established. The convolutional units in the original U-Net network are replaced with residual units to ensure fast network convergence. A Transformer model is added to the intermediate layers of the original U-Net network to enhance the network's global modeling capabilities. PixelShuffle pixel rearrangement is used instead of the transposed convolution used in the original U-Net network to restore more detailed information. An ASPP multi-scale feature fusion module is added before upsampling to expand the receptive field.

[0006] Based on the improved U-Net network model that integrates multi-scale and attention mechanisms, the dataset samples are used for model training to achieve automatic segmentation of target high-temperature anomaly regions.

[0007] Furthermore, the improved U-Net network model that integrates multi-scale and attention mechanisms mainly consists of five encoders and four decoders. The encoders are used to extract features from the target image, and the decoders are used to fuse the features of the target image. The encoders consist of a residual module and a max pooling layer, and the decoders consist of the residual module, an ASPP multi-scale feature fusion module, and an upsampling operation.

[0008] Furthermore, the residual module includes residual units, convolutional units, and a ReLU function. The convolutional units are used to extract features from the residual units, and the ReLU function retains and maps the extracted features. The residual unit consists of two 3×3 convolutional units and one 1×1 convolutional unit connected by residuals, and the ReLU function is connected after the residual connection.

[0009] Furthermore, the ReLU function is followed by a 2×2 max pooling layer, which is used to perform downsampling, remove redundant information, compress features, and simplify network complexity.

[0010] Furthermore, the high-level semantic features obtained after downsampling by the fourth max pooling layer are fed into the Transformer model after three convolutions. The Transformer model is used to reshape the features of the encoder, reshaping the resolution of the high-level semantic features output from the Transformer model to the resolution when they enter the Transformer model, thereby enhancing the global information representation. The Transformer model mainly includes two 1×1 convolutional layers, six Transformer layers, and a Hidden Feature obtained after passing through the six Transformer layers. The 1×1 convolutional layer is used to change the resolution of the feature map, and the Transformer Encoder is used to encode the input features.

[0011] In the Transformer model, the input feature map X∈R is first reshaped through a 1x1 convolutional layer. H×W×C The image is then converted into serialized data, and X is flattened into a 2D sequence. (H,W) is the resolution of the input feature map X, C is the number of channels in X, (P,P) is the resolution of each feature patch, and N = HW / P 2 It is the number of image patches, P2 ×C is the length of each sequence.

[0012] The data calculation formula in the Transformer layer is:

[0013] The input data z0 for the first layer of the Transformer Encoder is obtained using the following formula:

[0014]

[0015] in That is, the patch embedding after Linear Projection, E is the slice embedding projection, Epos is the position embedding, and the transformer layer also includes the L-layer Multihead Self-Attention (MSA) and Multi-Layer Perceptron (MLP) modules.

[0016] For the l-th layer of the Transformer Encoder, let its input be z. l-1 The output is z l The calculation process is as follows:

[0017] z′ l =MSA(LN(z) l-1 ))+z l-1 (2)

[0018] z l =MSA(LN(z′) l ))+z′ l (3)

[0019] Where LN is Layer Norm, z l For the encoded image representation, LayerNorm is applied before each MLP and MSA, and residual connections are performed after the MLP and MSA.

[0020] The MSA function first splits the input sequence according to the number of heads, then expands the dimensions and maps the features to Q, K, V, where Q, K, V are three matrices that are the outputs of the previous Encoder layer. After obtaining matrices Q, K, V, the output of Self-Attention can be calculated.

[0021] The formula for calculating self-attention is shown in Formula 4.

[0022]

[0023] The expression for multi-head self-attention is shown in Equation 5. Each head outputs a result matrix, these matrices are concatenated into a single matrix, and then multiplied by a weight matrix W. O This ensures that the final matrix size is consistent with the size of the result matrix of one head. Finally, the outputs of the QKV group of heads are concatenated to obtain the output of MSA.

[0024] MultiHead(Q,K,V)=Concat(head1,…,head h W O (5)

[0025]

[0026] in

[0027] W O This is the weight matrix. Wi represents the transformation matrix of the i-th head, and h represents the number of heads.

[0028] MLP will d model Magnified to mlp-ratio×d model Then shrink it to d model The process of mapping data to a high-dimensional space and then back to a low-dimensional space allows us to learn more abstract features and improve the expressive power of those features.

[0029] In this invention, L = 6, heads = 12, d k =d model / heads=64,d model =768, patch=1, H=W=8, C=1024. First, a 1×1 convolution maps the input features (8, 8, 1024) to (8, 8, 768). Then, the features are flattened into a 2D sequence and passed through 6 Transformer Layers to obtain the Hidden Feature (65, 768). The Hidden Feature (64 + classification token, 768) discards the classification token and has a size of (64, 768). Finally, the Hidden Feature is reshaped to (8, 8, 768), and a 1×1 convolution maps the reshaped Hidden Feature (8, 8, 768) to (8, 8, 1024).

[0030] Furthermore, the ASPP multi-scale feature fusion module is used to expand the receptive field to reduce the loss of detailed information. The ASPP multi-scale feature fusion module includes two 1×1 convolutional layers and three 3×3 convolutional layers with dilation rates of 2, 3, and 4.

[0031] Furthermore, the specific structure of the ASPP multi-scale feature fusion module is as follows: a 1×1 convolutional layer and three 3×3 convolutional layers with dilation rates of 2, 3, and 4 form four parallel branches. After obtaining feature information, the four branches are concatenated and then enter the 1×1 convolutional layer to change the number of channels, reducing the number of channels to twice the number of input channels to increase the frequency of information extraction, thereby increasing the amount of detailed information reflected. Finally, the output is from the ASPP multi-scale feature fusion module.

[0032] Furthermore, the upsampling operation is performed using PixelShuffe pixel reassembly in ESPCN (section 3.5). PixelShuffe pixel reassembly is used to effectively enlarge the scaled-down feature map, transforming the low-resolution feature map into a high-resolution feature map through convolution and multi-channel reassembly, thereby achieving the upsampling operation.

[0033] The PixelShuffle pixel reconstruction replaces the transposed convolution used in the original U-Net. In this invention, the input feature resolution is (H,W,C), the output feature resolution of the ASPP module is (H,W,r*C), and a high-resolution image (r*H, r*W, C / r) is obtained through PixelShuffle pixel reconstruction, where r = 2.

[0034] Furthermore, the improved U-Net network model that integrates multi-scale and attention mechanisms adopts a skip-link approach to fuse features of the same resolution between the encoder and decoder in order to recover more spatial information.

[0035] Furthermore, the dataset samples are divided into a training set, a validation set, and a test set. The training set is used to train the improved U-Net network model that integrates multi-scale and attention mechanisms. The validation set is used to adjust the parameters of the improved U-Net network model that integrates multi-scale and attention mechanisms. The test set is used to test whether the improved U-Net network model that integrates multi-scale and attention mechanisms is accurate.

[0036] Compared with the prior art, the beneficial effects of the present invention are:

[0037] This invention adds a multi-scale feature fusion module before upsampling in the U-Net network model to reduce the loss of detail information and expand the receptive field; it uses PixelShuffle pixel recombination instead of the transposed convolution used in the original U-Net to recover more detail information; and it adds a Transformer model to the intermediate layer of the U-Net network to enhance the network's global modeling capability. Attached Figure Description

[0038] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0039] In the attached diagram:

[0040] Figure 1 This is a flowchart illustrating the implementation of an improved U-Net high-temperature anomaly detection method that integrates multi-scale and attention mechanisms according to the present invention.

[0041] Figure 2 This is a schematic diagram of the Transformer model.

[0042] Figure 3 A schematic diagram of the ASPP multi-scale feature fusion module;

[0043] Figure 4 This invention presents an improved U-Net high-temperature anomaly detection method that integrates multi-scale and attention mechanisms, used to detect high-temperature anomaly regions in infrared images of electrical systems. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0045] Example 1

[0046] This invention provides an improved U-Net high-temperature anomaly detection method that integrates multi-scale and attention mechanisms, which basically includes the following steps:

[0047] Infrared images of power equipment were collected to create a dataset sample, which was then used to train an improved U-Net network model that integrates multi-scale and attention mechanisms.

[0048] An improved U-Net network model integrating multi-scale and attention mechanisms is established. The convolutional units in the original U-Net network are replaced with residual units to ensure fast network convergence. A Transformer model is added to the intermediate layers of the original U-Net network to enhance the network's global modeling capabilities. PixelShuffle pixel rearrangement is used instead of the transposed convolution used in the original U-Net network to restore more detailed information. An ASPP multi-scale feature fusion module is added before upsampling to expand the receptive field.

[0049] An improved U-Net network model based on a fusion of multi-scale and attention mechanisms is used to train the model using dataset samples to achieve automatic segmentation of target high-temperature anomaly regions.

[0050] Furthermore, such as Figure 1 As shown, the improved U-Net network model that integrates multi-scale and attention mechanisms mainly consists of five encoders and four decoders. The encoders are used to extract features from the target image, and the decoders are used to fuse the features of the target image. The encoder consists of a residual module and a max pooling layer, and the decoder consists of the residual module, an ASPP multi-scale feature fusion module, and an upsampling operation.

[0051] Furthermore, the residual module includes residual units, convolutional units, and a ReLU function. The convolutional units are used to extract features from the residual units, and the ReLU function retains and maps the extracted features through a function. The residual unit is formed by residual connection of two 3×3 convolutional units and one 1×1 convolutional unit, and a ReLU function is connected after the residual connection.

[0052] Furthermore, the ReLU function is followed by a 2×2 max pooling layer, which is used for downsampling, removing redundant information, compressing features, and simplifying the network complexity.

[0053] Furthermore, the high-level semantic features obtained after downsampling by the fourth max-pooling layer are fed into the Transformer model after three convolutions, such as... Figure 2As shown, the Transformer model is used to reshape the features of the encoder, reshaping the resolution of the high-level semantic features output from the Transformer model to the resolution when they enter the Transformer model, thereby enhancing the global information representation. The Transformer model mainly includes two 1×1 convolutional layers, six Transformer layers, and the Hidden Feature obtained after passing through the six Transformer layers. The 1×1 convolutional layer is used to change the resolution of the feature map, and the Transformer Encoder is used to encode the input features.

[0054] In the Transformer model, the input feature map X∈R is first reshaped through a 1x1 convolutional layer. H×W×C The image is then converted into serialized data, and X is flattened into a 2D sequence. (H,W) is the resolution of the input feature map X, C is the number of channels in X, (P,P) is the resolution of each feature patch, and N = HW / P 2 It is the number of image patches, P 2 ×C is the length of each sequence.

[0055] The data calculation formula in the Transformer layer is:

[0056] The input data z0 for the first layer of the Transformer Encoder is obtained using the following formula:

[0057]

[0058] in That is, the patch embedding after Linear Projection, E is the slice embedding projection, Epos is the position embedding, and the transformer layer also includes the L-layer Multihead Self-Attention (MSA) and Multi-Layer Perceptron (MLP) modules.

[0059] For the l-th layer of the Transformer Encoder, let its input be z. l-1 The output is z l The calculation process is as follows:

[0060] z′ l =MSA(LN(z) l-1 ))+z l-1 (2)

[0061] z l =MSA(LN(z′) l ))+z′ l (3)

[0062] Where LN is Layer Norm, z l For the encoded image representation, LayerNorm is applied before each MLP and MSA, and residual connections are performed after the MLP and MSA.

[0063] The MSA function first splits the input sequence according to the number of heads, then expands the dimensions and maps the features to Q, K, V, where Q, K, V are three matrices that are the outputs of the previous Encoder layer. After obtaining matrices Q, K, V, the output of Self-Attention can be calculated.

[0064] The formula for calculating self-attention is shown in Formula 4.

[0065]

[0066] The expression for multi-head self-attention is shown in Equation 5. Each head outputs a result matrix, these matrices are concatenated into a single matrix, and then multiplied by a weight matrix W. O This ensures that the final matrix size is consistent with the size of the result matrix for one head. Finally, the outputs of the QKV group of heads are concatenated to obtain the output of MSA.

[0067] MultiHead(Q,K,V)=Concat(head1,...,head h W O (5)

[0068]

[0069] in

[0070] W O This is the weight matrix. Wi represents the transformation matrix of the i-th head, and h represents the number of heads.

[0071] MLP will d model Magnified to mlp-ratio×d model Then shrink it to d model The process of mapping data to a high-dimensional space and then back to a low-dimensional space allows us to learn more abstract features and improve the expressive power of those features.

[0072] In this invention, L = 6, heads = 12, d k =dmodel / heads=64,d model =768, patch=1, H=W=8, C=1024. First, a 1×1 convolution maps the input features (8, 8, 1024) to (8, 8, 768). Then, the features are flattened into a 2D sequence and passed through 6 Transformer Layers to obtain the Hidden Feature (65, 768). The Hidden Feature (64 + classification token, 768) discards the classification token and has a size of (64, 768). Finally, the Hidden Feature is reshaped to (8, 8, 768), and a 1×1 convolution maps the reshaped Hidden Feature (8, 8, 768) to (8, 8, 1024).

[0073] Furthermore, such as Figure 3 As shown, the ASPP multi-scale feature fusion module is used to expand the receptive field to reduce the loss of detailed information. The ASPP multi-scale feature fusion module includes two 1×1 convolutional layers and three 3×3 convolutional layers with dilation rates of 2, 3, and 4.

[0074] Furthermore, the specific structure of the ASPP multi-scale feature fusion module is as follows: a 1×1 convolutional layer and three 3×3 convolutional layers with dilation rates of 2, 3, and 4 form four parallel branches. After obtaining feature information, the four branches are concatenated. The concatenated branches then enter a 1×1 convolutional layer to change the number of channels, reducing the number of channels to twice the number of input channels. This is used to increase the frequency of information extraction, thereby increasing the amount of detailed information reflected. Finally, the output is from the ASPP multi-scale feature fusion module.

[0075] Furthermore, the upsampling operation is performed using PixelShuffe pixel reassembly in ESPCN (section 3.5). PixelShuffe pixel reassembly is used to effectively enlarge the scaled-down feature map, transforming the low-resolution feature map into a high-resolution feature map through convolution and multi-channel reassembly, thus achieving the upsampling operation.

[0076] The PixelShuffle pixel reconstruction replaces the transposed convolution used in the original U-Net. In this invention, the input feature resolution is (H,W,C), and the output feature resolution of the ASPP module is (H,W,r*C). The high-resolution image (r*H, r*W, C / r) is obtained through PixelShuffle pixel reconstruction, where r = 2.

[0077] Furthermore, the improved U-Net network model, which integrates multi-scale and attention mechanisms, adopts a skip-link approach to fuse features of the same resolution between the encoder and decoder to achieve the recovery of more spatial information.

[0078] Furthermore, the dataset samples are divided into a training set, a validation set, and a test set. The training set is used to train the improved U-Net network model that integrates multi-scale and attention mechanisms. The validation set is used to adjust the parameters of the improved U-Net network model that integrates multi-scale and attention mechanisms. The test set is used to test whether the improved U-Net network model that integrates multi-scale and attention mechanisms is accurate.

[0079] In order to demonstrate the model effect of the present invention, such as Figure 4 As shown, this invention segments infrared images and accurately detects areas with abnormal high temperatures.

[0080] The parts of this invention not described in detail are well-known to those skilled in the art.

[0081] Those skilled in the art should recognize that the above embodiments are merely illustrative of the present invention and are not intended to limit the present invention. Any variations or modifications to the above embodiments that are within the essential spirit of the present invention will fall within the scope of the claims of the present invention.

Claims

1. An improved U-Net high-temperature anomaly detection method fusing multi-scale and attention mechanism, characterized in that, The application relates to an infrared image acquisition method and device for power equipment, and a high-temperature anomaly area automatic segmentation method and device. The application comprises the following steps: An infrared image of power equipment is acquired to establish a data set sample, which is used for training an improved U-Net network model fusing a multi-scale and attention mechanism; The improved U-Net network model fusing the multi-scale and attention mechanism is established, the convolution unit in the original U-Net network is replaced by a residual unit to ensure network rapid convergence; a Transformer model is added to the middle layer of the original U-Net network, which is used for realizing enhancement of the network global modeling capability; PixelShuffle pixel reorganization is used to replace the transposed convolution adopted in the original U-Net network to realize recovery of more detailed information, and an ASPP multi-scale feature fusion module is added before up-sampling to realize expansion of the receptive field; Based on the improved U-Net network model fusing the multi-scale and attention mechanism, the data set sample is used for model training to realize automatic segmentation of a target high-temperature anomaly area; The improved U-Net network model fusing the multi-scale and attention mechanism mainly comprises five encoders and four decoders, the encoders are used for extracting features of a target image, and the decoders are used for fusing features of the target image.

2. The improved U-Net high-temperature anomaly detection method fusing multi-scale and attention mechanism according to claim 1, characterized in that: The encoder is composed of a residual module and a maximum pooling layer, and the decoder is composed of the residual module, an ASPP multi-scale feature fusion module and an up-sampling operation.

3. The improved U-Net high-temperature anomaly detection method fusing multi-scale and attention mechanism according to claim 2, characterized in that: The residual module comprises a residual unit, a convolution unit and a ReLu function, the convolution unit is used for extracting features in the residual unit, and the ReLu function is used for retaining and mapping out the extracted features through a function, wherein the residual unit is connected in residual connection by two 3*3 convolution units and one 1*1 convolution unit, and the residual connection is followed by the ReLu function.

4. The improved U-Net high-temperature anomaly detection method fusing multi-scale and attention mechanism according to claim 3, characterized in that: The ReLu function is followed by a 2*2 maximum pooling layer, the maximum pooling layer is used for down-sampling, removing redundant information, compressing features to realize simplification of network complexity, wherein the maximum pooling layer has four. The high-level semantic features obtained after down-sampling of the fourth maximum pooling layer are sent into the Transformer model after three times of convolution, the Transformer model is used for remodeling features of the encoder, and the resolution of the high-level semantic features output from the Transformer model is remodeled to the resolution when the high-level semantic features enter the Transformer model, so as to realize enhancement of global information expression, the Transformer model mainly comprises two 1*1 convolution layers, six Transformer layers and Hidden Features obtained after the six Transformer layers, wherein the 1*1 convolution layer is used for changing the resolution of a feature map, and the Transformer Encoder is used for encoding input features.

5. The improved U-Net high-temperature anomaly detection method fusing multi-scale and attention mechanism according to claim 4, characterized in that: The ASPP multi-scale feature fusion module is used to expand the receptive field to reduce the loss of detail information, and the ASPP multi-scale feature fusion module comprises two 1*1 convolution layers and three 3*3 convolution layers with expansion rates of 2, 3 and 4.

6. The improved U-Net high-temperature anomaly detection method fusing multi-scale and attention mechanism according to claim 5, characterized in that: The specific structure of the ASPP multi-scale feature fusion module is that one 1*1 convolution layer and three 3*3 convolution layers with expansion rates of 2, 3 and 4 form four parallel branches, the four branches obtain feature information and are spliced, and after splicing, the 1*1 convolution layer is entered to change the channel number and reduce the channel number to twice the input channel number, which is used to improve the frequency of extracted information to increase the detail information of the reaction, and finally output from the ASPP multi-scale feature fusion module.

7. The improved U-Net high-temperature anomaly detection method fusing multi-scale and attention mechanism according to claim 2, characterized in that: The manner of the upsampling operation is PixelShuffe pixel reorganization in ESPCN, which is used to effectively enlarge the reduced feature map, and the low-resolution feature map is changed into a high-resolution feature map through convolution and multi-channel reorganization, realizing the upsampling operation.

8. The improved U-Net high-temperature anomaly detection method fusing multi-scale and attention mechanism according to claim 2, characterized in that: The improved U-Net network model fusing multi-scale and attention mechanism adopts a skip link mode to fuse the features of the same resolution of the encoder and the decoder to realize the recovery of more spatial information.

9. The improved U-Net high-temperature anomaly detection method fusing multi-scale and attention mechanism according to claim 1, characterized in that: The data set samples are divided into a training set, a validation set and a test set, the training set is used to train the improved U-Net network model fusing multi-scale and attention mechanism, the validation set is used to adjust the parameters of the improved U-Net network model fusing multi-scale and attention mechanism, and the test set is used to test whether the improved U-Net network model fusing multi-scale and attention mechanism is accurate.

Images