Small sample motor infrared image fault diagnosis method

Abstract

The invention discloses a small sample motor infrared image fault diagnosis method. According to the method, AE-DCGAN is used for data enhancement, and diversified motor infrared image samples are generated, so that the problem of small samples is solved, and the training effect of the model is greatly improved. In combination with dynamic task weighted meta learning, the model can adaptively adjust task weights of different fault types, so that the accuracy and adaptability of fault diagnosis are improved. And an adaptive multi-scale fusion module is adopted, so that the capability of the model in multi-scale image processing is enhanced, the model can process fault areas of different scales at the same time, and the segmentation precision and the reasoning efficiency are improved. Through combination of data enhancement, meta-learning and multi-scale fusion, high precision and high efficiency of motor fault diagnosis are realized, and the method has a wide application prospect.

Description

Technical Field

[0001] This invention relates to the fields of electrical equipment fault diagnosis and image feature extraction technology, specifically a method for fault diagnosis of small sample motor infrared images. Background Technology

[0002] As one of the core pieces of equipment widely used in modern industry, the monitoring of the motor's operating status and fault diagnosis are crucial for ensuring production safety and extending equipment lifespan. Traditional motor fault diagnosis methods mostly rely on human experience or simple physical models, resulting in low diagnostic accuracy and poor adaptability. In recent years, with the continuous improvement of computer technology and data processing capabilities, motor fault diagnosis methods based on deep learning and image processing technologies have gradually become a research hotspot. By analyzing infrared images of the motor during operation, early warning and precise location of motor faults can be achieved, thereby improving the efficiency and accuracy of fault diagnosis.

[0003] The shortcomings of existing technology: Current motor fault diagnosis methods still face several challenges when processing infrared images. For example, how to effectively improve the diagnostic accuracy of fault images with a small sample size, how to generate more samples through data augmentation methods, and how to better capture temperature changes in detailed areas of the image remain key issues in current research. Summary of the Invention

[0004] The purpose of this invention is to provide a method for diagnosing faults in small-sample motor infrared images, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for diagnosing faults in small-sample motor infrared images, the diagnostic method specifically including the following steps: S1. Load the original motor infrared image dataset and quickly expand the original motor infrared image dataset in the early stage of training using the Mosaic data augmentation method. S2. Establish a self-attention-enhanced generative adversarial network AE-DCGAN to enhance the infrared motor image dataset and generate the final training samples. S3. Employ dynamic task-weighted meta-learning to dynamically adjust the weights of different tasks and pay attention to the correlation between tasks; S4. Use the DeepLabV3+ model to perform semantic segmentation of the fault region, obtain the motor infrared image fault diagnosis model, and build an adaptive multi-scale fusion module. S5. Perform fault diagnosis.

[0006] Preferably, step S1 specifically includes: a1. From the original motor infrared image dataset Four images are randomly selected from the data. ; a2, Yes Perform random cropping and scaling operations to obtain the scaled image. The dimensions are recorded as : in, Indicates the image Perform random cropping. It performs scaling operations on the cropped image; a3. After cropping and scaling Images stitched together to form a new image The splicing method is to... Four images are placed at the four corners of the matrix to generate a new image: in, This indicates four images The images are stitched together according to the specified stitching order to obtain a new image. The dimensions are recorded as ; a4. Repeat steps a1-a3 to generate several stitched images. Expand the training dataset to generate a new sample set. This will include data from the original motor infrared image dataset. Enhanced image generated in the process, set to generate The new sample, the enhanced motor infrared image dataset, is represented as follows: .

[0007] Preferably, step S2 specifically includes: b1, Generator Through random noise Generate enhanced images; b2, Discriminator To determine whether an input image is a real image: The discriminator uses a convolutional neural network structure and focuses on key regions in the image through a convolutional attention mechanism. The discriminator process is as follows: in, This indicates that the discriminator is sensitive to the image. The judgment, This represents the Sigmoid activation function. This indicates that a convolution operation is performed on the image; b3. Use the variance of the feature space to represent the diversity of the image: in, This represents the feature vector extracted from the generated image through a pre-trained network. Indicates variance; b4. Introduce a metric for the response intensity of the fault region: in, This indicates the attention map or feature map in the fault region. The response intensity; b5. Combine the objective functions from steps b2, b3, and b4 and optimize the generator parameters. : ; The loss of the discriminator is defined as: in, This indicates that the discriminator recognizes the real image. The discrimination probability, This represents the probability that the discriminator classifies the generated image. b6. Use alternating optimization of the generator and discriminator. The generator loss is minimized through backpropagation, and the discriminator loss is minimized through backpropagation. in, Represents the loss function of the generator. This represents the loss function of the discriminator; b7. The model outputs the enhanced image as follows: in, This represents the generated image. The input is random noise; The final dataset is as follows: .

[0008] Preferably, step S3 specifically includes: c1, Assume that Each of the different tasks This represents a different type of fault: c2. Configure each task The weight is Weight Based on the task's error The adjustment is dynamic, and the specific formula is as follows: in, Indicates task The error, This represents the balancing parameter, used to control the impact of task errors on the weights; The task error is obtained by calculating the loss function for each task. : in, Indicates task loss function, Indicates task The dataset, Represents the parameters of the model; c3. Inter-task correlation Through computation task and The cosine similarity between them is used to obtain: in, Indicates task Features Indicates task Features express The model, express The model; c4. Design a reinforcement learning framework for task weight adjustment, including the error of the current task and the performance metrics of the model. Weighted actions assigned to each task The reward value is given based on the learning progress of the task and the overall model performance. , in, This indicates a change in segmentation accuracy. It is a regulatory factor. Indicates the change in meta-learning loss; The agent updates the task weight allocation strategy through Q-learning: in, It's the learning rate. It is a discount factor. That's the next step.

[0009] Preferably, step S4 specifically includes: d1, Input Image Temperature distribution from infrared images of the motor is used for subsequent fault diagnosis. in, Indicates the first Zhang input image, size is ; d2. In DeepLabV3+, the input image is first processed by convolution to extract basic features, and the image size is adjusted by stride. Then, an activation function is used to enhance non-linear features. Finally, dilated convolution is used to expand the receptive field and capture the range information of the image. The specific process is as follows: in, This represents the feature map after convolution. It is a convolution kernel with a size of 3×3; This indicates a dilated convolution kernel with a dilation rate of 6. This represents the feature map after dilated convolution, with a size of [size missing]. ; d3. Calculate the weighting coefficients based on the activation level of each feature map. Weight It is inversely proportional to the activation level of the feature map: d4. Fuse feature maps of different scales using a weighted summation method: d5. After optimization through feature fusion and attention mechanism, the model outputs the segmentation result of the fault region through the decoder and Softmax layer. The segmentation result indicates whether each pixel belongs to the normal region or the fault region.

[0010] Preferably, in step S2: the network layer structure of the generator G of AE-DCGAN is in the following order: input layer, hidden layer 1, hidden layer 2, attention mechanism layer, hidden layer 3, hidden layer 4, output layer; the attention mechanism layer of the generator G of AE-DCGAN includes a self-attention mechanism block, a convolutional block attention module and a multi-head attention mechanism block; the discriminator D network of AE-DCGAN is a 5-layer pyramid structure from bottom to top.

[0011] Preferably, the specific workflow of the attention mechanism layer of the generator G in the AE-DCGAN is as follows: t1, Given the input feature map First, the query matrix is ​​obtained through convolution operation. Key matrix Sum matrix : in, , , It is the weight matrix for learning; Self-attention determines an attention score by calculating the similarity between the query and the key. : in, It is the dimension of the key matrix. It is a normalization operation used to obtain the attention weights for each position.

[0012] Based on attention score Sum matrix The weighted feature map is obtained as follows: .

[0013] t2. Generate attention weights for each channel using the sigmoid activation function: in, Indicates the input feature map, This represents the sigmoid activation function. This represents a multilayer perceptron. and These represent global average pooling and max pooling operations, respectively. Spatial attention weights are generated using convolution operations: in, This represents the convolution operation. This means concatenating the pooled feature maps to generate attention weights in the spatial dimension. t3, for each head Calculate attention scores using different queries, keys, and values: in, , , It is the first The query, key, and value matrix of each head; The outputs of each head are concatenated and a linear transformation is applied to obtain the final output: in, It refers to the number of heads. It is a linear transformation matrix.

[0014] Compared with the prior art, the beneficial effects of the present invention are: This invention provides a method for fault diagnosis of motors using small-sample infrared images. It employs AE-DCGAN for data augmentation, generating diverse infrared image samples of motors to address the small-sample problem and significantly improve model training performance. Combined with dynamic task-weighted meta-learning, the model can adaptively adjust task weights for different fault types, thereby improving the accuracy and adaptability of fault diagnosis. Furthermore, an adaptive multi-scale fusion module enhances the model's ability in multi-scale image processing, enabling it to simultaneously handle fault regions of different scales, improving segmentation accuracy and inference efficiency. This invention, through the combination of data augmentation, meta-learning, and multi-scale fusion, achieves high precision and high efficiency in motor fault diagnosis, and has broad application prospects. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a diagram illustrating the training process of the motor infrared image fault diagnosis model of the present invention. Figure 3 This is a diagram of the generator G network structure of the AE-DCGAN of the present invention; Figure 4 This is a diagram of the discriminator D network structure of the AE-DCGAN of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0018] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "setting" should be interpreted broadly. For example, they can refer to a fixed connection or setting, a detachable connection or setting, or an integral connection or setting. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0019] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a number" means two or more, unless otherwise explicitly specified.

[0020] Example Please see Figure 1-4 As shown, the present invention provides a technical solution for fault diagnosis of small-sample motor infrared images: the diagnosis method specifically includes the following steps: S1. Load the original motor infrared image dataset and quickly expand the original motor infrared image dataset in the early stage of training using the Mosaic data augmentation method. a1. From the original motor infrared image dataset Four images are randomly selected from the data. ; a2, Yes Perform random cropping and scaling operations to obtain the scaled image. The dimensions are recorded as : in, Indicates the image Perform random cropping. It performs scaling operations on the cropped image; a3. After cropping and scaling Images stitched together to form a new image The splicing method is to... Four images are placed at the four corners of the matrix to generate a new image: in, This indicates four images The images are stitched together according to the specified stitching order to obtain a new image. The dimensions are recorded as ; a4. Repeat steps a1-a3 to generate several stitched images. Expand the training dataset to generate a new sample set. This will include data from the original motor infrared image dataset. Enhanced image generated in the process, set to generate The new sample, the enhanced motor infrared image dataset, is represented as follows: ; S2. A self-attention-enhanced generative adversarial network (AE-DCGAN) is established to enhance the infrared motor image dataset and generate the final training samples. The network layer structure of the generator G in AE-DCGAN is as follows: input layer, hidden layer 1, hidden layer 2, attention mechanism layer, hidden layer 3, hidden layer 4, and output layer. The attention mechanism layer of the generator G in AE-DCGAN includes a self-attention mechanism block, a convolutional block attention module, and a multi-head attention mechanism block. The discriminator D network of AE-DCGAN has a 5-layer pyramid structure from bottom to top. b1, Generator Through random noise Generate enhanced images; b2, Discriminator To determine whether an input image is a real image: The discriminator uses a convolutional neural network structure and focuses on key regions in the image through a convolutional attention mechanism. The discriminator process is as follows: in, This indicates that the discriminator is sensitive to the image. The judgment, This represents the Sigmoid activation function. This indicates that a convolution operation is performed on the image; b3. Use the variance of the feature space to represent the diversity of the image: in, This represents the feature vector extracted from the generated image through a pre-trained network. Indicates variance; b4. Introduce a metric for the response intensity of the fault region: in, This indicates the attention map or feature map in the fault region. The response intensity; b5. Combine the objective functions from steps b2, b3, and b4 and optimize the generator parameters. : ; The loss of the discriminator is defined as: in, This indicates that the discriminator recognizes the real image. The discrimination probability, This represents the probability that the discriminator classifies the generated image. b6. Use alternating optimization of the generator and discriminator. The generator loss is minimized through backpropagation, and the discriminator loss is minimized through backpropagation. in, Represents the loss function of the generator. This represents the loss function of the discriminator; b7. The model outputs the enhanced image as follows: in, This represents the generated image. The input is random noise; The final dataset is as follows: ; S3. Employ dynamic task-weighted meta-learning to dynamically adjust the weights of different tasks and pay attention to the correlation between tasks; c1. The input dataset is an enhanced infrared image dataset of motors. This dataset contains images of multiple fault types. Each task... To represent a specific fault type, the goal of the task is to accurately classify fault types by learning features from the dataset. Assume we have... Each of the different tasks This represents a different type of fault: c2. Configure each task The weight is Weight Based on the task's error The adjustment is dynamic, and the specific formula is as follows: in, Indicates task The error, This represents the balancing parameter, used to control the impact of task errors on the weights; The task error is obtained by calculating the loss function for each task. : in, Indicates task loss function, Indicates task The dataset, Represents the parameters of the model; c3. Inter-task correlation Through computation task and The cosine similarity between them is used to obtain: in, Indicates task Features Indicates task Features express The model, express The model; c4. Design a reinforcement learning framework for task weight adjustment, aiming to dynamically adjust task weights based on task error and learning progress status information, including the current task error and the model's performance metrics. Weighted actions assigned to each task The reward value is given based on the learning progress of the task and the overall model performance. , in, This indicates a change in segmentation accuracy. It is a regulatory factor. Indicates the change in meta-learning loss; The agent updates the task weight allocation strategy through Q-learning: in, It's the learning rate. It is a discount factor. This is the next step; S4. Use the DeepLabV3+ model to perform semantic segmentation of the fault area, obtain the motor infrared image fault diagnosis model, and build an adaptive multi-scale fusion module. This module adjusts the weight of each scale feature according to the characteristics of the input data through a dynamic weighting mechanism, thereby improving the ability to identify fault areas at different scales. d1, Input Image Temperature distribution from infrared images of the motor is used for subsequent fault diagnosis. in, Indicates the first Zhang input image, size is ; d2. In DeepLabV3+, the input image is first processed by convolution to extract basic features, and the image size is adjusted by stride. Then, an activation function is used to enhance non-linear features. Finally, dilated convolution is used to expand the receptive field and capture the range information of the image. The specific process is as follows: in, This represents the feature map after convolution. It is a convolution kernel with a size of 3×3; This indicates a dilated convolution kernel with a dilation rate of 6. This represents the feature map after dilated convolution, with a size of [size missing]. ; d3. After dilated convolution, multiple feature maps of different scales are obtained. These feature maps have different sizes and contain different levels of image information. Assume the input feature map is... Each feature map Image features corresponding to different scales.

[0021] To achieve dynamic weighting, the L2 norm of each scale feature map needs to be calculated as the initial weight. This weight reflects the contribution of that feature map to the model output, and the weighting coefficients are calculated based on the activation level of each feature map. Weight It is inversely proportional to the activation level of the feature map: Weight This reflects the relative importance of each feature map across all scales; feature maps with higher activation levels will receive higher weights. d4. Fuse feature maps of different scales using a weighted summation method: By using weighted summation, feature maps from different scales are fused according to their weights. More important feature maps contribute more to the final fused features, ensuring that important information such as motor fault areas can be better captured by the model. The fused feature maps are shown below. Further optimization using the ReLU activation function ensures the stability and effectiveness of the model.

[0022] d5. After optimization through feature fusion and attention mechanism, the model outputs the segmentation result of the fault region through the decoder and Softmax layer. The segmentation result indicates whether each pixel belongs to the normal region or the fault region. S5. Perform fault diagnosis: Load the motor infrared image fault diagnosis model trained in step S4, input the motor infrared image to be identified, use the segmentation output of the DeepLabV3+ model to classify each pixel in the infrared image to be identified, locate the fault part in the motor; determine the fault area through the segmentation results, and perform fault diagnosis.

[0023] Furthermore, the specific workflow of the attention mechanism layer of the generator G in AE-DCGAN is as follows: t1, Given the input feature map First, the query matrix is ​​obtained through convolution operation. Key matrix Sum matrix : in, , , It is the weight matrix for learning; Self-attention determines an attention score by calculating the similarity between the query and the key. : in, It is the dimension of the key matrix. It is a normalization operation used to obtain the attention weights for each position.

[0024] Based on attention score Sum matrix The weighted feature map is obtained as follows: .

[0025] t2 and CBAM employ two attention mechanisms: channel attention and spatial attention, to weight the image's channels and spatial dimensions respectively. The channel attention mechanism aims to learn the importance of each channel and dynamically adjust its weights. Specifically, it first performs global average pooling and global max pooling on the input feature map to obtain a channel-level description. Then, a small fully connected network is used to compute the weights of each channel, generating attention weights for each channel using a sigmoid activation function. in, Indicates the input feature map, This represents the sigmoid activation function. This represents a multilayer perceptron. and These represent global average pooling and max pooling operations, respectively. Spatial attention weights are generated using convolution operations: in, This represents the convolution operation. This means concatenating the pooled feature maps to generate attention weights in the spatial dimension. t3, for each head Calculate attention scores using different queries, keys, and values: in, , , It is the first The query, key, and value matrix of each head; The outputs of each head are concatenated and a linear transformation is applied to obtain the final output: in, It refers to the number of heads. It is a linear transformation matrix.

[0026] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for fault diagnosis of small-sample motors using infrared images, characterized in that: The diagnostic method specifically includes the following steps: S1. Load the original motor infrared image dataset and quickly expand the original motor infrared image dataset in the early stage of training using the Mosaic data augmentation method. S2. Establish a self-attention-enhanced generative adversarial network AE-DCGAN to enhance the infrared motor image dataset and generate the final training samples. S3. Employ dynamic task-weighted meta-learning to dynamically adjust the weights of different tasks and pay attention to the correlation between tasks; S4. Use the DeepLabV3+ model to perform semantic segmentation of the fault region, obtain the motor infrared image fault diagnosis model, and build an adaptive multi-scale fusion module. S5. Perform fault diagnosis; Step S2 specifically includes: b1, Generator Through random noise Generate enhanced images; b2, Discriminator To determine whether an input image is a real image: The discriminator uses a convolutional neural network structure and focuses on key regions in the image through a convolutional attention mechanism. The discriminator process is as follows: in, Indicates the discriminator's effect on the image The judgment, This represents the Sigmoid activation function. This indicates that a convolution operation is performed on the image; b3. Use the variance of the feature space to represent the diversity of the image: in, This represents the feature vector extracted from the generated image through a pre-trained network. Indicates variance; b4. Introduce a metric for the response intensity of the fault region: in, This indicates the attention map or feature map in the fault region. The response intensity; b5. Combine the objective functions from steps b2, b3, and b4 and optimize the generator parameters. : ； The loss of the discriminator is defined as: in, This indicates that the discriminator recognizes the real image. The discrimination probability, This represents the probability that the discriminator classifies the generated image. b6. Use alternating optimization of the generator and discriminator. The generator loss is minimized through backpropagation, and the discriminator loss is minimized through backpropagation. in, Represents the loss function of the generator. This represents the loss function of the discriminator; b7. The model outputs the enhanced image as follows: in, This represents the generated image. The input is random noise; The final dataset is as follows: ； In step S2: the network hierarchy of the generator G of AE-DCGAN is as follows: input layer, hidden layer 1, hidden layer 2, attention mechanism layer, hidden layer 3, hidden layer 4, output layer; the attention mechanism layer of the generator G of AE-DCGAN includes a self-attention mechanism block, a convolutional block attention module, and a multi-head attention mechanism block; the discriminator D network of AE-DCGAN is a 5-layer pyramid structure from bottom to top.

2. The method for diagnosing faults in small-sample motors using infrared images according to claim 1, characterized in that: Step S1 specifically includes: a1. From the original motor infrared image dataset Four images are randomly selected from the data. ; a2, Yes Perform random cropping and scaling operations to obtain the scaled image. The dimensions are recorded as : in, Indicates the image Perform random cropping. It performs scaling operations on the cropped image; a3. After cropping and scaling Images stitched together to form a new image The splicing method is to... Four images are placed at the four corners of the matrix to generate a new image: in, This indicates four images The images are stitched together according to the specified stitching order to obtain a new image. The dimensions are recorded as ; a4. Repeat steps a1, a2, and a3 to generate several stitched images. Expand the training dataset to generate a new sample set. This will include data from the original motor infrared image dataset. Enhanced image generated in the process, set to generate The new sample, the enhanced motor infrared image dataset, is represented as follows: 。 3. The method for diagnosing faults in small-sample motors using infrared images according to claim 1, characterized in that: Step S3 specifically includes: c1, set with Each of the different tasks Corresponding to a different type of fault: c2. Configure each task The weight is Weight Based on the task's error Dynamic adjustment, the specific formula is: in, Indicates task The error, This represents the balancing parameter, used to control the impact of task errors on the weights; The task error is obtained by calculating the loss function for each task. : in, Indicates task loss function, Indicates task The dataset, Represents the parameters of the model; c3. Inter-task correlation Through computation task and The cosine similarity between them is used to obtain: in, Indicates task Features Indicates task Features express The model, express The model; c4. Design a reinforcement learning framework for task weight adjustment, including the error of the current task and the performance metrics of the model. Weighted actions assigned to each task The reward value is given based on the learning progress of the task and the overall model performance. , in, This indicates a change in segmentation accuracy. It is a regulatory factor. Indicates the change in meta-learning loss; The agent updates the task weight allocation strategy through Q-learning: in, It's the learning rate. It is a discount factor. That's the next step.

4. The method for diagnosing faults in small-sample motors using infrared images according to claim 1, characterized in that: Step S4 specifically includes: d1, Input Image Temperature distribution from infrared images of the motor is used for subsequent fault diagnosis. in, Indicates the first Zhang input image, size is ; d2. In DeepLabV3+, the input image is first processed by convolution to extract basic features, and the image size is adjusted by stride. Then, an activation function is used to enhance non-linear features. Finally, dilated convolution is used to expand the receptive field and capture the range information of the image. The specific process includes: in, This represents the feature map after convolution. It is a convolution kernel with a size of 3×3; This indicates a dilated convolution kernel with a dilation rate of 6. This represents the feature map after dilated convolution, with a size of [size missing]. ; d3. Calculate the weighting coefficients based on the activation level of each feature map. Weight It is inversely proportional to the activation level of the feature map: d4. Fuse feature maps of different scales using a weighted summation method: d5. After optimization through feature fusion and attention mechanism, the model outputs the segmentation result of the fault region through the decoder and Softmax layer. The segmentation result indicates whether each pixel belongs to the normal region or the fault region.

5. The method for diagnosing faults in small-sample motors using infrared images according to claim 1, characterized in that: The specific workflow of the attention mechanism layer of the generator G in the AE-DCGAN includes: t1, Given the input feature map First, the query matrix is ​​obtained through convolution operation. Key matrix Sum matrix : in, , , It is the weight matrix for learning; Self-attention determines an attention score by calculating the similarity between the query and the key. : in, It is the dimension of the key matrix. It is a normalization operation used to obtain the attention weights at each position; Based on attention score Sum matrix The weighted feature map is obtained as follows: 。 t2. Generate attention weights for each channel using the sigmoid activation function: in, Indicates the input feature map, This represents the sigmoid activation function. This represents a multilayer perceptron. and These represent global average pooling and max pooling operations, respectively. Spatial attention weights are generated using convolution operations: in, This represents the convolution operation. This means concatenating the pooled feature maps to generate attention weights in the spatial dimension. t3, for each head Calculate attention scores using different queries, keys, and values: in, , , It is the first The query, key, and value matrix of each head; The outputs of each head are concatenated and a linear transformation is applied to obtain the final output: in, It refers to the number of heads. It is a linear transformation matrix.

Images