A pneumonia X-ray image analysis model and analysis method based on attention superposition and feature fusion

Abstract

The present invention discloses a pneumonia X-ray image analysis model and analysis method based on attention superposition and feature fusion, which uses a spatial attention superposition network to highlight the characteristics of lesions and improve the accuracy of attention guidance. First, the spatial attention value of the entire channel is calculated to obtain the initial attention score. Then, CBL and encoding and decoding operations are performed on the two channels respectively to obtain the attention score. Finally, the two attentions are multiplied with the initial attention. The initial attention assists the generation of the two parts of attention and strengthens their respective feature maps. In addition, this paper proposes a new feature fusion network, which obtains unique representations on their respective features through two channels with different representation capabilities, and can simultaneously improve the "accuracy" and "diversity" of the features.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a pneumonia X-ray image analysis model and analysis method based on attention superposition and feature fusion. Background Art

[0002] Establishing an image analysis model is the primary means of analyzing X-ray images. The main task of pneumonia detection is to extract features from the input lung medical images to obtain the region of interest (ROI). The extracted features are usually a complete description of the CXR, and distinguishing between pneumonia and non-pneumonia based on the extracted features is the difficulty of pneumonia detection. CheXNet proposed by Andrew Ng's team, consisting of a 121-layer DenSenet network, can classify 14 lung diseases, but this work cannot predict the location information of the lesion and can only obtain the probability of illness. In 2019, Hao Bing used Retinanet and Mask-RCNN, using single-stage and two-stage model training to improve the detection performance of pneumonia. Dmytro Poplavskiy et al. used the RetineNet SSD network model to detect pneumonia, added a global classification output to the model, and significantly expanded the dataset. Although this improved the model accuracy, the accuracy rate was only 24.781%. Jaiswal et al. used Mask R-CNN to effectively identify and localize pneumonia areas by adjusting thresholds during training and fusing prediction results. However, the need to adjust thresholds during training complicates the training process. Eunji et al. compared features of input images with prototypes to diagnose diseases. While they were able to provide global, prototype, and local explanations for the diagnostic results, their interpretability was still limited. Although these methods have achieved some success in pneumonia detection, they still suffer from issues such as large model parameters, long training times, and low detection accuracy.

[0003] Pneumonia X-ray images are very different from traditional target detection. Traditional target detection objects are mostly clear and colorful, while pneumonia lesion detection objects are mostly blurred and grayscale. Therefore, a special model should be designed for pneumonia X-ray image detection to improve the accuracy of model detection. Summary of the Invention

[0004] The purpose of the present invention is to provide a pneumonia X-ray image analysis model based on attention superposition and feature fusion to address the technical defects existing in the prior art.

[0005] Another object of the present invention is to provide a pneumonia X-ray image analysis method based on attention superposition and feature fusion.

[0006] The technical solution adopted to achieve the purpose of the present invention is:

[0007] A pneumonia X-ray image analysis model based on attention superposition and feature fusion, including a YOLOv5 backbone network model, a SasNet spatial attention superposition network model, an MfcNet multi-layer feature fusion network model, and a YOLOv5 prediction network model, wherein:

[0008] The YOLOv5 backbone network model extracts the feature representation of the target from the pneumonia X-ray sample data to obtain the backbone network features;

[0009] The SasNet spatial attention superposition network model includes an attention guidance module, a first channel feature calculation module, a second channel feature calculation module and two attention guidance superposition modules. The attention guidance module calculates the spatial attention value of the backbone network feature once to obtain the initial attention score a. i The first channel feature calculation module and the second channel feature calculation module perform CBL and encoding and decoding operations respectively to obtain an intermediate feature map. In the attention guidance superposition module, the intermediate feature map is used to calculate the attention score c i Respectively with the initial attention score a i Multiply them together to get the attention value a′ i , two sets of attention values a′ i Merge to get the feature map Out;

[0010] The MfcNet multi-layer feature fusion network model includes a channel attention module and a feature fusion module. The weight ω of each channel is first obtained by the channel attention module. In the feature fusion module, the channel is divided into a positive channel and a negative channel according to the importance and contribution of the weight ω of each channel to the final classification result. The negative channel is averaged and then fused with the positive channel features. Finally, a new feature map Out′ is output through the residual.

[0011] The YOLOv5 prediction network model includes a classification network model and a positioning regression network model. The classification network model classifies the new feature map, and the positioning regression network model locates the new feature map.

[0012] In the above technical solution, the attention guidance module combines the maximum value and average value of the backbone network features to obtain c i , after convolution and Sigmoid activation, the initial attention score a is obtained i , c i =[max(x i ), mean(x i )],a i =S(C(c i)), i is the position index, representing the vector value of all channels at position i, max represents the maximum value, mean represents the average value, [] is the splicing operation, C represents the convolution operation, and S represents the Sigmoid operation.

[0013] In the above technical solution, the attention value a′ obtained by the attention guidance superposition module i It can be expressed as:

[0014] a′ i =S(C(a i ×c i ))

[0015] The two sets of attention values are combined to obtain the feature map Out = [a′ i ×o i , x i ]

[0016] Among them i is the intermediate feature map calculated by the first channel feature calculation module and the second channel feature calculation module, x i is the original input feature.

[0017] In the above technical solution, in the channel attention module, after global average pooling, a feature map y of size 1×1×C is generated. The kernel size k is nonlinearly proportional to the channel dimension C, and γ×k+b is the simplest linear mapping. The channel dimension C is usually set to an integer power of 2, so:

[0018] C=φ(k)=2 (γ×k+b)

[0019]

[0020] C is the channel dimension, k is the kernel size, |t| odd Indicates selecting the nearest odd number, γ = 2, b = 1;

[0021] The kernel size k is adaptively determined by the channel dimension C, and then applied to the one-dimensional convolution to obtain the weight ω of each channel. The calculation formula of the weight can be expressed as:

[0022] ω=S(C1D k (y))

[0023] C1D stands for one-dimensional convolution. S stands for Sigmoid operation.

[0024] In the above technical solution, in the feature fusion module, the feature map is divided into positive channels and negative channels. The segmentation rules are as follows:

[0025]

[0026] Then, the eigenvalues of the positive channel and the feature mean of the negative channel are concatenated and output through the residual. The fusion formula can be expressed as:

[0027]

[0028] Among them, p is the eigenvalue of the positive channel, is the feature average of the negative channel, [] represents the concatenation operation, and x represents the original feature.

[0029] In the above technical solution, the classification network model uses BCEclsloss as the loss function of Classification Loss for classification.

[0030] In the above technical solution, BCEclsloss is given by the formula Calculate, where M is the number of categories, y ic is a symbolic function that takes 1 if the true category of sample i is equal to c, otherwise it takes 0, and pic is the predicted probability that the observed sample i belongs to category c.

[0031] In the above technical solution, the positioning regression network model GIOU_Loss is used as the loss function of the Bounding box for positioning.

[0032] In the above technical solution, GIOU_Loss can be calculated by the following formula:

[0033]

[0034]

[0035] L GIoU =1-GIoU

[0036] Among them, A and B are two arbitrary bboxes, and C is the minimum enclosing rectangle of A and B.

[0037] Another aspect of the present invention further includes a pneumonia X-ray image analysis method based on attention superposition and feature fusion, comprising the following steps:

[0038] Step 1: preprocess and enhance the data set consisting of pneumonia X-ray samples to obtain preprocessed and enhanced sample data;

[0039] Step 2: Processing the sample data using the pneumonia X-ray image analysis model to train the model;

[0040] Step 3: Input the pneumonia X-ray to be diagnosed and analyze it using the model trained in step 2.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. This paper uses Mosaic data augmentation to enrich the dataset and reduce overfitting. Secondly, it uses the BackBone component of YOLOv5 as the backbone network for feature extraction, enhancing the learning ability of CNN. This allows for lightweighting while maintaining accuracy and reducing computational bottlenecks and memory costs.

[0043] 2. This paper uses a spatial attention superposition network to highlight important features and a multi-layer feature fusion network to pick up more valuable features, further improving network performance based on the attention network.

[0044] 3. In the prediction part, this invention uses BCEclsloss as the loss function of Classification Loss; in the positioning regression network, GIOU_Loss is used as the loss function of Bounding box to further improve the accuracy of image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is the SasNet structure diagram;

[0046] Figure 2 It is the attention guidance module;

[0047] Figure 3 This is the MfcNet structure diagram. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] Example 1

[0050] This paper proposes an artificial intelligence pneumonia detection algorithm based on attention superposition and feature fusion. It uses a spatial attention superposition network to highlight lesion features and improve attention guidance accuracy.

[0051] A pneumonia X-ray image analysis model based on attention superposition and feature fusion, including a YOLOv5 backbone network model, a SasNet spatial attention superposition network model, an MfcNet multi-layer feature fusion network model, and a YOLOv5 prediction network model, wherein: first, the spatial attention value of the entire channel is calculated once, and the YOLOv5 backbone network model extracts the feature representation of the target from the pneumonia X-ray sample data to obtain the backbone network feature; then CBL and encoding and decoding operations are performed on the two channels respectively to obtain the attention score. Finally, the two attentions are multiplied with the initial attention. The initial attention assists the generation of the two parts of attention and strengthens their respective feature maps. The SasNet spatial attention superposition network model includes an attention guidance module, a first channel feature calculation module, a second channel feature calculation module, and two attention guidance superposition modules. The attention guidance module calculates the spatial attention value of the backbone network feature once to obtain the initial attention score a i The first channel feature calculation module and the second channel feature calculation module perform CBL and encoding and decoding operations respectively to obtain an intermediate feature map. In the attention guidance superposition module, the intermediate feature map is used to calculate the attention score c i Respectively with the initial attention score a i Multiply them together to get the attention value a′ i , two sets of attention values a′ i The feature map Out is obtained by merging; the MfcNet multi-layer feature fusion network model includes a channel attention module and a feature fusion module. The weight ω of each channel is first obtained through the channel attention module. In the feature fusion module, the channel is divided into a positive channel and a negative channel according to the importance and contribution of the weight ω of each channel to the final classification result. The negative channel is averaged and then fused with the positive channel features, and finally a new feature map Out′ is output through the residual; the YOLOv5 prediction network model is a classification network model and a positioning regression network model. The classification network model classifies the new feature map, and the positioning regression network model locates the new feature map.

[0052] In this embodiment, a new feature fusion network is proposed, which obtains unique representations on each feature through two channels with different representation capabilities, and can simultaneously improve the "accuracy" and "diversity" of features.

[0053] Example 2

[0054] In order to improve the network's ability to identify pneumonia, the lesion area is determined. Figure 1As shown in Figure 2, the network primarily consists of two modules: channel feature calculation and attention guidance. The channel feature calculation module splits the input channel features into two parts and performs feature extraction on each to obtain higher-dimensional features. The attention guidance module uses a simple spatial attention calculation to highlight important features. The attention guidance module is described in detail below.

[0055] The attention guidance module combines the maximum and average values of the backbone network features to obtain c i , after convolution and Sigmoid activation, the initial attention score a is obtained i ,

[0056] c i =[max(x i ), mean(x i )] (1)

[0057] a i =S(C(c i )) (2)

[0058] i is the position index, representing the vector value of all channels at position i, max represents the maximum value, mean represents the average value, [] is the splicing operation, C represents the convolution operation, and S represents the Sigmoid operation.

[0059] As a preference, the attention value a′ obtained by the attention guidance superposition module i It can be expressed as:

[0060] a′ i =S(C(a i ×c i )) (3)

[0061] The two sets of attention values are combined to obtain the feature map Out = [a′ i ×o i , x i ] (4)

[0062] Among them i is the intermediate feature map calculated by the first channel feature calculation module and the second channel feature calculation module, x i is the original input feature.

[0063] A good feature fusion network should meet the following two conditions: accuracy and diversity. Accuracy means that the majority of input features should contribute to the final output, while diversity means that different input features should differ from each other. If the differences in input features are consistent and accurate, multi-layer feature fusion can achieve better results. Therefore, this article refers to feature layers containing more information as active channels, and conversely, feature layers containing less information as passive channels. While ensuring feature diversity, we operate on both channels separately to improve the detection recall rate.

[0064] In summary, this paper proposes a multi-layer feature fusion network MfcNet. Unlike simple feature fusion, after effectively learning spatial attention, the model divides the channel into a positive channel and a negative channel based on the importance and contribution of each feature layer to the final classification result. Since the features of the negative channel have little significance for the result, they are averaged and then fused with the features of the positive channel. The final result is output through the residual. It can maximize the "accuracy" and "diversity" of features. For details, see Figure 3 The structure diagram of MfcNet.

[0065] To ensure computational performance and model complexity, in the channel attention module, after global average pooling (GAP), a feature map y of size 1×1×C is generated. The kernel size k is nonlinearly proportional to the channel dimension C, and γ×k+b is the simplest linear mapping. The channel dimension C is usually set to an integer power of 2, so:

[0066] C=φ(k)=2 (γ×k+b) (5)

[0067]

[0068] C is the channel dimension, k is the kernel size, |t| odd Indicates selecting the nearest odd number, γ = 2, b = 1;

[0069] Therefore, through the channel dimension C, the kernel size k can be adaptively determined, and then applied to the one-dimensional convolution to obtain the weight ω of each channel. The calculation formula of the weight can be expressed as:

[0070] ω=S(C1D k (y)) (7)

[0071] Here, C1D stands for one-dimensional convolution and S stands for Sigmoid operation.

[0072] As a preference, in the feature fusion module, as Figure 3 As shown, the feature map is divided into positive channels and negative channels (see Figure 3The segmentation rules are as follows:

[0073]

[0074] Then the eigenvalues of the positive channel and the feature mean of the negative channel are concatenated and output through the residual. According to the above analysis, the fusion formula can be expressed as

[0075]

[0076] Among them, p is the eigenvalue of the positive channel, is the feature average of the negative channel, [] represents the concatenation operation, and x represents the original feature.

[0077] Preferably, the classification network model uses BCEclsloss as the loss function of Classification Loss for classification.

[0078] BCEclsloss is given by the formula calculate,

[0079] Where M is the number of categories, y ic is a symbolic function that takes 1 if the true category of sample i is equal to c, otherwise it takes 0, and pic is the predicted probability that the observed sample i belongs to category c.

[0080] The positioning regression network model GIOU_Loss is used as the loss function of the Bounding Box for positioning. GIOU_Loss can be calculated by the following formula:

[0081]

[0082]

[0083] L GIoU =1-GIoU (13)

[0084] Among them, A and B are two arbitrary bboxes, and C is the minimum enclosing rectangle of A and B.

[0085] Example 3

[0086] A pneumonia X-ray image analysis method based on attention superposition and feature fusion includes the following steps:

[0087] Step 1: preprocess and enhance the data set consisting of pneumonia X-ray samples to obtain preprocessed and enhanced sample data;

[0088] Mosaic data augmentation: Four images are selected and spliced together using random scaling, cropping, and arrangement. This increases the number of small objects and improves network robustness. This also reduces GPU usage. During Mosaic augmentation training, data for the four images can be directly computed, reducing the need for a large mini-batch size. A single GPU can achieve good results.

[0089] Step 2: Processing the sample data using the pneumonia X-ray image analysis model to train the model;

[0090] Step 3: Input the pneumonia X-ray to be diagnosed and analyze it using the model trained in step 2.

[0091] Further applications include building a medical image-assisted diagnosis system using an AI-powered pneumonia detection algorithm based on attention superposition and feature fusion. A medical imaging scanning system captures patient images and transmits them to the medical image-assisted diagnosis system, where the AI-powered pneumonia detection algorithm, using attention superposition and feature fusion, performs detection and analysis on the backend terminal.

[0092] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A pneumonia X-ray image analysis model based on attention superposition and feature fusion, characterized by: It includes the YOLOv5 backbone network model, the SasNet spatial attention superposition network model, the MfcNet multi-layer feature fusion network model, and the YOLOv5 prediction network model, among which: The YOLOv5 backbone network model extracts the feature representation of the target from the pneumonia X-ray sample data to obtain the backbone network features; The SasNet spatial attention superposition network model includes an attention guidance module, a first channel feature calculation module, a second channel feature calculation module and two attention guidance superposition modules. The attention guidance module calculates the spatial attention value of the backbone network feature once to obtain the initial attention score a. i The first channel feature calculation module and the second channel feature calculation module perform CBL and encoding and decoding operations respectively to obtain an intermediate feature map. In the attention guidance superposition module, the intermediate feature map is used to calculate the attention score c i Respectively with the initial attention score a i Multiply them together to get the attention value a′ i , two sets of attention values a′ i Merge to get the feature map Out; The MfcNet multi-layer feature fusion network model includes a channel attention module and a feature fusion module. The weight ω of each channel is first obtained by the channel attention module. In the feature fusion module, the channel is divided into a positive channel and a negative channel according to the importance and contribution of the weight ω of each channel to the final classification result. The negative channel is averaged and then fused with the positive channel features. Finally, a new feature map Out′ is output through the residual. The YOLOv5 prediction network model includes a classification network model and a positioning regression network model. The classification network model classifies the new feature map, and the positioning regression network model locates the new feature map.

2. The pneumonia X-ray image analysis model based on attention superposition and feature fusion according to claim 1, characterized in that: The attention guidance module combines the maximum and average values of the backbone network features to obtain c i , after convolution and Sigmoid activation, the initial attention score a is obtained i , c i =[max(x i ),mean(x i )],a i =S(C(c i ), i is the position index, x i Represents the vector value of all channels at position i, max represents the maximum value, mean represents the average value, [] is the splicing operation, C represents the convolution operation, and S represents the Sigmoid operation.

3. The pneumonia X-ray image analysis model based on attention superposition and feature fusion according to claim 2, characterized in that: The attention value a′ obtained by the attention guidance superposition module i It can be expressed as: a′ i =S(C(a i ×c i )) The two sets of attention values are combined to obtain the feature map Out = [a′ i ×o i , x i ] Among them i is the intermediate feature map calculated by the first channel feature calculation module and the second channel feature calculation module, x i is the original input feature.

4. The pneumonia X-ray image analysis model based on attention superposition and feature fusion according to claim 3, characterized in that: In the channel attention module, after global average pooling, a feature map y of size 1×1×C is generated. The kernel size k is nonlinearly proportional to the channel dimension C, and γ×k+b is the simplest linear mapping. The channel dimension C is usually set to an integer power of 2, so: C=φ(k)=2 (γ×k+b) C is the channel dimension, k is the kernel size, |t| odd Indicates selecting the nearest odd number, γ = 2, b = 1; The kernel size k is adaptively determined by the channel dimension C, and then applied to the one-dimensional convolution to obtain the weight ω of each channel. The calculation formula of the weight can be expressed as: ω=S(C1D k (y)) C1D stands for one-dimensional convolution, and S stands for Sigmoid operation.

5. The pneumonia X-ray image analysis model based on attention superposition and feature fusion according to claim 4, characterized in that: In the feature fusion module, the feature map is divided into positive channels and negative channels. The segmentation rules are as follows: Then, the eigenvalues of the positive channel and the feature mean of the negative channel are concatenated and output through the residual. The fusion formula can be expressed as: Among them, p is the eigenvalue of the positive channel, is the feature average of the negative channel, [] represents the concatenation operation, and x represents the original feature.

6. The pneumonia X-ray image analysis model based on attention superposition and feature fusion according to claim 1, characterized in that: The classification network model uses BCEclsloss as the loss function of Classification Loss for classification.

7. The pneumonia X-ray image analysis model based on attention superposition and feature fusion according to claim 6, characterized in that: BCEclsloss is given by the formula Calculate, where M is the number of categories, y ic is a sign function, if the true category of sample i is equal to c, it takes 1, otherwise it takes 0, p ic is the predicted probability that the observed sample i belongs to category c.

8. The pneumonia X-ray image analysis model based on attention superposition and feature fusion according to claim 1, characterized in that: The positioning regression network model GIOU_Loss is used as the loss function of the Bounding box for positioning.

9. The pneumonia X-ray image analysis model based on attention superposition and feature fusion according to claim 8, characterized in that: GIOU_Loss is calculated by the following formula: L GIoU =1-GIoU Among them, A and B are two arbitrary bboxes, and C is the minimum enclosing rectangle of A and B.

10. A pneumonia X-ray image analysis method based on attention superposition and feature fusion, characterized in that: The following steps are involved: Step 1: Preprocess and enhance the dataset consisting of pneumonia X-ray samples to obtain preprocessed and enhanced sample data, and use Mosaic data enhancement to perform splicing preprocessing in a random scaling, cropping, and arrangement manner; Step 2, processing the sample data using the pneumonia X-ray image analysis model as described in claims 1-8 to train the model; Step 3: Input the pneumonia X-ray to be diagnosed and analyze it using the model trained in step 2.

Images