Multi-scale parallel dual-channel pneumoconiosis lesion classification and recognition method

Abstract

This paper discloses a multi-scale parallel dual-channel method for identifying and grading pneumoconiosis lesions. The specific implementation process is as follows: the collected original X-ray pneumoconiosis image data is staged; a multi-scale attention mechanism is used to perform feature grouping and parallel convolution operations to enhance the model's receptive field and generate a corresponding attention weight map; the feature map extracted by parallel convolution is combined with a detail encoder with a residual structure to capture detailed texture feature information; to further grasp the invisible features in the feature map, a parallel dual-channel network module is constructed to effectively embed implicit local and global information and collaboratively capture pixel-level semantic information in high-frequency components; finally, transfer learning and multi-scale convolution are used to fuse the different channel information in the feature map and input it into the detail encoder and Transformer layer stack respectively to identify the pneumoconiosis lesion grade. Experimental results show that this method has higher recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the technical fields of convolutional neural networks and pneumoconiosis lesion graded detection and identification, and specifically to a multi-scale parallel dual-channel pneumoconiosis lesion graded identification method, which can be used in the field of pneumoconiosis lesion graded detection. Background Art

[0002] With the acceleration of industrialization, the incidence of occupational diseases such as pneumoconiosis has gradually increased, making early diagnosis particularly critical. Traditional pneumoconiosis detection methods mainly rely on imaging techniques (such as X-ray examinations and CT scans) and manual diagnosis. These methods have the problems of low recognition accuracy, high manual intervention, and difficulty in full automation. Deep learning, as an efficient data-driven method, has demonstrated excellent performance in the field of medical image analysis in recent years. It can automatically extract lesion features from large amounts of medical image data, overcoming the limitations of traditional methods. However, traditional image analysis methods often face problems such as difficulty in feature extraction and poor generalization ability when processing multi-modal image data, resulting in unstable diagnostic results. To address these issues, the present invention proposes a novel dual-channel pneumoconiosis lesion classification recognition model with a multi-scale, heterogeneous approach. This method utilizes a multi-scale attention mechanism to perform feature grouping and parallel convolution operations, enhancing the model's receptive field, enabling it to fully extract lesion features from lung X-rays. After feature extraction, the features are input into a detail encoder and a dual-channel based on a stack of Transformer layers to capture detailed texture information and fuse local and global information, ultimately achieving comprehensive information integration and recognition of pneumoconiosis lesion classification. Summary of the Invention

[0003] In order to solve the problems of traditional medical image diagnosis methods that are difficult to extract comprehensive lesion features and have relatively weak generalization capabilities, the present invention builds a parallel convolution multi-scale convolution mechanism based on traditional convolutional neural networks to enhance the model's receptive field, form an attention weight map, and a parallel dual-channel collaborative mechanism. Then, a detail encoder and a Transformer stack are used to fuse local and global information at different levels through multi-channels to achieve accurate detection and diagnosis of pneumoconiosis. The specific implementation steps of the present invention are as follows:

[0004] 1. Use the multi-scale attention mechanism to construct the attention weight map. The steps are as follows:

[0005] (1a) Feature grouping:

[0006] Different channels are designed as batch dimensions, 1×1 and 3×3 parallel convolution operations are performed, and the channel dimensions are grouped into multiple sub-features. For a given input feature map X∈R C×H×W , the feature map is divided into G sub-features in the channel dimension, and the pattern within the group is X=[X0,X i,...,X G-1 ], X i =R C / / G×H×W .

[0007] (1b) Construct the attention weight map:

[0008] The global spatial information is encoded in the output of the 1×1 branch using 2D global average pooling, and the output of the smallest branch is directly converted to the required unique shape before the channel feature joint activation mechanism, i.e. By multiplying the convolution output of the parallel path with the matrix dot product operation, the weighted attention map is obtained. The comprehensive formula of 2D global average pooling is as follows:

[0009]

[0010] 2. Use the detail encoder to capture the image detail texture feature information. The steps are as follows:

[0011] (2a) Using inverse neural network to extract specific features of pneumoconiosis:

[0012] The features of a given input are represented as I∈R H×W and V∈R 3×H×W , the comprehensive formula of the feature representation extracted by the encoder is expressed as:

[0013]

[0014] Among them, E D (·) indicates the detail encoder.

[0015] (2b) Enhance detail texture features using interactive normalized networks:

[0016] By multiplying the attention weight map element by element to fuse the features from different paths, and adding them element by element to aggregate the features, through the convolution layer, ReLU activation function, batch normalization and maximum pooling layer, more critical and advanced detail texture features are extracted from the high-frequency components of the X-ray chest film, thereby enhancing the detail texture features of the pneumoconiosis image data.

[0017] 3. Use the Transformer layer stack to achieve the fusion of global and local feature information. The steps are as follows:

[0018] (3a) Using the Transformer layer stack to achieve the fusion of global and local feature information:

[0019] For a given input variable X∈R C×H×W , first apply an n×n standard convolution layer to encode local spatial information, followed by a point-wise convolution layer to generate the output variable XL ∈R H×W×d In order to fully capture the global feature information including spatial inductive bias, the present invention will input X L Decompose into N non-overlapping planar blocks X U ∈R P×N×d , P=WH, P is the number of patches, h≤n and w≤n are the height and width of the patch respectively. For each patch p∈{1,...,P}, the relationship between patches is encoded by applying a stack of Transformer layers (using Swish as activation function), (1) unfolding, (2) matrix multiplication to learn local representations, and (3) folding back to the output form for classification to obtain a fused feature representation X G ∈R d×P×N :

[0020] X G (P) = Transformer(X U (p)),1≤p≤P

[0021] 4. Use the transfer learning method for pre-training and model weight migration. The steps are as follows:

[0022] (4a) Transfer learning method for pre-training and model weight transfer:

[0023] First, the model is trained using a dataset of common lung diseases, and the optimal model weight coefficients are saved. Second, the weights are fine-tuned based on the characteristics of transfer learning and transferred to a dataset of pneumoconiosis grade classification for detection and diagnosis of pneumoconiosis grade classification.

[0024] The method of the present invention has the following advantages:

[0025] (1) The present invention can efficiently extract specific features from pneumoconiosis images and enhance them through a dual-channel fusion mechanism. This mechanism can effectively integrate information from each local area, significantly improve the deficiencies of feature representation, and enhance the robustness of the method and the stability of pneumoconiosis lesion grading detection.

[0026] (2) This invention introduces a detail encoder that can deeply learn the detailed texture feature information in pneumoconiosis images and reveal the distinguishing features between different severity levels. This mechanism can capture more comprehensive and accurate lesion information, thereby effectively improving the accuracy and reliability of pneumoconiosis lesion grading detection.

[0027] (3) The present invention proposes a dual-channel fusion detail encoder and Transformer module to aggregate information from different angles. This module can not only effectively integrate information from each severity category, but also alleviate the phenomenon of excessive feature loss, thereby more accurately capturing the connection characteristics between global and local features. The transfer learning method is also introduced, which enhances the richness of image feature representation and improves classification accuracy, thereby achieving more accurate detection of pneumoconiosis lesion levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is the overall framework diagram of the present invention

[0029] Figure 2 This is the confusion matrix classification effect diagram of a random experiment on a common lung disease dataset

[0030] Figure 3 This is the confusion matrix classification effect diagram of the random experiment on the pneumoconiosis grade classification dataset DETAILED DESCRIPTION

[0031] The specific implementation steps of the present invention are as follows:

[0032] 1. The original chest X-ray image and the labels provided by the radiologist are used as input, and 1×1 and 3×3 parallel convolution operations are performed. The channel dimension is grouped into multiple sub-features so that the spatial semantic features are evenly distributed in each feature group.

[0033] 2. Based on the feature grouping, the global spatial information is encoded in the output of the 1×1 branch using 2D global average pooling, and the output of the smallest branch is directly converted to the required unique shape before the channel feature joint activation mechanism, that is, By multiplying the convolution output of the parallel path with the matrix dot product operation, a weighted attention map containing accurate spatial position information is obtained. The comprehensive formula for 2D global average pooling is as follows:

[0034]

[0035] Finally, the output feature map within each group is obtained by computing a set of two generated spatial attention weights, followed by applying a sigmoid function to capture pixel-level pairwise relationships and enhance the global contextual information of all image pixels.

[0036] 3. Use the detail encoder built based on the reversible neural network to capture the features of the given input as I∈R H×W and V∈R 3×H×W , the comprehensive formula of the feature representation extracted by the encoder is expressed as:

[0037]

[0038] Among them, E D (·) indicates the detail encoder.

[0039] The information aggregation features are further integrated by performing element-by-element multiplication on the input weight map and element-by-element addition at the same time. The convolution block extracts more abstract and advanced detail texture features from the high-frequency components of the X-ray chest film through a multi-level interactive normalized network structure, and finally outputs the detailed texture feature results fused by multiple layers of convolution blocks.

[0040] 4. Use the Transformer layer stack to achieve the fusion of global and local feature information, and input X L Decompose into N non-overlapping planar blocks X U ∈R P×N×d , P=WH, P is the number of patches, h≤n and w≤n are the height and width of the patch respectively. For each patch p∈{1,...,P}, the relationship between patches is encoded by applying a stack of Transformer layers (using Swish as activation function), (1) unfolding, (2) matrix multiplication to learn local representations, and (3) folding back to the output form for classification to obtain a fused feature representation X G ∈R d×P×N :

[0041] X G (P) = Transformer(X U (p)),1≤p≤P

[0042] X G ∈R P×N×d Fold to X F ∈R P×N×d , and then project it into the low-dimensional space C using point-by-point convolution and concatenate it with the input X. Finally, these cascade features are fused through a 3×3 convolution layer to obtain the detection result.

[0043] 5. Use the transfer learning method for pre-training to save the weights and transfer fine-tuning to apply to the grading detection and recognition of pneumoconiosis lesions.

[0044] The effect of the present invention is further verified by the following experiments:

[0045] The experiment was validated on a dataset of common lung diseases and pneumoconiosis classification. The dataset included 2008 cases of viral pneumonia, 2009 cases of bacterial pneumonia, 1546 cases of pneumothorax, 2034 cases of tuberculosis, 2031 cases of coronavirus pneumonia, and 2013 cases of health data extracted using Chest X-ray 8. Furthermore, 1248 cases of pneumoconiosis were collected from the Affiliated Hospital of Southwest Medical University, encompassing a total of seven common lung diseases. The proposed method achieved an average accuracy of 0.9025 in randomized experiments on the dataset, demonstrating good stability.

[0046] The dataset for the classification of pneumoconiosis grades was derived from chest X-ray image data from the Affiliated Hospital of Southwest Medical University. A database of 1,760 anonymous digital radiographs (DR) posteroanterior chest radiographs (CXRs) showing evidence of pneumoconiosis was generated from January 2017 to June 2020. The inclusion criteria for this dataset are as follows: (1) all patients with a history of dust exposure; (2) patients with a good chest radiograph, or at least an acceptable quality that meets the International Labor Organization guidelines; and (3) all positive cases have a diagnosis certificate from a pneumoconiosis diagnosis unit. All patients were divided into four categories, namely 0 (n=512), 1 (n=352), 2 (n=478), and 3 (n=418). The average accuracy of the present invention in a randomized experiment on the pneumoconiosis grade classification dataset reached 0.7771, confirming the effectiveness of this invention.

Claims

1. A multi-scale, parallel, dual-channel pneumoconiosis lesion classification and identification method, comprising the following steps: (1) Using the multi-scale attention mechanism to group features and construct the attention weight map includes the following steps: (1a) Feature grouping using parallel convolution operations Different channels are designed as batch dimensions, 1×1 and 3×3 parallel convolution operations are performed, and the channel dimensions are grouped into multiple sub-features. For a given input feature map X∈R C×H×W , the feature map is divided into G sub-features in the channel dimension, and the pattern within the group is X=[X0,X i ,...,X G-1 ], X i =R C / / G×H×W , where C is the number of feature map channels, H is the height of the feature map, and W is the width of the feature map; (1b) Constructing the attention weight map using 2D global average pooling The global spatial information is encoded in the output of the 1×1 branch, and the output of the smallest branch will be directly converted to the required dimensional shape before the channel feature joint activation mechanism, that is, By multiplying the convolution output of the parallel path with the matrix dot product operation, the weighted attention map is obtained; the comprehensive formula of 2D global average pooling is as follows: in, Z c is the feature attention weight map, H is the height of the feature map, and W is the width of the feature map; (2) Introducing a detail encoder to capture image detail texture feature information includes the following steps: An inverse neural network is used to construct a residual texture encoder to extract specific features of pneumoconiosis, ensuring that the unique characteristics of each category are preserved while capturing the common texture features across multiple categories. This mechanism not only enhances the model's ability to express lesion features, but also improves the reliability and generalization of the recognition results. The features of a given input are represented as I∈R H×W and V∈R 3×H×W , the comprehensive formula of the feature representation extracted by the encoder is expressed as: Among them, E D (·) represents the detail encoder, Y is the texture heat map, S is the shared code, H is the height of the feature map, and W is the width of the feature map; (3) Using the Transformer layer stack to achieve the fusion of global and local feature information and introduce transfer learning, including the following steps: (3a) For a given input variable X∈R C×H×W , first apply an n×n standard convolution layer to encode local spatial information, followed by a point-wise convolution layer to generate the output variable X∈R H×W×d ; In order to fully capture the global feature information including spatial inductive bias, the input X is decomposed into N non-overlapping planar blocks X U ∈R P×N×d , P is the number of patches, H≤N and W≤N are the height and width of the patch respectively; for each patch p∈{1,...,P}, the relationship between patches is encoded by applying a stack of Transformer layers, (1) unfolding, (2) matrix multiplication to learn local representations, and (3) folding back to the output form for classification to obtain a fused feature representation X G ∈R d×P×N : X G (P)=Transformer(X U (p)),1≤p≤P; (3b) The model is tested and trained using a dataset of common lung diseases, and the model weight coefficients of the optimal results are saved. Based on the characteristics of transfer learning, the weights are fine-tuned and transferred to a dataset of pneumoconiosis grade classification to identify the level of pneumoconiosis lesions.

Images