Multi-scale parallel dual-channel pneumoconiosis lesion grading identification method

Abstract

The invention discloses a multi-scale parallel dual-channel pneumoconiosis lesion grading identification method. The specific implementation process is as follows: staged marking is performed on collected original X-ray pneumoconiosis image data; performing feature grouping and parallel convolution operation by using a multi-scale attention mechanism to enhance a model receptive field and generate a corresponding attention weight map; capturing detail texture feature information by using a feature map extracted by parallel convolution and further combining a detail encoder of a residual structure; in order to further grasp invisible features in the feature map, a parallel dual-channel network module is constructed, so that implicit local and global information is effectively embedded, and pixel-level semantic information in a high-frequency component is cooperatively captured; and finally, fusing different channel information in the feature map by utilizing transfer learning and multi-scale convolution, and respectively inputting the fused information into a detail encoder and a Transform layer stack to identify the lesion classification of the pneumoconiosis. Experimental results show that the method has higher recognition degree.

Description

Technical Field

[0001] The present invention relates to the technical fields of convolutional neural network and pneumoconiosis lesion grading detection and recognition, and in particular to a multi-scale parallel dual-channel pneumoconiosis lesion grading recognition method, which can be used in the field of pneumoconiosis lesion grading detection. Background Art

[0002] With the acceleration of industrialization, the incidence of occupational diseases such as pneumoconiosis has gradually increased, and early diagnosis is particularly critical. Traditional pneumoconiosis detection methods mainly rely on imaging technology (such as X-ray examination, CT scan) and manual diagnosis. These methods have the problems of low recognition accuracy, much manual intervention and difficulty in full automation. Deep learning, as an efficient data-driven method, has shown excellent performance in the field of medical image analysis in recent years. It can automatically extract lesion features from a large amount of medical image data, overcoming the limitations of traditional methods. However, traditional image analysis methods often face problems such as difficulty in feature extraction and poor generalization ability when processing image data of different modalities, resulting in unstable diagnostic results. In response to these problems, the present invention proposes a new dual-channel pneumoconiosis lesion grading recognition model with multi-scale different-path pathways. This method uses a multi-scale attention mechanism to perform feature grouping and parallel convolution operations to enhance the model receptive field, so that it can fully extract lesion features in lung X-ray chest films. After feature extraction, it is input into the detail encoder and the dual channel based on the stacking of Transformer layers to capture detailed texture information and fuse local and global information operations, and finally realize comprehensive information integration and recognition of pneumoconiosis lesion grading. Summary of the invention

[0003] In order to solve the problems that traditional medical image diagnosis methods are difficult to extract comprehensive lesion features and have relatively weak generalization capabilities, the present invention constructs a parallel convolution multi-scale convolution mechanism based on the traditional convolutional neural network to enhance the model receptive field, form an attention weight map, and a parallel dual-channel collaborative mechanism, and then uses a detail encoder and a Transformer stack multi-channel fusion of local and global information at different levels to achieve accurate detection and diagnosis of pneumoconiosis. The specific implementation steps of the present invention are as follows:

[0004] 1. Use the multi-scale attention mechanism to construct the attention weight map. The steps are as follows:

[0005] (1a) Feature grouping:

[0006] Different channels are designed as batch dimensions, 1×1 and 3×3 parallel convolution operations are performed, and the channel dimensions are grouped into multiple sub-features. For a given input feature map X∈R C×H×W , the feature map is divided into G sub-features in the channel dimension, and the pattern within the group is X = [X0, X i,...,X G-1 ], X i =R C / / G×H×W .

[0007] (1b) Construct the attention weight map:

[0008] The global spatial information is encoded in the output of the 1×1 branch using 2D global average pooling, while the output of the smallest branch is directly converted to the required unique shape before the channel feature joint activation mechanism, i.e. By multiplying the convolution output of the parallel path with the matrix dot product operation, the weighted attention map is obtained. The comprehensive formula of 2D global average pooling is as follows:

[0009]

[0010] 2. Use the detail encoder to capture the image detail texture feature information. The steps are as follows:

[0011] (2a) Using inverse neural network to extract specific features of pneumoconiosis:

[0012] The features of a given input are represented as I∈R H×W and V∈R 3×H×W , the comprehensive formula of the feature representation extracted by the encoder is expressed as:

[0013]

[0014] Among them, E D (·) indicates the detail encoder.

[0015] (2b) Enhance detailed texture features using an interactive normalized network:

[0016] By multiplying the attention weight map element by element to fuse the features from different paths, and adding them element by element to aggregate the features, through the convolution layer, ReLU activation function, batch normalization and maximum pooling layer, more critical and advanced detail texture features are extracted from the high-frequency components of the X-ray chest film, that is, the detail texture features of the pneumoconiosis image data are enhanced.

[0017] 3. Use the Transformer layer stack to achieve the fusion of global and local feature information. The steps are as follows:

[0018] (3a) Using the Transformer layer stack to achieve the fusion of global and local feature information:

[0019] For a given input variable X∈R C×H×W , first apply a n×n standard convolutional layer to encode local spatial information, followed by a point-wise convolutional layer to generate the output variable XL ∈R H×W×d In order to fully capture the global feature information including spatial inductive bias, the present invention inputs X L Decompose into N non-overlapping planar blocks X U ∈R P×N×d , P=WH, P is the number of patches, h≤n and w≤n are the height and width of the patch, respectively. For each patch p∈{1,...,P}, the relationship between patches is encoded by applying a stack of Transformer layers (using Swish as the activation function), (1) unfolding, (2) matrix multiplication to learn local representations, and (3) folding back to the output form for classification to obtain a fused feature representation X G ∈R d×P×N :

[0020] X G (P) = Transformer(X U (p)),1≤p≤P

[0021] 4. Use the transfer learning method for pre-training and model weight migration. The steps are as follows:

[0022] (4a) Transfer learning method for pre-training and model weight migration:

[0023] First, the model is trained using a common lung disease dataset, and the optimal model weight coefficient is saved. Second, the weights are fine-tuned based on the characteristics of transfer learning and transferred to a dataset of pneumoconiosis grade classification for detection and diagnosis of pneumoconiosis grade classification.

[0024] The method of the present invention has the following advantages:

[0025] (1) The present invention can efficiently extract the specific features in pneumoconiosis images and enhance the features through a dual-channel fusion mechanism. This mechanism can effectively integrate the information of each part, significantly improve the shortcomings of feature representation, and improve the robustness of the method and the stability of pneumoconiosis lesion grading detection.

[0026] (2) The present invention introduces a detail encoder that can deeply learn the detail texture feature information in pneumoconiosis images and reveal the distinguishing features between different severity levels. This mechanism can capture more comprehensive and accurate lesion information, thereby effectively improving the accuracy and reliability of pneumoconiosis lesion grading detection.

[0027] (3) The present invention proposes a dual-channel fusion detail encoder and a Transformer module for aggregating information from different angles. This module can not only effectively integrate the information of each severity category, but also alleviate the phenomenon of excessive feature loss, thereby more accurately capturing the connection characteristics between global and local features. The transfer learning method is introduced, through which the richness of image feature representation is enhanced, the accuracy of classification is improved, and a more accurate detection of pneumoconiosis lesion level is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is the overall framework diagram of the present invention

[0029] Figure 2 This is a confusion matrix classification effect diagram of a random experiment on a common lung disease dataset

[0030] Figure 3 This is the confusion matrix classification effect diagram of the random experiment on the pneumoconiosis grade classification data set DETAILED DESCRIPTION

[0031] The specific implementation steps of the present invention are as follows:

[0032] 1. The original chest X-ray image and the label provided by the radiologist are taken as input, and 1×1 and 3×3 parallel convolution operations are performed, and the channel dimension is grouped into multiple sub-features so that the spatial semantic features are evenly distributed in each feature group.

[0033] 2. Based on the feature grouping, the global spatial information is encoded in the output of the 1×1 branch using 2D global average pooling, and the output of the smallest branch will be directly converted to the required unique shape before the channel feature joint activation mechanism, that is, By multiplying the convolution output of the parallel path with the matrix dot product operation, a weighted attention map containing accurate spatial position information is obtained. The comprehensive formula for 2D global average pooling is as follows:

[0034]

[0035] Finally, the output feature map within each group is obtained by computing a set of two generated spatial attention weight values, followed by applying a sigmoid function to capture pixel-level pairwise relationships and enhance the global contextual information of all image pixels.

[0036] 3. Use a detail encoder built on a reversible neural network to capture the features of a given input as I∈R H×W and V∈R 3×H×W , the comprehensive formula of the feature representation extracted by the encoder is expressed as:

[0037]

[0038] Among them, E D (·) indicates the detail encoder.

[0039] The information aggregation features are further integrated by multiplying the input processed weight map element by element and adding element by element at the same time. The convolution block extracts more abstract and advanced detail texture features from the high-frequency components of the X-ray chest film through a multi-level interactive normalized network structure, and finally outputs the detailed texture feature results fused by multiple layers of convolution blocks.

[0040] 4. Use the Transformer layer stack to achieve the fusion of global and local feature information, and input X L Decompose into N non-overlapping planar blocks X U ∈R P×N×d , P=WH, P is the number of patches, h≤n and w≤n are the height and width of the patch, respectively. For each patch p∈{1,...,P}, the relationship between patches is encoded by applying a stack of Transformer layers (using Swish as the activation function), (1) unfolding, (2) matrix multiplication to learn local representations, and (3) folding back to the output form for classification to obtain a fused feature representation X G ∈R d×P×N :

[0041] X G (P) = Transformer(X U (p)),1≤p≤P

[0042] X G ∈R P×N×d Fold to X F ∈R P×N×d , and then project it into a low-dimensional space C using point-by-point convolution and concatenate it with the input X. Finally, these concatenated features are fused through a 3×3 convolution layer to obtain the detection result.

[0043] 5. Use the transfer learning method for pre-training to save the weights and transfer the fine-tuning to apply to the grading detection and identification of pneumoconiosis lesions.

[0044] The effect of the present invention is further verified by the following experiments:

[0045] The experiment was verified on the common lung disease and pneumoconiosis classification data sets. The common lung disease data set selected Chest X-ray 8 to extract 2008 cases of viral pneumonia data; 2009 cases of bacterial pneumonia data; 1546 cases of pneumothorax data; 2034 cases of tuberculosis data; 2031 cases of coronavirus pneumonia data; 2013 cases of health data; in addition, 1248 cases of pneumoconiosis data from the Affiliated Hospital of Southwest Medical University were also selected, covering a total of seven common lung diseases. The average accuracy of the present invention in the random experiment of the common lung disease data set reached 0.9025, showing good stability.

[0046] The pneumoconiosis classification dataset is derived from chest X-ray image data from the Affiliated Hospital of Southwest Medical University. A database of 1,760 anonymous digital radiographs (DR) posteroanterior chest radiographs (CXRs) showing evidence of pneumoconiosis was generated from January 2017 to June 2020. The inclusion criteria of this dataset are as follows: (1) all patients with a history of dust exposure; (2) patients with a good chest radiograph, or at least an acceptable quality that meets the guidelines of the International Labor Organization; (3) all positive cases have a diagnosis certificate from a pneumoconiosis diagnosis qualification unit. All patients were divided into four categories, namely 0 (n = 512), 1 (n = 352), 2 (n = 478) and 3 (n = 418). The average accuracy of the present invention in the randomized experiment of the pneumoconiosis classification dataset reached 0.7771, which confirmed the effectiveness of this invention.

Claims

1. A multi-scale parallel dual-channel pneumoconiosis lesion classification and identification method, comprising the following steps: (1) Use multi-scale attention mechanism to group features and construct attention weight map; (2) Introducing a detail encoder to capture image detail texture feature information; (3) Use the Transformer layer stack to achieve the fusion of global and local feature information and introduce transfer learning.

2. The multi-scale parallel dual-channel pneumoconiosis lesion classification and identification method according to claim 1 is characterized in that: The step (1) of using a multi-scale attention mechanism to perform feature grouping and construct an attention weight map includes the following steps: (2a) Feature grouping using parallel convolution operations Different channels are designed as batch dimensions, 1×1 and 3×3 parallel convolution operations are performed, and the channel dimensions are grouped into multiple sub-features. For a given input feature map X∈R C×H×W , the feature map is divided into G sub-features in the channel dimension, and the pattern within the group is X = [X0, X i ,...,X G-1 ], X i =R C / / G×H×W , where C is the number of feature map channels, H is the height of the feature map, and W is the width of the feature map; (2b) Constructing the attention weight map using 2D global average pooling The present invention encodes the global spatial information in the output of the 1×1 branch, and the output of the smallest branch will be directly converted to the required unique shape before the channel feature joint activation mechanism, that is, By multiplying the convolution output of the parallel path with the matrix dot product operation, the weighted attention map is obtained; the comprehensive formula of 2D global average pooling is as follows: Among them, Z c is the feature attention weight map, H is the height of the feature map, and W is the width of the feature map.

3. The multi-scale parallel dual-channel pneumoconiosis lesion classification and identification method according to claim 1 is characterized in that The step (2) of introducing a detail encoder to capture image detail texture feature information comprises the following steps: (3a) Using inverse neural networks to extract specific features of pneumoconiosis, ensuring that the unique features of each category are retained while capturing the common features among multiple categories; This mechanism not only enhances the model's ability to express lesion features, but also improves the reliability and generalization of the recognition results. The features of a given input are represented as I∈R H×W and V∈R 3×H×W , the comprehensive formula of the feature representation extracted by the encoder is expressed as: Among them, E D (·) represents the detail encoder, Y is the texture heat map, S is the shared code, H is the height of the feature map, and W is the width of the feature map.

4. The multi-scale parallel dual-channel pneumoconiosis lesion classification and identification method according to claim 1 is characterized in that: The use of the Transformer layer stack in step (3) to achieve the fusion of global and local feature information includes the following steps: (4a) For a given input variable X∈R C×H×W , first apply a n×n standard convolutional layer to encode local spatial information, followed by a point-wise convolutional layer to generate the output variable X∈R H×W×d ; In order to fully capture the global feature information including spatial inductive bias, the present invention decomposes the input X into N non-overlapping planar blocks X U ∈R P×N×d , P=WH, P is the number of patches, h≤n and w≤n are the height and width of the patch respectively; for each patch p∈{1,...,P}, the relationship between patches is encoded by applying a stack of Transformer layers (using Swish as the activation function), (1) unfolding, (2) matrix multiplication to learn local representations, and (3) folding back to the output form for classification to obtain the fused feature representation X G ∈R d×P×N : X G (P)=Transformer(X U (p)),1≤p≤P; (4b) The model is tested and trained using a common lung disease dataset, and the model weight coefficients of the optimal results are saved. The weights are fine-tuned based on the characteristics of transfer learning and transferred to a dataset with pneumoconiosis grade classification to identify the level of pneumoconiosis lesions.

Images