Class imbalance X-ray chest radiography data set-oriented pneumoconiosis staging diagnosis system
By pre-treating X-ray chest radiographs and custom deformable and separable convolutional treatment, combined with clinical diagnosis information, the accuracy of pneumoconiosis stage diagnosis is solved, and efficient and accurate pneumoconiosis stage diagnosis is achieved.
Patent Information
- Application Number
- CN202510798905.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-16
- Publication Date
- 2025-08-29
AI Technical Summary
Existing medical image classification technology is difficult to accurately distinguish the different stages of pneumoconiosis on X-ray chest radiographs, resulting in limited early diagnosis and treatment.
Preprocessing and custom deformable separable convolution are used to process X-ray chest radiographs, balance the data sets and extract feature differences, accurately classify them in combination with clinical diagnostic information, and model training is performed using custom deformable separable convolution modules, feature fusion modules, residual blocks and convolution block attention modules.
It significantly improves the accuracy and robustness of the staged diagnosis of pneumoconiosis, provides doctors with fast and effective auxiliary decision-making tools, and can maintain efficient and accurate detection in complex environments.
Smart Images
Figure CN120565037A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of medical image pneumoconiosis staging diagnosis and processing. It uses preprocessing and customized deformable and separable convolution to preprocess and extract features of class-imbalanced X-ray chest films, and introduces domain knowledge containing clinical diagnostic information for constraint guidance to develop an accurate diagnostic model that distinguishes the differences and connections between different stages of pneumoconiosis. Background Art
[0002] Pneumoconiosis is the most common occupational disease, and its diagnosis relies on chest X-ray analysis. Pneumoconiosis is one of the most serious occupational diseases worldwide, with a delayed onset and incubation period. Once contracted, it is incurable. Therefore, staging of pneumoconiosis is crucial for disease control. Currently, diagnosis can be performed by specialized pneumoconiosis diagnostic institutions or physicians according to relevant diagnostic criteria. However, this method is time-consuming and requires comprehensive consideration of multiple factors, including the patient's occupational history, clinical manifestations, imaging studies, and pulmonary function tests, hindering early diagnosis and treatment. Chest X-rays are a common screening method for pneumoconiosis, enabling early detection of lung abnormalities. Even before obvious symptoms develop, chest X-rays can reveal subtle changes in the lungs, such as increased, thickened, and disrupted lung markings, as well as small shadows. These changes can aid in early detection of pneumoconiosis, allowing for more time for subsequent diagnosis and treatment. Despite this, chest X-rays of different stages of pneumoconiosis can still show some similarities, hindering accurate classification. Currently, in the field of medical imaging, deep convolutional neural networks are commonly used to solve medical image classification problems, but there is no mature technology for accurately distinguishing the stages of pneumoconiosis. Summary of the Invention
[0003] The present invention discloses a pneumoconiosis staging diagnosis system for class-imbalanced X-ray chest film datasets. Based on X-ray chest films, the system uses preprocessing and customized deformable and separable convolution to balance the dataset and extract the feature differences between different stages of pneumoconiosis. It also introduces domain knowledge containing clinical diagnostic information to guide the accurate classification of novel coronavirus infection, and has good robustness.
[0004] In order to achieve the above objectives, the following technical solutions are adopted: A pneumoconiosis staging system for class-imbalanced X-ray chest film datasets, including a preprocessing module, a custom deformable separable convolution module, a feature fusion module, a residual block, and a convolution block attention module: Preprocessing module: Lesion segmentation, grayscale normalization, median filtering, and dataset balancing methods were used to extract regions of interest from pneumoconiosis X-ray chest radiographs. The images were resized to 1024*1024 pixels and randomly divided into training, validation, and test sets according to a 6:2:2 ratio.
[0005] Lesion segmentation: Using thresholding + morphological optimization, image pixels are divided into different categories by setting multiple thresholds, which can quickly achieve preliminary lesion segmentation. Morphological optimization is based on the principles of mathematical morphology and operates on binary images. The computational complexity is relatively small, and processing can be completed in a short time, improving segmentation efficiency. It can achieve automatic or semi-automatic delineation of lesion boundaries, classifying each pixel or voxel in the image as belonging to the lesion or non-lesion category, thereby generating a lesion segmentation result image corresponding to the original image, in which the lesion area is highlighted for further diagnosis and analysis by doctors; Grayscale normalization: Normalizes the grayscale values of an image to the range [0, 1]. The core principle is to adjust the grayscale value of each pixel in the image through mathematical transformation, redistributing the grayscale values to stretch the grayscale range of the image to the entire representable range, thereby enhancing the contrast between different grayscale areas in the image and making the details in the image easier to observe. Median filtering: For each pixel in the image, a 3×3 rectangular neighborhood is selected with that pixel as the center. The grayscale values of all pixels in this neighborhood are then sorted, and the median value is taken as the filtered grayscale value of the center pixel. For a 3×3 neighborhood, there are 9 pixels in total. The grayscale values of these 9 pixels are sorted from small to large (or from large to small). The grayscale value at the 5th position is the median value, which is used to replace the original grayscale value of the center pixel. Balancing the dataset: Faced with limited image samples, data augmentation is a widely used technique to optimize image quality and increase sample diversity. Data augmentation implemented during image preprocessing primarily includes color gamut enhancement and geometric transformation strategies. Color gamut enhancement generates new samples by adjusting the brightness of each image channel, but this method is less effective than geometric enhancement in improving sample accuracy. Geometric transformations include horizontal flipping and vertical flipping. Horizontal flipping mirrors the image along the vertical axis, while vertical flipping mirrors the image along the horizontal axis. Rotating an image around its center point by a certain angle magnifies or reduces the image. Enlarging an image can highlight image details, while reducing it can simulate the effect of observing from a distance. The original raster data of the sample set is large in size, and existing computing resources are insufficient to support efficient model inference and prediction. This can lead to insufficient GPU memory and reduced inference speed, especially when the image size is too large. To ensure smooth training, after completing chest radiograph sample annotation, Python was used to perform sliding window cropping on the corrected, fused images and the corresponding labeled images. The cropping specifications were set to 128 × 128 pixels, and a 10% overlap was maintained between adjacent windows. In object detection tasks, images of varying scales can help the model learn the characteristics of objects at different sizes. The image was shifted horizontally and vertically. Translation changes the position of the object within the image, making the model more adaptable to changes in the object's position. This study employed color gamut enhancement and geometric enhancement methods to expand sample capacity. The original samples were subjected to rotation, flipping, Gaussian noise, color enhancement, and contrast enhancement, thereby increasing sample diversity. The resulting pneumoconiosis pretraining dataset contained 5,097 images of 1,024 × 1,024 pixels.
[0006] Its custom deformable and separable convolution module: adaptively adjusts the size and shape of the receptive field according to the image content. For images with variable target shapes and complex backgrounds, it can dynamically adjust the convolution operation according to the characteristics of the input data. For targets of different scales, it can adaptively change the size and position of the convolution kernel, making the model more flexible in detecting or identifying objects of different sizes. The custom deformable and separable convolution module specifically includes depthwise convolution (channel-by-channel convolution) and point-by-point convolution: Depth convolution: The input feature map is , where H is the height and W is the width, is the number of input channels. The convolution kernel of the depthwise convolution is ,in is the size of the convolution kernel; for each channel of the depth convolution, the convolution kernel performs convolution operation on the corresponding channel of the input feature map to obtain the output feature map ,in and is the height and width of the output feature map. The first ( The specific formula for the value of each position is as follows:
[0007] Point-wise convolution: output of depth-wise convolution As the input of point-by-point convolution, the convolution kernel of point-by-point convolution is ,in Is the number of output channels. The output of point-by-point convolution , for the output feature map The first ( The specific formula for the value of each position is as follows:
[0008] The core of deformable convolution is to generate an offset through an additional convolution layer, and then sample the input feature map according to the offset. Assume that the input feature map is X The offset is generated through a convolutional layer and is recorded as ,in is the number of sampling points in the convolution kernel (for The convolution kernel, ). For the output feature map Middle The specific formula for the value of each position is as follows:
[0009] in is the convolution kernel, is the convolution kernel The relative position of the sampling points, is the first The offset of the sampling points. It may not be an integer position, and bilinear interpolation is usually needed to calculate the value at that position. , and yes of and coordinate, , , , , the bilinear interpolation formula is as follows:
[0010] in are the weights for bilinear interpolation.
[0011] Feature fusion module: combines features at different levels so that the model can not only make accurate judgments based on detailed information, but also determine the category of the image based on semantic information, thereby improving classification accuracy.
[0012] The same-scale splicing operation formula is as follows:
[0013] in , Indicates that there are n feature maps of the same scale, and the spatial size of each feature map is H , , , , Indicates the corresponding channel number, ,in =1,2, , . It represents the fused feature map obtained after the feature map is spliced in the channel dimension, but the number of channels becomes ,Right now . Represents the spatial position of the element in the feature map , Represents the channel index in the fused feature map ; The calculation formula for the extrusion operation is:
[0014] Assume that the input feature map is , where C is the number of channels, H and W are the height and width of the feature map respectively. After the squeezing operation, a vector of length C is obtained. .in Represents a vector No. elements, Represents the input feature map No. channels, It is Position in channel The elements at .
[0015] The incentive operation calculation formula is:
[0016]
[0017] The vector Through a A fully connected layer of neurons (r is the reduction ratio, used to reduce the number of parameters), and uses the ReLU activation function. is the weight matrix of the first fully connected layer, is the output of this layer. Through another neurons in a fully connected layer and use Activation function, get a length of The weight vector ,in is the weight matrix of the second fully connected layer; The reweighting operation is calculated as:
[0018] The obtained weight vector With the input feature map Perform channel-by-channel multiplication and reweight the feature map of each channel to obtain the final output feature map .in is the c-th channel of the output feature map Y, is the cth element of the weight vector s, Represents element-wise multiplication.
[0019] Residual blocks: By introducing shortcut connections, they allow gradients to propagate more directly to previous layers, alleviating the vanishing gradient problem. This allows for deeper network training. In the pneumoconiosis staging diagnosis task, the residual block primarily uses a basic residual block. The basic residual block uses the shallower ResNet18 architecture, consisting of two 3×3 convolutional layers and a shortcut connection. The formulas used are described below.
[0020] Assume that the input feature map is , the output feature map is , then the calculation formula of the first convolutional layer is as follows:
[0021] in, is the weight of the first convolutional layer, is the bias term, BN represents the batch normalization operation, ReLU is the activation function, and * represents the convolution operation.
[0022] The calculation formula of the second convolutional layer is as follows:
[0023] in, are the weights of the second convolutional layer, is the bias term.
[0024] If the input feature map X and The number of channels is the same, then the shortcut connection directly adds the input X to If the number of channels is different, a 1×1 convolutional layer is needed to adjust the dimension of the input X so that its number of channels is the same as Same. Assume that the output of the shortcut connection is , the calculation formula for shortcut connection is as follows:
[0025] in, is the weight of the 1×1 convolutional layer, is the bias term.
[0026] The output of the convolution operation Output of shortcut connection Add and use ReLU activation function to get the final output :
[0027] Convolutional block attention module: helps the network focus on important areas in the image more intelligently, thereby improving the performance of the model. Convolutional block attention module, input feature map , through global average pooling, aggregation along the spatial dimensions (H and W) is performed to obtain a C-dimensional vector , and its calculation formula is:
[0028] in =1, , C. At the same time, a C-dimensional vector is obtained along the spatial dimension through global maximum pooling , which is calculated as follows:
[0029] right and The inputs are fed into a shared multi-layer perceptron (MLP). The MLP contains a hidden layer whose number of neurons is usually set to a small ratio of the number of input channels C, such as C / r (r is the reduction ratio).
[0030] Let the weight matrix of MLP be and , the bias is and .for , after MLP processing, we get MLP( ) ( )+ ,in is the activation function. Similarly, for Similar processing is also performed. Then the two results after MLP processing are added together and then a Sigmoid activation function is used to obtain the channel attention weight. ,Right now (MLP( )+MLP( )).
[0031] Finally, the channel attention weight Multiply the original input feature map F channel by channel to obtain the feature map enhanced by channel attention ,in Represents element-wise multiplication.
[0032] Input feature map processed by channel attention module First, Perform global average pooling and global maximum pooling along the channel dimension to obtain two 1×H×W feature maps. and The calculation formula is as follows:
[0033]
[0034] in 1, ,H, 1, , W.
[0035] Will and Splicing is performed on the channel dimension to obtain a 2×H×W feature map. Then it passes through a convolution layer with a convolution kernel size of 7×7, a stride of 1, and a padding of 3. Let the convolution kernel weight be , the bias is , after the convolution operation, we get The calculation formula is as follows: Conv([ ]) ) in[ ] indicates the splicing operation, is the Sigmoid activation function.
[0036] Finally, the spatial attention weights And the feature map after processing by the channel attention module Perform element-by-element multiplication to obtain the final feature map processed by CBAM .
[0037] The beneficial effects of the present invention are: The present invention implements a pneumoconiosis staging system for class-imbalanced X-ray chest radiograph datasets to achieve pneumoconiosis staging diagnosis. Preprocessing and custom deformable separable convolutions are used to balance the dataset and extract features between different stages of pneumoconiosis. The system also uses case-specific features extracted from diagnostic reports to provide guidance and constraints, significantly improving diagnostic accuracy and providing physicians with a powerful decision-making tool. The present invention enables efficient and rapid detection within a relatively short processing time while maintaining a high accuracy rate. Even in complex detection environments, the model demonstrates excellent robustness and can therefore be applied to the diagnosis of pneumoconiosis. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] Figure 1 This is the overall network architecture of the pneumoconiosis staging system for class-imbalanced X-ray chest film datasets. DSCL stands for Custom Deformable Separable Convolutional Module, and CBAM stands for Convolutional Block Attention Module.
[0039] Figure 2 This is the operation flow of the preprocessing module.
[0040] Figure 3 A network architecture for custom deformable and separable modules.
[0041] Figure 4 This is the network architecture of the feature fusion module.
[0042] Figure 5 The network architecture of the residual block.
[0043] Figure 6 Attention module network architecture for convolutional blocks.
[0044] Figure 7 The figure shows the comparison of the test set accuracy of the dataset processed by lesion segmentation, grayscale normalization and median filtering based on the present invention and the other three methods.
[0045] Figure 8 The figure shows the comparison of the test set accuracy between the present invention and the other three methods after balancing the data. DETAILED DESCRIPTION
[0046] The present invention is described in detail below with reference to specific embodiments.
[0047] A pneumoconiosis staging system for class-imbalanced X-ray chest film datasets, the overall network architecture is as follows Figure 1 Shown, including: A1. Preprocessing module: Lesion segmentation, grayscale normalization, median filtering, and dataset balancing are used to extract regions of interest from pneumoconiosis X-ray chest radiographs. The images are resized to 1024*1024 pixels and randomly divided into training, validation, and test sets according to a 6:2:2 ratio. The specific preprocessing process is as follows: Figure 2 shown.
[0048] Lesion segmentation: Using thresholding + morphological optimization, image pixels are divided into different categories by setting multiple thresholds, which can quickly achieve preliminary lesion segmentation. Morphological optimization is based on the principles of mathematical morphology and operates on binary images. The computational complexity is relatively small, and processing can be completed in a short time, improving segmentation efficiency. It can achieve automatic or semi-automatic delineation of lesion boundaries, classifying each pixel or voxel in the image as belonging to the lesion or non-lesion category, thereby generating a lesion segmentation result image corresponding to the original image, in which the lesion area is highlighted for further diagnosis and analysis by doctors; Grayscale normalization: Normalizes the grayscale values of an image to the range [0, 1]. The core principle is to adjust the grayscale value of each pixel in the image through mathematical transformation, redistributing the grayscale values to stretch the grayscale range of the image to the entire representable range, thereby enhancing the contrast between different grayscale areas in the image and making the details in the image easier to observe. Median filtering: For each pixel in the image, a 3×3 rectangular neighborhood is selected with that pixel as the center. The grayscale values of all pixels in this neighborhood are then sorted, and the median value is taken as the filtered grayscale value of the center pixel. For a 3×3 neighborhood, there are 9 pixels in total. The grayscale values of these 9 pixels are sorted from small to large (or from large to small). The grayscale value at the 5th position is the median value, which is used to replace the original grayscale value of the center pixel. Balancing the dataset: Faced with limited image samples, data augmentation is a widely used technique to optimize image quality and increase sample diversity. Data augmentation implemented during image preprocessing primarily includes color gamut enhancement and geometric transformation strategies. Color gamut enhancement generates new samples by adjusting the brightness of each image channel, but this method is less effective than geometric enhancement in improving sample accuracy. Geometric transformations include horizontal flipping and vertical flipping. Horizontal flipping mirrors the image along the vertical axis, while vertical flipping mirrors the image along the horizontal axis. Rotating an image around its center point by a certain angle magnifies or reduces the image. Enlarging an image can highlight image details, while reducing it can simulate the effect of observing from a distance. The original raster data of the sample set is large in size, and existing computing resources are insufficient to support efficient model inference and prediction. This can lead to insufficient GPU memory and reduced inference speed, especially when the image size is too large. To ensure smooth training, after completing chest radiograph sample annotation, Python was used to perform sliding window cropping on the corrected, fused images and the corresponding labeled images. The cropping specifications were set to 128 × 128 pixels, and a 10% overlap was maintained between adjacent windows. In object detection tasks, images of varying scales can help the model learn the characteristics of objects at different sizes. The image was shifted horizontally and vertically. Translation changes the position of the object within the image, making the model more adaptable to changes in the object's position. This study employed color gamut enhancement and geometric enhancement methods to expand sample capacity. The original samples were subjected to rotation, flipping, Gaussian noise, color enhancement, and contrast enhancement, thereby increasing sample diversity. The resulting pneumoconiosis pretraining dataset contained 5,097 images of 1,024 × 1,024 pixels.
[0049] A2. Input the preprocessed image into the custom deformable separable convolution module DSCL to extract the global features of the region and obtain the image feature representation; The custom deformable separable convolutional module network structure involved in step A2 is as follows: Figure 3 This network is improved based on MobileNet-V2. The improvement mainly lies in redesigning the MobileNet-V2 backbone module and connecting the features extracted at each stage along the feature channel to finally obtain the feature representation of the target area of the pneumoconiosis chest X-ray image.
[0050] Among them, the important design of the custom deformable separable convolution module is to introduce additional learnable offsets, so that the sampling points of the convolution kernel can adaptively adjust their positions according to the content of the input data, and more flexibly capture irregular shapes and geometric changes in the image. At the same time, the separable convolution splits the convolution operation into multiple steps. The depth-wise separable convolution first performs channel-by-channel convolution and then performs point-by-point convolution, considering regional and channel information respectively, further improving the flexibility and efficiency of sampling. The separable convolution part greatly reduces the amount of calculation and parameters by decomposing the convolution operation into smaller operations such as channel-by-channel convolution and point-by-point convolution. It can automatically adjust the behavior of the convolution kernel according to the characteristics of the input data, so that it is more robust when processing diverse image content, reduces the risk of overfitting, and improves the accuracy and stability of the model in different scenarios. The present invention adopts a custom deformable separable convolution, and then connects the convolution output results along the channel for further processing.
[0051] The specific steps of step A2 are as follows: The input image enters the custom deformable separable convolution module for depth convolution (channel-by-channel convolution) to obtain the output feature map .
[0052] like Figure 3 As shown, the input image , where H is the height and W is the width, is the number of input channels. The convolution kernel of the depthwise convolution is ,in is the size of the convolution kernel.
[0053] For each channel of the depth convolution, the convolution kernel performs convolution operation on the corresponding channel of the input feature map to obtain the output feature map ,in and is the height and width of the output feature map. The first ( The specific formula for the value of each position is as follows
[0054] The summation here is performed within the size of the convolution kernel, and the padding and step size must be considered. impact.
[0055] Output of depthwise convolution As the input of point-by-point convolution, the convolution kernel of point-by-point convolution is ,in Is the number of output channels. The output of point-by-point convolution , for the output feature map The first ( The specific formula for the value of each position is as follows:
[0056] The core of deformable convolution is to generate offsets through additional convolutional layers and then sample the input feature map according to the offsets. Assume that the input feature map is , the offset is generated through a convolutional layer and is recorded as ,in is the number of sampling points in the convolution kernel (for The convolution kernel, ). For the output feature map Middle The specific formula for the value of each position is as follows:
[0057] in is the convolution kernel, is the convolution kernel The relative position of the sampling points, is the first The offset of the sampling points. It may not be an integer position, and usually bilinear interpolation is needed to calculate the value at that position.
[0058] set up , and yes of and coordinate, , , , , the bilinear interpolation formula is as follows:
[0059] in are the weights for bilinear interpolation.
[0060] A3. Input the output of the custom deformable separable convolution operation into the feature fusion module to extract the specific features of patients at different stages. The resulting feature map is represented by Y. The feature fusion module is as follows: Figure 4 As shown, it includes same-scale splicing, squeezing operation, excitation operation, and re-weighting operation.
[0061] The specific steps are as follows: A31. Extract and analyze the specific features of different stages of pneumoconiosis and perform feature fusion operations to meet the input requirements. The feature fusion module structure is as follows: Figure 4 shown.
[0062] The same-scale splicing operation formula is as follows:
[0063] in , Indicates that there are n feature maps of the same scale, and the spatial size of each feature map is H , , , , Indicates the corresponding channel number, ,in =1,2, , . It represents the fused feature map obtained after the feature map is spliced in the channel dimension, but the number of channels becomes ,Right now . Represents the spatial position of the element in the feature map , Represents the channel index in the fused feature map .
[0064] A32. The calculation formula for extrusion operation is:
[0065] Assume that the input feature map is , where C is the number of channels, H and W are the height and width of the feature map respectively. After the squeezing operation, a vector of length C is obtained. .in Represents a vector No. elements, Represents the input feature map No. channels, It is Position in channel The elements at .
[0066] A33. The calculation formula for incentive operation is:
[0067]
[0068] The vector Through a A fully connected layer of neurons (r is the reduction ratio, used to reduce the number of parameters), and uses the ReLU activation function. is the weight matrix of the first fully connected layer, is the output of this layer. Through another neurons in a fully connected layer and use Activation function, get a length of The weight vector ,in is the weight matrix of the second fully connected layer.
[0069] A34. The calculation formula for the reweighting operation is:
[0070] The obtained weight vector With the input feature map Perform channel-by-channel multiplication and reweight the feature map of each channel to obtain the final output feature map .in is the c-th channel of the output feature map Y, is the cth element of the weight vector s, Represents element-wise multiplication.
[0071] A4. The residual block introduces shortcut connections, allowing the gradient to propagate more directly to the previous layer, thereby alleviating the gradient vanishing problem. This allows the network to be trained deeper. In the task of pneumoconiosis staging diagnosis, the residual block is mainly a basic residual block. The basic residual block uses a shallower ResNet network ResNet18, which consists of two 3×3 convolutional layers and a shortcut connection. The residual block structure is as follows Figure 5 As shown, the residual block includes convolution operations and shortcut connections. A41. Assume that the input feature map is , the output feature map is , then the calculation formula of the first convolutional layer is as follows:
[0072] in, is the weight of the first convolutional layer, is the bias term, BN represents the batch normalization operation, ReLU is the activation function, and * represents the convolution operation.
[0073] The calculation formula of the second convolutional layer is as follows:
[0074] in, are the weights of the second convolutional layer, is the bias term.
[0075] A42. If the input feature map X and The number of channels is the same, then the shortcut connection directly adds the input X to If the number of channels is different, a 1×1 convolutional layer is needed to adjust the dimension of the input X so that its number of channels is the same as Same. Assume that the output of the shortcut connection is , the calculation formula for shortcut connection is as follows:
[0076] in, is the weight of the 1×1 convolutional layer, is the bias term.
[0077] The output of the convolution operation Output of shortcut connection Add and use ReLU activation function to get the final output :
[0078] A5. Convolutional Block Attention Module (CBAM) helps the network focus on important areas in the image more intelligently, thereby improving the performance of the model. The structure of the convolutional block attention module is as follows: Figure 6 As shown, the convolutional block attention module includes a channel attention module and a spatial attention module.
[0079] A51. Channel attention module, input feature map , through global average pooling, aggregation along the spatial dimensions (H and W) is performed to obtain a C-dimensional vector , and its calculation formula is:
[0080] in =1, , C.
[0081] A52. At the same time, a C-dimensional vector is obtained along the spatial dimension through global maximum pooling , which is calculated as follows:
[0082] right and The inputs are fed into a shared multi-layer perceptron (MLP). The MLP contains a hidden layer whose number of neurons is usually set to a small ratio of the number of input channels C, such as C / r (r is the reduction ratio).
[0083] Let the weight matrix of MLP be and , the bias is and .for , after MLP processing, we get MLP( ) ( )+ ,in is the activation function. Similarly, for Similar processing is also performed. Then the two results after MLP processing are added together and then a Sigmoid activation function is used to obtain the channel attention weight. ,Right now (MLP( )+MLP( )).
[0084] Finally, the channel attention weight Multiply the original input feature map F channel by channel to obtain the feature map enhanced by channel attention ,in Represents element-wise multiplication.
[0085] Input feature map processed by channel attention module First, Perform global average pooling and global maximum pooling along the channel dimension to obtain two 1×H×W feature maps. and The calculation formula is as follows:
[0086]
[0087] in 1, ,H, 1, , W.
[0088] Will and Splicing is performed on the channel dimension to obtain a 2×H×W feature map. Then it passes through a convolution layer with a convolution kernel size of 7×7, a stride of 1, and a padding of 3. Let the convolution kernel weight be , the bias is , after the convolution operation, we get The calculation formula is as follows: Conv([ ]) ) in[ ] indicates the splicing operation, is the Sigmoid activation function.
[0089] Finally, the spatial attention weights And the feature map after processing by the channel attention module Perform element-by-element multiplication to obtain the final feature map processed by CBAM .
[0090] A6. The entire network uses a joint loss function to minimize the distance between the predicted category and the true category.
[0091] The sparse cross loss function is a commonly used loss function for multi-classification problems in deep learning. In multi-classification tasks, the true label of each sample is usually represented by an integer, which represents the category index to which the sample belongs. Label smoothing is a smoothing process that converts the original hard label (0 or 1) into a soft label. Combining these two loss functions, through special encoding and calculation of sparse category labels, can pay more attention to the classification of minority class samples and increase the penalty for minority class prediction errors, thereby prompting the model to work harder to learn the characteristics of minority classes, improve the recognition ability of minority classes, and to a certain extent alleviate the problem of the model biased towards the majority class caused by category imbalance. The relevant calculation formula is as follows:
[0092] log
[0093] in, represents the number of samples, It is samples are predicted to be their true categories The probability of yes Class, The first The probability of the class, is the smoothing coefficient, and its value range is [0,1].
[0094] The joint loss function expression is as follows:
[0095] in is a hyperparameter, is the sparse cross entropy function, is the label smoothing loss function.
[0096] Figure 7 and Figure 8 The following is a comparison of the test accuracy of the present invention with three other methods using different preprocessing operations. Method 1 uses MobileNet-V2 for classification, Method 2 uses U-Net for classification, and Method 3 uses ResNet for classification. As can be seen from the figure, the present invention has the highest accuracy compared to the other two methods, ultimately reaching 86.43%.
[0097] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention.
Claims
1. A pneumoconiosis staging diagnosis system for class-imbalanced X-ray chest film datasets, characterized by: Including preprocessing module, custom deformable separable convolution module, feature fusion module, residual block, convolution block attention module: Customized Deformable Separable Convolution Module: Adaptively adjusts the size and shape of the receptive field based on the image content. For images with variable object shapes and complex backgrounds, it can dynamically adjust the convolution operation based on the characteristics of the input data. For objects of different scales, it can adaptively change the size and position of the convolution kernel, making the model more flexible in detecting or recognizing objects of different sizes. Feature fusion module: The output of the custom deformable separable convolution operation is input into the feature fusion module to extract the specific features of patients at different stages, and the resulting feature map is represented by Y; The feature fusion module combines features at different levels, enabling the model to make accurate judgments based on detailed information while also determining the image category based on semantic information, thus improving classification accuracy. Residual block: The output of the feature fusion operation is input into the residual block to learn the residual between the input and the target; By introducing shortcut connections, the gradient can be propagated more directly to the previous layer, thereby alleviating the gradient disappearance problem; Convolutional block attention module: The output of the residual operation is input into the convolutional block attention module, which simultaneously captures key information in the spatial and channel dimensions, adaptively adjusts feature weights, enhances the model's attention to important features and suppresses irrelevant information; helps the network focus more intelligently on important areas in the image, thereby improving model performance.
2. The pneumoconiosis staging diagnosis system according to claim 1, characterized in that: The preprocessing module includes lesion segmentation, grayscale normalization, median filtering and data set balancing.
3. The pneumoconiosis staging diagnosis system according to claim 1, characterized in that: The custom deformable separable convolution module specifically includes channel-by-channel convolution and point-by-point convolution: Channel-by-channel convolution: The input feature map is , where H is the height and W is the width, is the number of input channels; the convolution kernel of the depthwise convolution is ,in is the size of the convolution kernel; for each channel of the depth convolution, the convolution kernel performs convolution operation on the corresponding channel of the input feature map to obtain the output feature map ,in and is the height and width of the output feature map. The first ( The specific formula for the value of each position is as follows: ; Point-by-point convolution: output of depth-wise convolution As the input of point-by-point convolution, the convolution kernel of point-by-point convolution is ,in is the number of output channels; the output of point-by-point convolution , for the output feature map The first ( The specific formula for the value of each position is as follows: ; The core of deformable convolution is to generate offsets through additional convolutional layers and then sample the input feature map according to the offsets; assuming the input feature map is , the offset is generated through a convolutional layer and is recorded as ,in is the number of sampling points in the convolution kernel, for The convolution kernel, For the output feature map Middle The specific formula for the value of each position is as follows: ;in is the convolution kernel, is the convolution kernel The relative position of the sampling points, is the first The offset of the sampling points; due to It may not be an integer position, and bilinear interpolation is usually needed to calculate the value of the position; let , and yes of and coordinate, , , , , the bilinear interpolation formula is as follows: ;in are the weights for bilinear interpolation.
4. The pneumoconiosis staging diagnosis system according to claim 1, characterized in that: The feature fusion module includes same-scale splicing, squeezing operation, excitation operation, and re-weighting operation: The same-scale splicing operation formula is as follows: ;in , Indicates that there are n feature maps of the same scale, and the spatial size of each feature map is H , , , , Indicates the corresponding channel number, ,in =1,2, , It represents the fused feature map obtained after the feature map is spliced in the channel dimension, but the number of channels becomes ,Right now Represents the spatial position of the element in the feature map , Represents the channel index in the fused feature map ; The calculation formula for the extrusion operation is: ; Assume that the input feature map is , where C is the number of channels, H and W are the height and width of the feature map respectively; after the squeezing operation, a vector of length C is obtained ;in Represents a vector No. elements, Represents the input feature map No. channels, It is Position in channel The elements at The incentive operation calculation formula is: ; ; The vector Through a A fully connected layer of neurons (r is the reduction ratio, used to reduce the number of parameters), and uses the ReLU activation function; is the weight matrix of the first fully connected layer, is the output of this layer; Through another neurons in a fully connected layer and use Activation function, get a length of The weight vector ,in is the weight matrix of the second fully connected layer; The reweighting operation is calculated as: ; The obtained weight vector With the input feature map Perform channel-by-channel multiplication and reweight the feature map of each channel to obtain the final output feature map ;in is the c-th channel of the output feature map Y, is the cth element of the weight vector s, Represents element-wise multiplication.
5. The pneumoconiosis staging diagnosis system according to claim 1, characterized in that: The residual block includes convolution operations and shortcut connections. By introducing shortcut connections, the gradient vanishing and gradient exploding problems in deep neural networks are alleviated, allowing the network to be trained deeper. Assume that the input feature map is , the output feature map is , then the calculation formula of the first convolutional layer is as follows: ;in, is the weight of the first convolutional layer, is the bias term, BN represents batch normalization, ReLU is the activation function, and * represents the convolution operation; the calculation formula for the second convolutional layer is as follows: ;in, are the weights of the second convolutional layer, is the bias term; If the input feature map X and The number of channels is the same, then the shortcut connection directly adds the input X to If the number of channels is different, a 1×1 convolutional layer is needed to adjust the dimension of the input X so that its number of channels is the same as Same; assuming the output of the shortcut connection is , the calculation formula for shortcut connection is as follows: ;in, is the weight of the 1×1 convolutional layer, is the bias term; The output of the convolution operation Output of shortcut connection Add and use ReLU activation function to get the final output : 。 6. The pneumoconiosis staging diagnosis system according to claim 1, characterized in that: The convolution block attention module includes a channel attention module and a spatial attention module, and the input feature map , through global average pooling, aggregation along the spatial dimension, we get a C dimensional vector , and its calculation formula is: ;in At the same time, a C-dimensional vector is obtained along the spatial dimension through global maximum pooling , which is calculated as follows: ;right and Each input is fed into a shared multi-layer perceptron (MLP); the MLP contains one hidden layer whose number of neurons is usually set to the number of input channels. C a smaller proportion of; Let the weight matrix of MLP be and , the bias is and ;for , after MLP processing, we get ,in is the activation function; similarly, for Similar processing is also performed; then the two results after MLP processing are added together, and then a Sigmoid activation function is used to obtain the channel attention weight ,Right now Finally, the channel attention weight With the original input feature map F Perform channel-by-channel multiplication to obtain a feature map enhanced by channel attention ,in Represents element-wise multiplication; Input feature map processed by channel attention module ; First of all, Perform global average pooling and global maximum pooling along the channel dimension respectively, and obtain two 1×H×W Feature map and The calculation formula is as follows: ; ;in 1, ,H, 1, , W ; Will and Splicing is performed on the channel dimension to obtain a 2×H×W The feature map is then passed through a convolution layer with a convolution kernel size of 7×7, a step size of 1, and a padding of 3; let the convolution kernel weight be , the bias is , after the convolution operation, we get The calculation formula is as follows: ； in Represents a splicing operation, is the Sigmoid activation function; finally, the spatial attention weight And the feature map after processing by the channel attention module Perform element-by-element multiplication to obtain the final feature map processed by CBAM 。