Multistage supervision lung X-ray image segmentation method

Abstract

The invention belongs to the technical field of image separation, and particularly discloses a lung X-ray image segmentation method with multi-level supervision. The method comprises the following steps: (1) reading an X-ray image file; (2) preprocessing the image through filtering, correction and the like; and (3) detecting a lung region in the image by using the lung region segmentation algorithm model. According to the algorithm model, U-Net is used as a main network, a multi-scale feature extraction module and an attention feature module are added in Shartcut Consection, the variety diversity of image features is increased, the background noise influence is reduced, meanwhile, a supervision mechanism is added in different branches of the U-Net, the background influence is further reduced, edge details of a lung region are effectively reserved, and accurate segmentation of the lung region is achieved. And (4) marking a lung contour area in the X-ray image, so that a doctor can carry out next detection conveniently. The method achieves the balance between the segmentation precision and the real-time performance, has very high practical value and economic value, and provides a new thought for intelligent medical treatment.

Description

Technical Field

[0001] This invention belongs to the field of image segmentation technology, specifically relating to a multi-level supervised lung X-ray image segmentation method. Background Technology

[0002] With the deepening development of clinical diagnosis and rehabilitation, medical images such as X-rays have become increasingly important. Traditional manual diagnosis can no longer meet the ever-changing needs of medicine, thus generating great interest in developing computer-aided diagnostic systems. In the field of medical image processing, identifying regions of interest (ROIs) is a crucial step. X-ray lung scans have become the preferred method for detecting lung diseases. However, due to the complex characteristics of medical images, such as weak texture, poor contrast, and fluctuations in appearance and location, successfully segmenting lung regions from medical images still presents certain challenges.

[0003] Researchers have proposed numerous methods to improve the accuracy and real-time performance of image segmentation. These methods can be broadly categorized into model-based and deep learning-based approaches. Given the superior detection performance of deep learning-based methods, they have become the mainstream research direction. Currently, many deep learning-based models employ multi-scale structures and attention schemes to acquire more features and suppress background influence. However, these methods still have some shortcomings. On the one hand, the features extracted by these multi-scale structures are independent of each other, limiting the ability to represent global information in the image. On the other hand, the complex structure of the attention module significantly increases the number of network model parameters, reducing operational efficiency. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a multi-level supervised lung X-ray image segmentation method that achieves a balance between segmentation accuracy and real-time performance, possessing high practical and economic value and offering new ideas for smart healthcare.

[0005] The specific technical solution adopted in this invention is as follows:

[0006] A multi-level supervised lung X-ray image segmentation method includes the following steps:

[0007] (1) Read X-ray images from the local disk;

[0008] (2) Preprocess the read X-ray images;

[0009] (3) Input the preprocessed image into a multi-level supervised lung region segmentation network model to extract the lung region;

[0010] (4) Mark the extracted lung regions through step (3) to complete the segmentation of the lung X-ray image;

[0011] Among them, the multi-level supervised lung region segmentation network model uses U-Net as the backbone network and adds a multi-scale feature extraction module and an attention feature module in ShortcutConnection.

[0012] The specific steps for adding the multi-scale feature extraction module and the attention feature module are as follows:

[0013] The input image is converted into a 4-dimensional tensor [N,C,H,W], and then the image is input into the multi-level supervised lung region segmentation network model in the form of a tensor.

[0014] N represents the number of images processed at once, which is set to 1 here;

[0015] C represents the number of image channels. The input image is in RGB representation, therefore C = 3.

[0016] H and W represent the height and width of the image, respectively. After bilinear interpolation, the image scale is uniformly scaled to 512×512, thus obtaining a 4-dimensional tensor tensor = [1,3,512,512].

[0017] The multi-feature extraction module comprises five branches, including three dilated convolutional layers with dilation rates of 6, 12, and 18, a 1×1 convolutional layer, and a global average pooling layer. Finally, a 1×1 convolutional layer is used to reduce computational cost, as shown in the following formula:

[0018] F p =Up(conv(ap(F)))

[0019] F s =conv(F1+F2+F3+F4+F p )

[0020] In the formula, F p It is a global equal pooling feature, F s It is a feature resulting from the fusion of multiple branches.

[0021] F is the intermediate feature map obtained after processing the input tensor through the U-Net backbone network.

[0022] ap is the global average pooling operation.

[0023] conv is the convolution operation.

[0024] Up represents the upsampling operation.

[0025] F1, F2, F3, and F4 are multi-scale feature maps output by the four convolutional layer branches of the multi-feature extraction module.

[0026] The attention feature module includes a spatial attention component and a soft attention component;

[0027] The spatial attention module has three branches, denoted as B1, B2, and B3, where the feature map F processed by the multi-scale feature extraction module... s It can be represented as F C×H×W Where C, H and W represent the number of channels, height and width, respectively, and the feature map processed by the multi-scale feature extraction module is input into the three branches of the spatial attention module;

[0028] Branch B1: To obtain the interaction feature F of size W×H×C in branch B1. B1 The input feature F is remarked 90 degrees counterclockwise along the H-axis to obtain F. r Contextual information is extracted using a parallel structure consisting of average pooling and max pooling, resulting in a feature F of size 2×H×C. rp ; F rp A feature F of size 1×H×C is obtained through a 3×3 convolutional layer. B1 Normalized feature F B1 Attention weights W are generated through a Sigmoid activation layer. B1 Finally, the attention weight W is... B1 To F B1 Multiplication generates attention graphs The specific formula is as follows:

[0029]

[0030] In order to The size is the same as the input feature F, rotated 90 degrees clockwise along the H axis;

[0031] In branch B2, the input features are not rotated; instead, spatial attention is constructed directly. To derive the correlation between width and channels, the input features F are rotated 90 degrees counterclockwise along the W-axis in branch B3.

[0032] The output of the spatial attention module is a combination of three branches, denoted as F. B123 :

[0033]

[0034] The goal of the soft attention module is to achieve F B123 Appropriately assign attention weights to each region in the feature weight map W in the soft attention module. f It is generated through 3×3 and 1×1 convolutional layers and a sigmoid activation layer, and the feature weight feature map W is generated. f With F B123 Multiplying them together yields the final feature map F. final :

[0035] F final =W f ×F B123

[0036] Final feature map F final As input for subsequent modules of U-Net.

[0037] In the preprocessing operation, the input image is I and the output image is O. The image preprocessing process is described by the following formula:

[0038] O = median[rect(I)]

[0039] In the formula, median indicates that the median filtering algorithm is used. The specific process is to create a 3×3 template, sort the pixels in the template according to their values, generate a monotonically increasing or decreasing two-dimensional data sequence, replace all pixels in this area with the median pixel, and finally slide the template until all pixels have been processed.

[0040] rect represents the image geometric correction algorithm, which addresses image distortion, rotation, or scaling caused by factors such as patient position and X-ray machine mechanical factors during image capture.

[0041] The multi-level supervised lung region segmentation network model is trained using weighted cross-entropy loss and binarization loss. The dataset contains 434 images, with 350 images used for training and 84 images used for testing. During training, the loss for each feature branch at each scale is calculated using a supervised method, and the losses at each scale are fused according to their relative proportions to ensure that the sum of the relative proportion losses is 1. The weighted cross-entropy loss is defined as follows:

[0042]

[0043] In the formula, gt represents the true label, p represents the predicted value, and γ is initially set to 5;

[0044] Binarization loss is defined as follows:

[0045]

[0046] In the formula, N is the total number of pixels in the image, and y i It's a real label. This is the model output, where log is the natural logarithm;

[0047] Each branch of the supervision process is assigned weights of w1 = 0.1, w2 = 0.2, w3 = 0.3, and w4 = 0.4. The loss function for each branch is L, defined as follows:

[0048] L i =L IoU +LBCE

[0049] The final loss function formula for the model is:

[0050]

[0051] The beneficial effects of this invention are:

[0052] The multi-level supervised lung region segmentation network model of this invention uses U-Net as the backbone network, and adds a multi-scale feature extraction module and an attention feature module to the ShortcutConnection. The multi-scale feature extraction module uses dilated convolutions with various dilation rates as filters to extract features at multiple scales and lung edge information; the attention module is used to reduce the influence of background noise in the image, thereby effectively segmenting the lung region. Finally, the lung region is marked in the image, which facilitates doctors to carry out further targeted examinations. This invention achieves a balance between segmentation accuracy and real-time performance, and has high practical and economic value, providing a new approach for smart healthcare. Attached Figure Description

[0053] Figure 1 This is a flowchart of the method of the present invention;

[0054] Figure 2 This is a network structure diagram of the present invention;

[0055] Figure 3 This is a structural diagram of the multi-scale feature extraction module and the attention module;

[0056] Figure 4 It is the original X-ray input image;

[0057] Figure 5 This is a segmentation diagram of the lung region;

[0058] Figure 6 This is a comparison chart of the truth value results and the segmentation results; Detailed Implementation

[0059] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0060] This embodiment provides a multi-level supervised lung X-ray image segmentation method, specifically implemented as follows: Figure 1 As shown, it includes the following steps:

[0061] (1) Read the X-ray image from the local disk to complete the data reading.

[0062] (2) Perform preprocessing operations such as filtering and correction on the acquired images. Assuming the input image is I and the output image is O, the image preprocessing process can be described by the following formula:

[0063] O = median[rect(I)]

[0064] In the formula, `median` indicates the use of a median filtering algorithm. The specific process involves creating a 3×3 template, sorting the pixels within the template according to their values ​​to generate a monotonically increasing (or decreasing) two-dimensional data sequence, replacing all pixels in this region with the median pixel, and finally sliding the template until all pixels have been processed. `rect` represents the image correction algorithm, restoring distorted images to a normal state. After preprocessing, image noise and distortion are eliminated, which is beneficial for subsequent processing steps.

[0065] (3) The processed image is input into a multi-level supervised lung region segmentation network model for lung region extraction. The algorithm model structure is as follows: Figure 2 As shown, the model uses U-Net as the backbone network, and adds a multi-scale feature extraction module and an attention feature module in the Shortcut Connection. The specific operation is as follows:

[0066] First, the input image is converted into a 4-dimensional tensor [N,C,H,W]. Then, the image is input into the multi-level supervised lung region segmentation network model in tensor form. N represents the number of images processed at one time, which is set to 1 here; C represents the number of image channels. Here, the image is read in RGB representation, so C=3; H and W represent the height and width of the image, respectively. After bilinear interpolation, the image scale is uniformly scaled to 512×512, so the resulting 4-dimensional tensor tensor is [1,3,512,512].

[0067] like Figure 3 As shown, the multi-feature extraction module comprises five branches, including three dilated convolutional layers with dilation rates of 6, 12, and 18, and a 1×1 convolutional layer, thereby extracting contextual information at different scales. The global contextual information in the multi-scale features extracted from the four branches is constrained. To address this issue, a global average pooling layer is added to provide richer global contextual data. Finally, a 1×1 convolutional layer is used to reduce computational cost. The specific formula is as follows:

[0068] F p =Up(conv(ap(F)))

[0069] F s =conv(F1+F2+F3+F4+F p )

[0070] like Figure 3As shown, the attention feature module consists of two parts: spatial attention and soft attention. Spatial attention emphasizes the lung region in the image to reduce the influence of the background; soft attention assigns different weights to regions in the feature map.

[0071] Spatial Attention Module: The spatial attention module has three branches, denoted as B1, B2, and B3. These three branches are used to extract the correlation between different dimensions. Branch B1 extracts the relationship between height and channels, branch B2 is a general spatial attention mechanism, and branch B3 extracts the relationship between width and channels. The feature map processed by the multi-scale feature extraction module is F. C×H×W Where C, H, and W represent the number, height, and width of channels, respectively. The input is fed into the three branches of the spatial attention module.

[0072] Taking branch B1 as an example, in order to obtain the interaction feature F of size W×H×C in branch B1... B1 The input feature F is remarked 90 degrees counterclockwise along the H-axis to obtain F. r Contextual information is extracted using a parallel structure consisting of average pooling and max pooling, resulting in a feature F of size 2×H×C. rp F rp A feature F of size 1×H×C is obtained through a 3×3 convolutional layer. B1 Normalized feature F B1 Attention weights W are generated through a Sigmoid activation layer. B1 Finally, the attention weight W is... B1 To F B1 Multiplication generates attention graphs The specific formula is as follows:

[0073]

[0074] From a framework perspective, the other branches are structurally similar, but they differ from one another to prevent them from being used interchangeably. For example, in branch B2, the input features are not rotated; instead, spatial attention is constructed directly. To derive the correlation between width and channels, the input features F are rotated 90 degrees counterclockwise along the W-axis in branch B3.

[0075] The output of a spatial attention block is a combination of three branches, denoted as F. B123 :

[0076]

[0077] Feature weight map W f It is generated through 3×3 and 1×1 convolutional layers and a sigmoid activation layer. The feature map F is obtained by multiplying the feature weight feature map with the input. final :

[0078] F final =W f ×F B123

[0079] By adding the above two modules to the U-Net backbone network, the accuracy and computational efficiency of lung region extraction can be improved.

[0080] The model is trained using weighted cross-entropy loss and binarization loss. The dataset contains 434 images, with 350 used for training and 84 for testing. During training, the loss for each feature branch at each scale is calculated using a supervised method, and the losses from each scale are fused according to their relative proportions to ensure that the sum of the relative proportion losses is 1. The weighted cross-entropy loss is defined as follows:

[0081]

[0082] In this formula, gt represents the true label, p represents the predicted value, and γ is initially set to 5.

[0083] Binarization loss is defined as follows:

[0084]

[0085] In this formula, N is the total number of pixels in the image, and y i It's a real label. This is the model output, and log is the natural logarithm.

[0086] The model employs a hierarchical supervised training strategy, assigning different weights to each branch of the supervision process. Since deeper models extract more discriminative features, each branch is assigned weights of w1 = 0.1, w2 = 0.2, w3 = 0.3, and w4 = 0.4, respectively. The loss function for each branch is L, defined as follows:

[0087] L i =L IoU +L BCE

[0088] The final loss function formula for the model is:

[0089]

[0090] (4) Use the algorithm model obtained in step (3) to segment the lung region of the X-ray image and mark the lung region. Figure 4 It is the original X-ray input image. Figure 5 This is a segmentation diagram of the lung region. Figure 6 This is a comparison chart of the true value result and the segmentation result.

[0091] As another embodiment of the present invention, due to differences in equipment and doctor operation, chest X-ray images may include part of the upper limb. This is manifested as the image including part of the upper arm (mainly the humerus and muscle tissue, and generally excluding the forearm) and the blank area between the upper arm and the trunk. As a lung segmentation algorithm, the upper arm image is interference noise that needs to be filtered. Therefore, it can be pre-trimmed by image recognition during preprocessing to remove the upper arm image, which helps to reduce the model's computational consumption and improve efficiency.

[0092] The specific method involves binarizing the image to be preprocessed, and then judging along the pixel rows at the bottom edge of the image (when processing the left side of the image, the pixel rows along the square from the origin of the X-axis to the perpendicular bisector of the image, i.e., the left half of the image), obtaining the set of pixel coordinates with a gray value of 0 (i.e., black area), and obtaining them row by row from bottom to top until no more pixel coordinate sets can be obtained. If the X-coordinate of the leftmost pixel in the last set of pixel coordinates is not zero (i.e., it locates the vertex position of the blank area in the left armpit part of the image), then the X-coordinate of that pixel is used as the cropping point, and the left part of the image is removed along the Y-axis. Due to the physiological structure of the human body, the left part is the upper arm, and there is still a certain black blank area between the lung area to be processed and the newly cropped left edge. Similarly, the right upper arm area is cropped, and the right half of the image is processed along the negative X-axis, thereby achieving image cropping and reducing the workload of subsequent model processing.

Claims

1. A multi-level supervised lung X-ray image segmentation method, characterized in that: Includes the following steps, (1) Read X-ray images from the local disk; (2) Preprocess the read X-ray images; (3) Input the preprocessed image into a multi-level supervised lung region segmentation network model to extract the lung region; (4) Mark the extracted lung regions through step (3) to complete the segmentation of the lung X-ray image; Among them, the multi-level supervised lung region segmentation network model uses U-Net as the backbone network and adds a multi-scale feature extraction module and an attention feature module in the Shortcut Connection.

2. The method for multi-level supervised lung X-ray image segmentation according to claim 1, characterized in that: The specific steps for adding the multi-scale feature extraction module and the attention feature module are as follows: The input image is converted into a 4-dimensional tensor [N,C,H,W], and then the image is input into the multi-level supervised lung region segmentation network model in the form of a tensor. N represents the number of images processed at once, which is set to 1 here; C represents the number of image channels. The input image is in RGB representation, therefore C = 3. H and W represent the height and width of the image, respectively. After bilinear interpolation, the image scale is uniformly scaled to 512×512, thus obtaining a 4-dimensional tensor tensor = [1,3,512,512]. The multi-feature extraction module comprises five branches, including three dilated convolutional layers with dilation rates of 6, 12, and 18, a 1×1 convolutional layer, and a global average pooling layer. Finally, a 1×1 convolutional layer is used to reduce computational cost, as shown in the following formula: F p =Up(conv(ap(F))) F s =conv(F1+F2+F3+F4+F p ) In the formula, F p It is a global equal pooling feature, F s It is a feature resulting from the fusion of multiple branches. F is the intermediate feature map obtained after processing the input tensor through the U-Net backbone network. ap is the global average pooling operation. conv is the convolution operation. Up represents the upsampling operation. F1, F2, F3, and F4 are multi-scale feature maps output by the four convolutional layer branches of the multi-feature extraction module.

3. The method for multi-level supervised lung X-ray image segmentation according to claim 2, characterized in that: The attention feature module includes a spatial attention component and a soft attention component; The spatial attention module has three branches, denoted as B1, B2, and B3. The feature map processed by the multi-scale feature extraction module is F. C×H×W Where C, H and W represent the number of channels, height and width, respectively, and the feature map processed by the multi-scale feature extraction module is input into the three branches of the spatial attention module; Branch B1: To obtain the interaction feature F of size W×H×C in branch B1. B1 , input feature F C×H×W Remark 90 degrees counterclockwise along the H-axis to obtain F. r Contextual information is extracted using a parallel structure consisting of average pooling and max pooling, resulting in a feature F of size 2×H×C. rp ; F rp A feature F of size 1×H×C is obtained through a 3×3 convolutional layer. B1 Normalized feature F B1 Attention weights W are generated through a Sigmoid activation layer. B1 Finally, the attention weight W is... B1 To F B1 Multiplication Generate attention map The specific formula is as follows: In order to The size is the same as the input feature F, rotated 90 degrees clockwise along the H axis; In branch B2, the input features are not rotated, but spatial attention is directly constructed; in order to derive the correlation between width and channels, the input features F are rotated 90 degrees counterclockwise along the W axis in branch B3. The output of the spatial attention module is a combination of three branches, denoted as F. B123 :

4. The method for multi-level supervised lung X-ray image segmentation according to claim 3, characterized in that: The goal of the soft attention module is to achieve F B123 Appropriately allocate attention weights to each region within the process; Feature weight map W in the soft attention module f It is generated through 3×3 and 1×1 convolutional layers and a sigmoid activation layer, and the feature weight feature map W is generated. f With F B123 Multiplying them together yields the final feature map F. final : F final =W f ×F B123 Final feature map F final As input for subsequent modules of U-Net.

5. The method for multi-level supervised lung X-ray image segmentation according to claim 1, characterized in that: In the preprocessing operation, the input image is I and the output image is O. The image preprocessing process is described by the following formula: O = median[rect(I)] In the formula, median indicates that the median filtering algorithm is used. The specific process is to create a 3×3 template, sort the pixels in the template according to their values, generate a monotonically increasing or decreasing two-dimensional data sequence, replace all pixels in this area with the median pixel, and finally slide the template until all pixels have been processed. rect represents the image geometric correction algorithm.

6. The method for multi-level supervised lung X-ray image segmentation according to claim 1, characterized in that: The multi-level supervised lung region segmentation network model is trained using weighted cross-entropy loss and binarization loss. The dataset contains 434 images, with 350 images used for training and 84 images used for testing. During training, the loss for each feature branch at each scale is calculated using a supervised method, and the losses at each scale are fused according to their relative proportions to ensure that the sum of the relative proportion losses is 1. The weighted cross-entropy loss is defined as follows: In the formula, gt represents the true label, p represents the predicted value, and γ is initially set to 5; Binarization loss is defined as follows: In the formula, N is the total number of pixels in the image, and y i It's a real label. This is the model output, where log is the natural logarithm; Each branch of the supervision process is assigned weights of w1 = 0.1, w2 = 0.2, w3 = 0.3, and w4 = 0.

4. The loss function for each branch is L, defined as follows: L i =L IoU +L BCE The final loss function formula for the model is:

Images