Automatic segmentation method of gross tumor target volume in non-small cell lung cancer

Abstract

The present invention provides a method for automatic segmentation of a gross tumor target area of ​​non-small cell lung cancer, comprising: a step of obtaining a case image containing a gross tumor area of ​​non-small cell lung cancer; a step of processing the case image by a convolutional neural network including at least one attention residual block to obtain a feature map of the case image; and a step of obtaining an automatic segmentation result of the tumor target area of ​​the case image based on the feature map, wherein in the attention residual block, the space and channel cascade attention layer includes a space attention stage and a channel attention stage, both of which respectively obtain a compressed global maximum pooling feature map and a global average pooling feature map and splice them, and at the same time use variable scale convolution to realize channel fusion, and multiply the activated weight back to the feature map to obtain a feature map after the spatial feature and channel feature are recalibrated. The method of the present invention can obtain a better automatic segmentation effect.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to the field of automatic segmentation of medical images based on deep learning, which is applied to the automatic segmentation of gross tumor target areas of non-small cell lung cancer in small sample data sets. Background Art

[0002] The mortality rate of lung cancer ranks first in the world, and the incidence rate is also increasing year by year. Among them, non-small cell lung cancer accounts for about 80-85% of lung cancer. For patients who cannot undergo surgery or refuse to undergo surgery, radical radiotherapy is still one of the effective means of treating non-small cell lung cancer. In radiotherapy, physicians need to segment tumor tissues such as gross tumor target areas according to medical imaging information, and then physicists formulate radiotherapy plans. The manual segmentation of target areas by physicians is labor-intensive and inefficient, and the segmentation results are easily affected by subjective experience, working status and other factors, resulting in problems of wrong segmentation and missed segmentation. If the actual tumor tissue is omitted when the segmentation area is segmented, the tumor cannot be effectively treated during radiotherapy; if the surrounding normal tissue is included when the segmentation area is segmented, unnecessary radiation damage will be caused to the normal tissue. It can be seen that the segmentation accuracy directly affects the effect of radiotherapy. Therefore, in order to improve the work efficiency of doctors and realize high-precision radiotherapy automation and intelligence, it is necessary to study the automatic segmentation method of non-small cell lung cancer target areas.

[0003] Traditional CT image segmentation technology designs specific feature extraction algorithms based on the contrast, brightness or texture of the image, and then combines it with a specific classifier to complete the automatic image segmentation task. Due to the influence of partial volume effect, motion artifacts, etc., traditional image segmentation methods are not effective in CT medical image segmentation. With the development of graphics processors, it has become possible to use deep neural networks for automatic CT image segmentation. Deep learning uses a deep neural network architecture with multiple hidden layers between the input and output layers of the network architecture to learn features from the data set by modeling complex nonlinear relationships.

[0004] The commonly used medical image segmentation network is mainly a U-shaped network represented by U-Net. The U-Net network structure is divided into a downsampling stage and an upsampling stage. There are only convolutional layers and pooling layers in the network structure, and no fully connected layers. The shallower high-resolution layers in the network are used to solve the problem of pixel positioning, and the deeper layers are used to solve the problem of pixel classification, so as to achieve image semantic level segmentation. Unlike the FCN network, the upsampling stage and the downsampling stage of U-Net use the same number of convolution operations, and use a jump-layer connection structure to connect the downsampling layer to the upsampling layer, so that the features extracted by the downsampling layer can be directly passed to the upsampling layer, which makes the pixel positioning of the U-Net network more accurate and the segmentation accuracy higher. Summary of the invention

[0005] However, the simple module composed of convolution and pooling used in U-Net has poor feature extraction capabilities and requires large sample data sets for training. However, it is difficult to obtain a high-quality, standardized large sample set of non-small cell lung cancer target areas. In addition, since the non-small cell lung cancer target area occupies a small area in the CT image, the positive and negative samples of each batch of images sent to the network are unbalanced, the network gradient disappears during training, and the network is difficult to extract effective features. Therefore, developing a module that can effectively extract target area features based on a small sample data set to improve the accuracy of automatic target segmentation is a technical problem that needs to be overcome urgently.

[0006] In view of the problems of low segmentation accuracy of non-small cell lung cancer target area in existing segmentation algorithms and great difficulty in training neural networks, the present invention provides a method for automatic segmentation of gross tumor target areas of non-small cell lung cancer, which enables a deep learning network to extract more features from unclear boundaries, effectively alleviating the problem of gradient vanishing, thereby achieving better automatic segmentation effects.

[0007] In order to solve the above technical problems, the present invention provides a method for automatic segmentation of a gross tumor target area of ​​non-small cell lung cancer, characterized in that it includes: a step of obtaining a case image containing a gross tumor area of ​​non-small cell lung cancer; a step of processing the case image through a convolutional neural network to obtain a feature map of the case image, and a step of obtaining an automatic segmentation result of the gross tumor target area of ​​non-small cell lung cancer of the case image based on the feature map, wherein the convolutional neural network includes at least one attention residual block, each attention residual block includes: a convolution layer, a batch normalization layer, a space and channel cascade attention layer, and a jump layer connection from block input to block output, in the attention residual block, the space and channel cascade attention layer includes a spatial attention stage and a channel attention stage, both of which perform global maximum pooling and global average pooling along the channel dimension of the feature map and the two spatial dimensions of width and height, respectively, to obtain a compressed global maximum pooling feature map and a global average pooling feature map and splice them, and use variable scale convolution to realize channel fusion, and multiply the activated weights back to the feature map to obtain a feature map after the spatial features and channel features are recalibrated.

[0008] Preferably, the convolutional neural network includes a U-shaped network of arbitrary depth, the input of the attention residual block is the feature map of the previous module in the convolutional neural network, the features are recalibrated through the convolution layer, the batch normalization layer, and the spatial and channel cascade attention layer, and spliced ​​with the feature map of the previous module connected through the jump layer, and the spliced ​​feature map is output.

[0009] Preferably, the attention residual block is any one of an identical attention residual block and a convolutional attention residual block, wherein in the identical attention residual block, the convolutional layer, the batch normalization layer, and the spatial and channel cascaded attention layer are cascaded in sequence, and the jump-layer connection from the block input to the block output is a through connection; in the convolutional attention residual block, the convolutional layer, the batch normalization layer, and the spatial and channel cascaded attention layer are cascaded in sequence, and the jump-layer connection from the block input to the block output is a connection through a connected convolutional layer.

[0010] Preferably, in the constant attention residual block, the convolution layer is composed of multiple 3×3 convolution kernels with a step size of 1, and when the weighted sliding average method is used, the decay rate (momentum) of the batch normalization layer is set to 0.99; in the convolutional attention residual block, the convolution layer is composed of multiple 3×3 convolution kernels with a step size of 2, and when the weighted sliding average method is used, the decay rate (momentum) of the batch normalization layer is set to 0.99, and the connection convolution layer is composed of multiple 3×3 convolution kernels with a step size of 2.

[0011] Preferably, the convolutional neural network includes a U-shaped network of arbitrary depth, and in the downsampling stage of the U-shaped network, each downsampling layer includes one of the constant attention residual blocks and one convolutional attention residual block, and in the upsampling stage of the U-shaped network, each upsampling layer includes one of the constant attention residual blocks and one upsampling operation.

[0012] Preferably, in the spatial attention stage, the feature map is subjected to global maximum pooling and global average pooling along the channel dimension to obtain two feature maps z s1 , z s2 , concatenate the two feature maps, use a convolution kernel with a step size of 1 and 3×3 to compress the two images into one image, and use the sigmoid activation function to map the importance of each position of the spatial feature to the interval [0,1], that is, to obtain the probability value of the tumor target area in this area. After activation, the spatial weight feature map of the feature map is multiplied back to the original feature map to obtain the feature map after the spatial feature is recalibrated. The detailed definition is as follows:

[0013] z s1 =MAX(u s (i),i∈[1,C]),z∈R s

[0014]

[0015] C represents the number of channels, R s The latter represents the spatial set of feature map sizes, u s (i) represents the feature map vector space for the i-th channel,

[0016] The channel attention stage performs global maximum pooling and global average pooling along the width and height of the feature map to obtain a 1×1×C channel feature map z after the feature map is compressed. c1 , z c2 , concatenate the two columns of feature maps, and use a variable-scale convolution kernel with a step size of 1, 1×1×k to fuse the two columns of channels. The size of the variable-scale convolution kernel is related to the number of channels N, that is, k=log2C. The convolution kernel multiplies the activated non-small cell lung cancer channel weight back to the feature map to obtain the feature map after the channel feature is recalibrated. The detailed definition is as follows:

[0017] z c1 =MAX(u c (i, j), i∈[1, W] j∈[1, H]), z∈R C

[0018]

[0019] The variable-scale convolution uses a variable-scale k×1×1 convolution kernel, where k=log2C. The scale of the convolution kernel increases with the number of channels in the feature map, and the receptive field also increases accordingly. C represents the number of channels, R C The latter represents the spatial set of channel sizes, H, W represent the height and width of the feature map respectively, i, j represent the coordinates of the feature map, and u c (i, j) represents the channel of feature map coordinates i, j.

[0020] Preferably, the case image is a CT image of the patient's lungs with a slice thickness and a slice interval of 5 mm.

[0021] Preferably, the method further includes a step in which an expert performs target area segmentation on the patient image to obtain a segmentation benchmark truth value; and a network model training step: using the training set data including the first specified number of case images and the validation set data including the second specified number of case images to train the convolutional neural network model, using the validation set to evaluate the network performance at the end of each training, and adjusting the learnable parameters to obtain the target feature distribution, and storing the network model with the smallest calculated error value between the automatic segmentation result and the segmentation benchmark truth value.

[0022] By adopting the technical solution of the present invention, a method for automatic segmentation of the gross tumor target area of ​​non-small cell lung cancer can be provided, which enables the deep learning network to extract more features from unclear boundaries, effectively alleviate the problem of gradient vanishing, and thus obtain better automatic segmentation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1is a flow chart showing the method for automatic segmentation of a gross tumor target area of ​​non-small cell lung cancer of the present invention;

[0024] Figure 2 It is a schematic diagram of two attention residual block structures;

[0025] Figure 3 It is a schematic diagram to illustrate the spatial and channel attention cascade mechanism;

[0026] Figure 4 It is a computer image representing the original image in the dataset and the ground truth;

[0027] Figure 5 is a schematic diagram showing the structure of a U-Net benchmark network as a comparative example;

[0028] Figure 6 As an example, Figure 4 The structure of the network after the baseline network is optimized is shown;

[0029] Figure 7 Dice indicator box plot of the baseline network and the optimized network;

[0030] Figure 8 The HD95 indicator box plot of the baseline network and the optimized network;

[0031] Fig. 9 ASD indicator box plot of the baseline network and the optimized network;

[0032] Fig.10 It is a schematic diagram comparing the segmentation results of the baseline network and the optimized network. DETAILED DESCRIPTION

[0033] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementations. This part describes the present invention based on the accompanying drawings, but the examples described in the accompanying drawings are only schematic diagrams of the segmentation method, and other embodiments developed based on the examples of the present invention are all within the protection scope of the present invention.

[0034] Figure 1 FIG. 4 is a flow chart showing the method for automatically segmenting a large tumor target area of ​​non-small cell lung cancer according to the present invention. Figure 1As shown, the method for automatic segmentation of a macroscopic tumor target area of ​​non-small cell lung cancer of the present invention comprises a step (S1) of acquiring a case image containing a macroscopic tumor area of ​​non-small cell lung cancer; a step (S2) of processing the case image by a convolutional neural network including at least one attention residual block to obtain a feature map of the case image, and a step (S3) of obtaining an automatic segmentation result of the macroscopic tumor target area of ​​non-small cell lung cancer of the case image based on the feature map. The improvement of the present invention over the prior art is that the convolutional neural network includes at least one attention residual block, and each attention residual block includes: a convolutional layer, a batch normalization layer, a spatial and channel cascaded attention layer, and a jump layer connection from a block input to a block output.

[0035] Below, a specific implementation of the present invention is described by taking a convolutional neural network as a U-net as an example. However, the convolutional neural network in the present invention is not limited to U-net, but can also be other model structures extended from the U-net, such as Res-Unet, Dense-Unet, etc.

[0036] Considering that the entire U-shaped network needs to gradually compress the scale of the feature map, it is also necessary to maintain the original scale to extract features while keeping the information contained in the network feature map steadily reduced. Therefore, in one embodiment of the present invention, a convolutional attention residual block that reduces the scale of the feature map to extract features and an equal attention residual block are designed for the U-shaped network. The basic structures of these two residual blocks are similar, such as Figure 1 shown.

[0037] Figure 2 is a schematic diagram of two attention residual block structures. Figure 2 In the example, the input of the constant attention residual block is the feature map of the previous module in the neural network. After the convolution layer, batch normalization layer, and spatial and channel cascade attention layer, the feature map is recalibrated, and the feature map is spliced ​​with the feature map of the input module through the jump layer connection, and the output is the spliced ​​feature map. The convolutional attention residual block works to change the size of the feature map so that the image information remains unchanged overall. The feature map of the previous layer is convolved with a stride of 2 and spliced ​​with the feature map that has passed through the convolution batch layer, normalization layer, and spatial and channel cascade attention layer, and the feature map with changed size is output.

[0038] The spatial and channel cascade attention layers are used in the module to obtain the features of the input image. The spatial attention mechanism simulates the prior knowledge outlined by the physician. Central non-small cell lung cancer is mainly distributed in the area close to the mediastinum, and peripheral non-small cell lung cancer is mainly distributed in the area around the inner wall of the chest cavity. The specific lesion area in the CT image is determined by training the spatial attention layer, and more attention is allocated, ignoring other irrelevant areas. Channel attention recalibrates the weights of the feature maps, allocates more attention to the feature maps containing more segmentation information, suppresses irrelevant feature maps, and makes the network more suitable for the segmentation of non-small cell lung cancer.

[0039] In one embodiment of the present invention, the constant attention residual block is composed of a 3×3 convolutional layer with a stride of 1, a batch normalization layer, and an attention layer cascaded, and the input is connected to the output by a jump-layer connection. The convolutional attention residual block is composed of a 3×3 convolutional layer with a stride of 2, a batch normalization layer, and an attention layer, and the input is connected to the output through a 3×3 convolutional layer with a stride of 2 through a jump-layer connection.

[0040] The overall structure of the residual block can be expressed as:

[0041] y l =h(x l )+F(x l ,W l )

[0042] x l+1 =f(y l )

[0043] where x l and x l+1 represents the input of two adjacent residual blocks, F(x l ,W l ) represents the network output after convolutional layer, batch normalization layer and channel and spatial cascade attention mechanism, W l represents the convolution kernel in the network, f represents the activation function, and the training goal of the residual block is to form an identity mapping. l ) represents the output after one layer of convolution, y l Represents the network output after splicing.

[0044] The batch normalization layer is an approximate whitening preprocessing operation. It is embedded in the network so that the feature map input of each layer is normalized to the standard normal distribution. Then, the feature distribution to be learned by the original network is restored by introducing learnable parameters. Since the deep learning network mainly learns the target feature distribution, the batch normalization layer is nested in each layer to make the data distribution entering the middle layer relatively fixed, alleviating the problem of gradient disappearance in the neural network. The specific operation is as follows:

[0045] First, calculate the average value u of a batch feature map of the input BN layer B :

[0046]

[0047] Where m is the number of feature maps, x i is the i-th feature map. Then calculate the sample deviation σ of the feature map input to the BN layer B :

[0048]

[0049] Normalize the sample data of the feature map:

[0050]

[0051] In order to ensure that the feature distribution of this layer can be restored, the parameters of the training distribution enable the network to:

[0052]

[0053] γ and β refer to the learned proportional coefficients and biases that restore the original data distribution under this layer.

[0054] When the network data is forward propagated, the data distribution of the original layer can be restored through the distributed parameters, and the parameters are trained through the chain derivative rule when the error is back propagated. When the test set is used as input, the sliding average method is used to calculate the mean of the small batch, and the decay rate is set to 0.99.

[0055] The network attention layer is an attention structure that designs channel and spatial attention cascades based on the characteristics of non-small cell lung cancer CT images. It refers to determining specific lesion areas in CT images through prior knowledge and allocating more attention, while ignoring other irrelevant areas. The network uses a residual module embedded with an attention mechanism, in which the spatial and channel cascade attention mechanism is as follows: Figure 2 shown.

[0056] Figure 3 This is a schematic diagram for explaining the cascade mechanism of spatial and channel attention. The spatial and channel attention network quantifies the information contained in the feature map and assigns more weight to areas that are prone to cancer to segment areas that are adhered to normal tissue and difficult to identify. Through the cascade channel attention, channels with more significant features are activated. The specific operations are as follows:

[0057] Attention to the feature map space. When the feature map of C×W×H dimension enters the module, each channel of the feature map is firstly global max pooled to obtain the feature map z s1 :

[0058] zs1 =MAX(u s (i), i∈[1,C]), z∈R s

[0059] Perform global average pooling z on each channel of the feature map s2 :

[0060]

[0061] C represents the number of channels, R s The latter represents the spatial set of feature map sizes, u s (i) represents the feature map vector space of the i-th channel, which is compressed into two 1×W×H feature maps, which respectively measure the significant information and average information at the image scale. The two feature maps are aggregated and sent to the convolution layer. The probability value of the cancerous area may appear in each pixel of W×H is obtained through the activation function. The 1×W×H probability is multiplied back to the C×W×H feature map. The area prone to cancer is more likely to be activated by the next layer.

[0062] Attention is paid to the feature map channel. After the C×W×H feature map of the first step of spatial attention, global maximum pooling of the channel dimension is performed along the width and height directions respectively. c1 :

[0063] z c1 =MAX(u c (i, j), i∈[1, W] j∈[1, H]), z∈R C

[0064] Global average pooling z in the channel dimension c2 :

[0065]

[0066] C represents the number of channels, R C The latter represents the spatial set of channel sizes, H, W represent the height and width of the feature map respectively, i, j represent the coordinates of the feature map, and u c (i, j) represents the channel of feature map coordinates i, j. The channel is compressed into two columns C×1×1 to obtain significant information and average information at the channel scale. Considering that the number of channels C will surge with the deepening of the number of layers, a variable scale k×1×1 convolution kernel is used, where k=log2C. As the number of channels in the feature map increases, the receptive field also increases, realizing information fusion between channels, obtaining the probability value of each channel C containing lesion information, and multiplying the probability of C×1×1 back to the C×W×H feature map, giving more weight to the channels of significant features.

[0067] (Example)

[0068] Next, an example of the above-mentioned embodiment of the present invention is described. It involves processing a case image by a convolutional neural network to obtain a feature map of the case image, and provides the following example of target segmentation of a non-small cell lung cancer CT image. The computer CPU used in this example is Intel (R) Core (TM) i7-8700, 3.2GHZ, the GPU is NVIDIA GeForce GTX 1070 Ti, and the memory is 23.0GB.

[0069] 1. Establish the overall network model

[0070] At present, the image segmentation network mainly uses the encoding and decoding structure. In the encoding stage of the image, the image resolution needs to be gradually reduced. Therefore, it is necessary to use the identity-identity attention residual block to ensure that the resolution of the feature map remains unchanged, and use the identity-convolution attention residual block to reduce the scale of the feature map. The two residual blocks are used alternately to deepen the number of network layers. After obtaining the underlying features of the image, the deconvolution method is used to gradually restore the image size to complete the image segmentation and obtain the image probability map. The probability of the pixel point determines whether the point belongs to the non-small cell lung cancer target area or the background.

[0071] 2. Spatial and channel cascaded attention residual blocks

[0072] In the spatial attention stage, the feature map is subjected to global maximum pooling and global average pooling along the channel dimension to obtain two feature maps z s1 , z s2 , concatenate the two feature maps, compress the two images into one image using a convolution kernel with a step size of 1 and 3×3, and use the sigmoid activation function to map the importance of each position of the spatial feature to the interval [0,1], that is, obtain the probability value of the tumor target area in this area. After activation, multiply the spatial weight feature map of the feature map back to the original feature map to obtain the feature map after the spatial feature is recalibrated.

[0073] The detailed definitions are as follows:

[0074] z s1 =F sq (u s )=MAX(u s (i), i∈[1,C]), z∈R s

[0075]

[0076] In the formula, C represents the number of channels, R s The latter represents the spatial set of feature map sizes, u s (i) represents the feature map vector space for the i-th channel.

[0077] In the channel attention stage, global maximum pooling and global average pooling are performed along the width and height of the feature map to obtain a 1×1×C channel feature map z after feature map compression. c1 , z c2 , concatenate the two columns of feature maps, and use a variable-scale convolution kernel with a step size of 1, 1×1×k to fuse the two columns of channels. The size of the variable-scale convolution kernel is related to the number of channels N, that is, k=log2C. The convolution kernel multiplies the activated non-small cell lung cancer channel weight back to the feature map to obtain the feature map after the channel feature is recalibrated. The detailed definition is as follows:

[0078] z c1 =F sq (u c )=MAX(u c (i, j), i∈[1, W] j∈[1, H]), z∈R C

[0079]

[0080] In the formula, C represents the number of channels, R C The latter represents the spatial set of channel sizes, H, W represent the height and width of the feature map respectively, i, j represent the coordinates of the feature map, and u c (i, j) represents the channel of feature map coordinates i, j.

[0081] 3. Obtaining training and test images

[0082] The case images used for training and testing in this embodiment are CT images, which are obtained by spiral tomography from the level of the patient's larynx to the bottom of the lungs using a CT simulator, with a slice thickness and slice interval of 5 mm. After segmentation by an experienced expert and peer review by two other experts, the segmentation information of the gross tumor target area is obtained.

[0083] Parse the DICOM file containing the patient information, map the pixel values ​​of the CT image to the range of 0-255, adjust the window width to 400, and the window level to 40. Parse the segmentation file of this patient, find the contour point sequence corresponding to the gross tumor target area, and map the sequence in the absolute coordinate system to the 512×512 image coordinate system. Fill the image contour to obtain the CT mask image in png format. The processed CT image and its corresponding mask image are shown in the figure. Figure 4 As shown, Figure 4 (a) in the figure represents the original image in the dataset. Figure 4 (b) in the figure represents the segmentation benchmark truth value (the corresponding tumor target area position in the CT image).

[0084] 4. Evaluate segmentation results

[0085] The Dice parameter is an evaluation index to measure the image segmentation effect. The calculation formula is as follows:

[0086]

[0087] TP (true positive) represents the true positive area in the image, FP (false positive) represents the false positive area in the image, and FN (false negative) represents the false negative area in the image. Dice represents the ratio of the intersection and union of the predicted value and the true value. The higher the Dice value, the more similar the automatic segmentation of the organ at risk is to the baseline value of the manual segmentation by the doctor.

[0088] Hausdorff distance (HD) is the distance of the maximum mismatch between the metric space and the true subset. In order to avoid the influence of outlier noise points on the evaluation results, the data of the top 5% (HD95) of the distance between each point set from large to small are usually selected. The calculation formula of HD is as follows:

[0089] H(A,B)=max(h(A,B),h(B,A))

[0090] in:

[0091] h(A,B)=max(a∈A)min(b∈B)||ab||

[0092] h(B,A)=max(b∈B)min(a∈A)||ba||

[0093] Average Symmetric Surface Distance (ASSD) measures the average distance between two point sets and is calculated as follows:

[0094]

[0095] In this embodiment, U-Net is used as a reference network, i.e., a comparative example. The structure of the reference network is as follows: Figure 5 As shown in the figure, the left side of U-Net is the downsampling stage, which includes multiple cascaded downsampling layers from top to bottom, each of which includes a convolution layer and a maximum pooling layer. The right side of U-Net is the upsampling stage, which includes multiple cascaded upsampling layers from bottom to top, each of which includes a convolution layer and an upsampling operation.

[0096] The structure of the optimized network in the embodiment of the present invention is as follows: Figure 6As shown. The embodiment of the present invention is a network that optimizes the above-mentioned benchmark network. Specifically, in the downsampling stage, the convolutional layer with constant feature map scale in the network is replaced with a constant attention residual block, and the maximum pooling layer with reduced feature map scale is replaced with a convolutional attention residual block. In the upsampling stage, the convolutional layer with constant feature map scale is replaced with a constant attention residual block to construct an optimized network.

[0097] 115 medical record images (69 in the training set, 23 in the validation set, and 23 in the test set) were used in this embodiment. The number of training generations was 30, and the training time for each generation was about 9.86 minutes. At the end of each training, the validation set was used to evaluate the network performance, and the learnable parameters were adjusted to obtain the target feature distribution, and the network model with the smallest calculated error value (loss value) between the automatic segmentation result and the benchmark true value was stored. ImageDataGenerator was used to perform random data enhancement on each batch, including rotation, horizontal or vertical flipping, scaling, translation, and scale transformation. The average of the indicators calculated for all CT slices of each patient in the test set was used as the segmentation index for this case. The indicator calculation results are shown as follows. Figure 7 to Figure 9 As shown:

[0098] Figure 7 It is the Dice indicator box plot of the baseline network and the optimized network. Figure 7 It can be seen that although the quartile of the optimized Dice box plot is 0.71, which is smaller than the quartile of the baseline network (0.74), and the upper quartile is 0.75, which is smaller than the upper quartile of the baseline network (0.79), the single-case Dice maximum value is 0.86 and the lower quartile is 0.64, both of which are better than the baseline network after the spatial and channel cascade attention mechanism is optimized. By reducing the fitting ability of the mask map of samples with a specific distribution, the network consistency is stronger and the segmentation results are more stable.

[0099] Figure 8 It is the HD95 indicator box plot of the baseline network and the optimized network. Figure 8 The HD95 in is a measure of the maximum difference in distance between two point sets. A HD95 of 0 means that the two point sets completely overlap. As can be seen from the figure, the HD95 values ​​before and after optimization are 17.72 mm and 17.69 mm, which are basically the same, but the upper and lower quartiles are reduced. In particular, the HD95 value of the segmentation result of a single patient is higher than that before optimization. The introduction of the spatial and channel cascade attention mechanism has little effect on the distortion caused by the segmentation. This may be due to the use of a benchmark network with the same structure, and the pixel area features that are mis-segmented by the benchmark network have certain commonalities.

[0100] Fig. 9 It is the ASD indicator box plot of the baseline network and the optimized network. Fig. 9 The ASSD in is a measure of the difference between the single pixel points between the mask image of the automatic segmentation of the gross tumor target area of ​​non-small cell lung cancer and the ground truth (GT). Fig. 9 It can be seen that the quartile of the baseline network is 5.81mm, which is basically the same as the quartile of the optimized network of 5.52mm, but the upper quartile of the baseline network is 11.74mm, which is much larger than the 6.5mm of the optimized network. The difference between the lower quartile and the upper quartile of the optimized network is only 2.13mm. The segmentation result of the optimized network should be a certain expansion around the GT value.

[0101] Fig.10 This is a schematic diagram comparing the segmentation results of the baseline network and the optimized network (the upper row is the segmentation result of the baseline network, the lower row is the segmentation result of the optimized network, the solid line is the GT value, and the dotted line is the automatic segmentation result)

[0102] according to Fig.10 It can be seen that after the introduction of the spatial and channel cascade attention mechanism optimization, the pixel-level segmentation results are more detailed; when dealing with tumor targets adhered to the chest wall or other tissues, the difference between the automatic segmentation contour and the GT value is smaller than the difference between the automatic segmentation contour and the GT value of the baseline network.

[0103] Through the above evaluation indicators, it can be concluded that the introduction of spatial and channel cascade attention mechanism can effectively improve the automatic segmentation accuracy of the network.

Claims

1. A method for automatic segmentation of a macroscopic tumor target area of ​​non-small cell lung cancer, characterized in that: include: Steps of acquiring a case image containing a gross tumor region of a non-small cell lung cancer; The step of processing the case image by a convolutional neural network to obtain a feature map of the case image, and The step of obtaining the automatic segmentation result of the gross tumor target area of ​​non-small cell lung cancer in the case image based on the feature map, The convolutional neural network is a U-shaped network with arbitrary depth, and the convolutional neural network includes at least one attention residual block, each of which includes: a convolutional layer, a batch normalization layer, a spatial and channel cascaded attention layer, and a jump layer connection from block input to block output. In the attention residual block, the spatial and channel cascade attention layer includes a spatial attention stage and a channel attention stage, both of which perform global maximum pooling and global average pooling along the channel dimension and the width and height spatial dimensions of the feature map, respectively, to obtain the compressed global maximum pooling feature map and the global average pooling feature map and splice them, and use variable scale convolution to achieve channel fusion, and multiply the activated weights back to the feature map to obtain the feature map after the spatial features and channel features are recalibrated; The attention residual block is any one of an identical attention residual block and a convolutional attention residual block, wherein, In the constant attention residual block, the convolution layer, the batch normalization layer, and the spatial and channel cascaded attention layer are cascaded in sequence, and the jump layer connection from the block input to the block output is a through connection. In the convolutional attention residual block, the convolution layer, the batch normalization layer, and the spatial and channel cascaded attention layer are cascaded in sequence, and the jump layer connection from the block input to the block output is a connection through a connected convolution layer, In the constant attention residual block, the convolution layer consists of multiple 3×3 convolution kernels with a step size of 1. When using the weighted moving average method, the decay rate of the batch normalization layer is set to 0.

99. In the convolutional attention residual block, the convolution layer consists of multiple 3×3 convolution kernels with a step size of 2. When using the weighted sliding average method, the attenuation rate of the batch normalization layer is set to 0.99, and the connection convolution layer consists of multiple 3×3 convolution kernels with a step size of 2. In the downsampling stage of the U-shaped network, each downsampling level includes one constant attention residual block and one convolutional attention residual block. In the upsampling stage of the U-shaped network, each upsampling level includes 1 of the identical attention residual blocks and 1 upsampling operation.

2. The method for automatic segmentation of a macroscopic tumor target area of ​​non-small cell lung cancer according to claim 1, characterized in that: The input of the attention residual block is the feature map of the previous module in the convolutional neural network. The features are recalibrated through the convolution layer, the batch normalization layer, and the spatial and channel cascade attention layer, and are spliced ​​with the feature map of the previous module connected through the jump layer, and the spliced ​​feature map is output.

3. The method for automatic segmentation of a macroscopic tumor target area of ​​non-small cell lung cancer according to claim 1 or 2, characterized in that: The spatial attention stage performs global maximum pooling and global average pooling on the feature map along the channel dimension to obtain two feature maps z s1 , z s2 , concatenate the two feature maps, use a convolution kernel with a step size of 1 and 3×3 to compress the two images into one image, use the sigmoid activation function to map the importance of each position of the spatial feature to the interval [0, 1], that is, obtain the probability value of the tumor target area in this area, and after activation, multiply the spatial weight feature map of the feature map back to the original feature map to obtain the feature map after the spatial feature is recalibrated. The detailed definition is as follows: z s1 =MAX(u s (i),i∈[1,C]),z∈R s C represents the number of channels, R s Represents the spatial set of feature map sizes, u s (i) represents the feature map vector space for the i-th channel, The channel attention stage performs global maximum pooling and global average pooling along the width and height of the feature map to obtain a 1×1×C channel feature map z after the feature map is compressed. c1 , z c2 , concatenate the two columns of feature maps, use a variable-scale convolution kernel with a step size of 1, 1×1×k to fuse the two columns of channels, where the size of the variable-scale convolution kernel is related to the number of channels N, that is, k=log2C. The convolution kernel multiplies the activated non-small cell lung cancer channel weight back to the feature map to obtain the feature map after the channel feature is recalibrated. The detailed definition is as follows: z c1 =MAX(u c (i,j),i∈[1,W]j∈[1,H]),z∈R C The variable-scale convolution uses a variable-scale k×1×1 convolution kernel, where k=log2C. The scale of the convolution kernel increases with the number of channels in the feature map, and the receptive field also increases accordingly. C represents the number of channels, R C Represents the spatial set of channel sizes, H, W represent the height and width of the feature map respectively, i, j represent the coordinates of the feature map, u c (i, j) represents the channel of feature map coordinates i, j.

4. The method for automatic segmentation of a macroscopic tumor target area of ​​non-small cell lung cancer according to claim 1 or 2, characterized in that: The case image is a CT image of the patient's lungs, with a slice thickness and a slice interval of 5 mm.

5. The method for automatic segmentation of a macroscopic tumor target area of ​​non-small cell lung cancer according to claim 1 or 2, characterized in that: Also includes: The step of obtaining a segmentation benchmark truth value by having an expert perform target segmentation on the patient image; and Network model training steps: use training set data including a first specified number of case images and validation set data including a second specified number of case images to train the convolutional neural network model, use the validation set to evaluate the network performance at the end of each training, adjust the learnable parameters to obtain the target feature distribution, and store the network model with the smallest calculated error between the automatic segmentation result and the segmentation benchmark true value.

Images