A 3D CT image lesion segmentation method based on self-attention feedback mechanism

Abstract

This invention provides a method for lesion segmentation in 3D CT images based on a self-attention feedback mechanism. The method involves collecting 3D CT scan images of patients, performing normalization and enhancement processing on the image data to delineate lesion regions, constructing a self-attention module, designing a self-attention feedback mechanism, and building an encoder-decoder deep learning neural network. The network is trained using five-fold cross-validation, and the resulting model is a 3D CT lesion segmentation model. This trained model is then used to segment gastric cancer lesions in patient CT images. This method solves the problems of poor and inaccurate CT segmentation accuracy.

Description

Technical Field

[0001] This invention belongs to the field of medical imaging technology, and in particular relates to a method for lesion segmentation in three-dimensional CT images based on a self-attention feedback mechanism. Background Technology

[0002] Stomach cancer is the fifth most common cancer and the third leading cause of cancer-related death worldwide. Surgical treatment is the primary approach to treating stomach cancer, and preoperative examinations, including endoscopy and computed tomography (CT) scans, are crucial for surgical success. Endoscopy can clearly and directly reveal the extent of tumor lesions, but it is invasive to the stomach and limited to the detection of pathological findings. It cannot provide the comprehensive view of stomach lesions and the relationship between the tumor and surrounding tissues as a CT scan. Furthermore, CT imaging can provide more preoperative images to guide clinical information, with accurate segmentation of stomach cancer lesions being crucial for quantifying lesion distribution and guiding surgical success. However, due to the heterogeneity of stomach cancer cells and the irregular shape of the stomach, accurate segmentation of stomach cancer lesions presents significant challenges.

[0003] In recent years, with the success of deep learning in the field of medical imaging, many methods for medical image segmentation have been proposed and have achieved excellent results. However, these methods are either used for organ segmentation, liver tumor segmentation, or cell segmentation, and therefore cannot be well adapted to the task of segmenting lesions in 3D CT images of gastric cancer lesions. For the task of segmenting gastric cancer lesions in CT images, 3D improved feature pyramid networks and multi-attention guided multi-task networks have been proposed, but the gain in segmentation results is mediocre for both methods, and there is still much room for improvement in the task of segmenting gastric cancer lesions in CT images.

[0004] In summary, existing methods for segmenting lesions in 3D CT images of gastric cancer have the following problems: First, high complexity: some existing lesion segmentation methods may involve complex algorithms and processes, requiring a large amount of computing resources and time, which limits their practical application in clinical practice; Second, unclear boundaries: some gastric cancer lesions may have unclear boundaries in the images, which poses a challenge to the segmentation algorithm and may lead to the lesion boundaries being incorrectly located or segmented. Summary of the Invention

[0005] The purpose of this invention is to provide a three-dimensional CT image lesion segmentation method based on a self-attention feedback mechanism, so as to solve the problems of huge computational load, cumbersome steps, and poor and inaccurate segmentation accuracy in the prior art.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is a method for lesion segmentation in three-dimensional CT images based on a self-attention feedback mechanism, comprising the following steps:

[0007] S1. Acquire patient's three-dimensional CT scan image data;

[0008] S2. Mark the location of lesion areas in the image and perform data standardization and enhancement processing;

[0009] S3. Construct a self-attention module and determine voxel energy;

[0010] S4. Construct a self-attention feedback mechanism;

[0011] S5. Construct an encoder-decoder deep learning network model;

[0012] S6. Train the deep learning network model using five-fold cross-validation;

[0013] S7. Use the trained model to segment lesions in CT images.

[0014] Furthermore, the CT scan image data in S1 includes both enhanced CT scans and plain CT scans, with a slice thickness requirement of less than 2 mm and an intra-slice resolution requirement of 1.5 × 1.5 mm. 2 The following data sample contains at least 50 cases.

[0015] Furthermore, the specific steps of S2 are as follows: Using open-source tools, the lesion region is marked layer by layer in each scan data, and three-dimensional correction is performed in the transverse, coronal, and sagittal planes. The marking results are then cross-validated. The CT data of the lesion region are sorted in ascending order, and the 0.5% value is taken as the lower bound (lower), and the 99.5% value as the upper bound (upper). The entire CT data is cropped to the upper and lower bound regions for standardization. The cropping method is as follows:

[0016]

[0017] In the above formula, values ​​greater than upper in each CT data point are changed to upper, and values ​​less than lower are changed to lower.

[0018] The marking and three-dimensional correction of the lesion area requires at least two imaging professionals.

[0019] Furthermore, the specific steps of S3 are as follows:

[0020] Constructing a self-attention module:

[0021]

[0022] Among them, F in For the model feature map, Linear represents linear variation, and F... in Transformed into three features of the same size: query matrix Q, key matrix K, and value matrix V, QKT Let F be the self-attention weight matrix. in The correlation between each voxel, s is the scaling factor, and the softmax function projects the value to the interval 0 to 1; F out This is the output of the self-attention module;

[0023] The method for calculating voxel energy is as follows:

[0024]

[0025] Where n is F in The number of voxels, SaW[i,j] represents the correlation between voxels i and j, V[i] represents the energy of voxel i, and Proj(V) represents the projection of V onto F. in In a space of the same size, the voxel energy matrix VE is finally obtained.

[0026] Furthermore, the specific steps of S4 are as follows:

[0027] The self-attention feedback method is as follows:

[0028]

[0029] Where ReLU is the activation function; This is the feature map of the i-th layer of the network; Input CT images into the network and perform 1 / 2 i Scaling by a factor of 1; For voxel energy, and perform 8 / 2 i Multi-fold scaling; CNR is a module group for convolution, normalization, and activation functions; The i-th layer network feature map is used to introduce self-attention feedback information.

[0030] Furthermore, the specific steps of S5 are as follows:

[0031] The deep learning network model consists of two modules: an encoder and a decoder. A six-layer cascaded convolutional network is used as the encoder, and a self-attention module is added to the deep encoder network to learn self-attention information. The encoder's feature extraction process based on the input CT is as follows:

[0032]

[0033] Among them, CT input The model takes the received CT images as input, and E0 and F0 are the encoder and corresponding auxiliary outputs of the shallowest layer network. i For the i-th cascaded encoder, F i For feature map F i-1 After encoder E iThe characteristic output;

[0034] The decoder uses a cascaded pure convolutional network to decode the features learned from each level of the encoder, ultimately obtaining the lesion region segmentation result from the 3D CT image:

[0035]

[0036] Among them, D i Let S be the cascaded decoder of the i-th layer, D0 be the decoder of the shallowest layer network, and Seg be the decoder of the i-th layer network. out For network models of CT input Results of lesion region segmentation.

[0037] The self-attention module is included in the E4 and E5 cascade encoders.

[0038] Furthermore, the specific steps of S6 are as follows:

[0039] The standardized data and corresponding labeled data are divided into five equal parts. Four parts are used as the training set each time, and the remaining part is used as the test set. During network model training, Dice loss and cross-entropy loss are used to optimize the model parameters.

[0040]

[0041] Among them, L dice For the Dice loss calculation process, L CE For the calculation process of cross-entropy loss, p i ∈[0,1] represents the model output, indicating the probability that the i-th voxel is a lesion region; q i ∈{0,1} represents labeled data, where 0 indicates that the i-th voxel is not a lesion and 1 indicates that the i-th voxel is a lesion.

[0042] Furthermore, the specific steps of S7 are as follows:

[0043] The scanned CT image data, after being standardized using the lower and upper values ​​of S2, is directly input into the five network models trained by S6. The models output five segmentation probability maps. The average of these five probability maps is taken to obtain the final probability map. The following process is then used to transform the probability map into a segmentation result:

[0044]

[0045] In the above formula, P out The average segmentation result of the five trained network models is used. This formula treats voxels with a probability greater than 0.5 as lesions and voxels with a probability less than or equal to 0.5 as non-lesion areas.

[0046] The beneficial effects of this invention are: it establishes a three-dimensional CT image segmentation model for gastric cancer lesions, which can accurately segment the areas where lesions exist in CT images of gastric cancer patients; this is of great significance for quantifying the distribution of gastric cancer lesions and guiding successful surgery. Furthermore, compared with existing technologies, the method of this invention is simpler in steps, easier to implement, simpler to operate, and has better results. It can effectively solve the problems of low accuracy and inaccuracy in existing gastric cancer lesion segmentation methods by utilizing the established gastric cancer lesion segmentation model. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of a three-dimensional CT image gastric cancer lesion segmentation method based on a self-attention feedback mechanism;

[0049] Figure 2 This is a schematic diagram of a self-attention module;

[0050] Figure 3 This is a schematic diagram of a self-attention feedback mechanism;

[0051] Figure 4 This is a diagram of an encoder-decoder network model. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] like Figure 1 This invention discloses a method for lesion segmentation in three-dimensional CT images based on a self-attention feedback mechanism; specifically, it includes the following steps:

[0054] S1. Collect 3D CT scan image data of gastric cancer patients; collect 3D CT scan data of pathologically diagnosed gastric cancer patients from the hospital's imaging database. CT scan data includes enhanced CT scans and plain CT scans. The slice thickness should be less than 2mm, and the intra-slice resolution should be 1.5×1.5mm. 2 The following data sample size is at least 50 cases.

[0055] S2. The location of gastric cancer lesions in the images was labeled, and the data was standardized and enhanced. Open-source tools were used to label the gastric cancer region layer by layer for each scan, and 3D corrections were performed in the transverse, coronal, and sagittal planes. The labeling results were then cross-validated to ensure data accuracy. To further ensure accuracy, at least two radiologists were required to label and perform 3D corrections on the gastric cancer region. The CT values ​​of the gastric cancer region were sorted in ascending order, and the 0.5% value was taken as the lower bound, and the 99.5% value as the upper bound. The entire CT data was cropped to the upper and lower bound regions for standardization. The cropping method is as follows:

[0056]

[0057] In the above formula, values ​​greater than upper in each CT data point are changed to upper, and values ​​less than lower are changed to lower, thus standardizing the CT data.

[0058] S3. Construct a self-attention module:

[0059] like Figure 2 The self-attention module takes a network feature map as input and outputs a feature map with self-attention information. On one hand, the self-attention module enhances the model's global feature learning ability; on the other hand, it calculates voxel energy. Higher energy indicates a greater probability that the region is a gastric cancer lesion, allowing the model to efficiently learn lesion information. The calculation method for the attention module is as follows:

[0060]

[0061] Among them, F in For the model feature map, Linear represents linear variation, and F... in Transformed into three features of the same size: query matrix Q, key matrix K, and value matrix V, QK T This represents the self-attention weight matrix, recording f in The correlation between each voxel is calculated, s is the scaling factor, and the softmax function projects the values ​​to the interval 0 to 1; finally, the output of the self-attention module is F. out This represents a feature map that incorporates self-attention information.

[0062] The method for calculating voxel energy is as follows:

[0063]

[0064] Where n is F in Number of middle voxels, SaW = QK TSaW[i,j] represents the correlation between the i-th voxel and the j-th voxel, V[i] is the column summation of the i-th voxel in SaW, representing the energy of the i-th voxel, and Proj projects V onto F. in In a space of the same size, the voxel energy matrix VE is finally obtained.

[0065] S4. Construct a self-attention feedback mechanism:

[0066] like Figure 3 The self-attention feedback mechanism can feed back deep network self-attention information (voxel energy) to shallow layers, enabling the entire network to efficiently focus on the gastric cancer region and complete the segmentation task. The self-attention feedback method is as follows:

[0067]

[0068] In the above formula, ReLU is the activation function used to increase the nonlinearity of the network model; This is the feature map of the i-th layer of the network; Input CT images into the network and perform 1 / 2 i Scaling by a factor of 1; For voxel energy, and perform 8 / 2 i Magnification; CNR is a module group consisting of "Convolution + Norm + ReLU"; This is the feature map of the i-th layer of the network that incorporates self-attention feedback information, and... Same size.

[0069] S5. Construct an encoder-decoder deep learning network model:

[0070] like Figure 4 The system consists of two modules: an encoder and a decoder. A six-layer cascaded convolutional network is used as the encoder, and a self-attention module is added to the deep encoder network to learn self-attention information. The encoder extracts features from the input CT image as follows:

[0071]

[0072] In the above formula, CT input The model takes the received CT images as input, and E0 and F0 are the encoder and corresponding special output of the shallowest layer network. i For the i-th cascaded encoder, F i For feature map F i-1 After encoder E iThe feature output is obtained. The self-attention module is included in E4 and E5. The decoder uses a cascaded pure convolutional network to decode the features learned from each level of the encoder, ultimately obtaining the segmentation result of the gastric cancer lesion region in the 3D CT image. The decoder calculation process is as follows:

[0073]

[0074] Among them, D i Let S be the cascaded decoder of the i-th layer, D0 be the decoder of the shallowest layer network, and Seg be the decoder of the i-th layer network. out For network models of CT input Segmentation results of gastric cancer lesions.

[0075] S6. Train the network using five-fold cross-validation:

[0076] Five-fold cross-validation is used to train the network model. Five-fold cross-validation divides the standardized data and corresponding labeled data into five equal parts. Each time, four parts are used as the training set, and the remaining part is used as the test set. This method can train five network models and cover all the sample data.

[0077] During network model training, Dice loss and cross-entropy loss are used to optimize the model parameters. The calculation formulas are as follows:

[0078]

[0079] In the above formula, L dice For the Dice loss calculation process, L CE This describes the cross-entropy loss calculation process. n represents the total number of elements in the input CT image data, and p... i ∈[0,1] represents the model output, indicating the probability that the i-th voxel is a gastric cancer lesion region. q i ∈{0,1} represents labeled data, where 0 indicates that the i-th voxel is not a gastric cancer lesion, and 1 indicates that the i-th voxel is a gastric cancer lesion.

[0080] S7. Use the trained model to segment lesions in CT images of gastric cancer patients:

[0081] The scanned CT images of gastric cancer, after being standardized using the lower and upper values ​​of S2, are directly input into five trained network models. The models output five segmentation probability maps. The average of these five probability maps is taken to obtain the final probability map. The following process is then used to convert the probability map into segmentation results:

[0082]

[0083] In the above formula, P outThe average segmentation results of the five trained network models are used to determine the gastric cancer lesions. Voxels with a probability greater than 0.5 are considered as gastric cancer lesions, and voxels with a probability less than or equal to 0.5 are considered as non-lesion areas.

[0084] Example 1

[0085] The model results are evaluated using the dice score, recall, precision, and average surface distance (ASD). Recall primarily measures false negatives, precision primarily measures false positives, and the dice score comprehensively measures the model's segmentation performance by considering true positives, false positives, and false negatives. To calculate dice, recall, and precision, the true positive (TP), false positive (FP), and false negative (FN) metrics of the model's segmentation results must first be calculated. The calculation method is as follows:

[0086]

[0087] Where p i ,q i ∈{0,1}, where 0 represents a negative voxel and 1 represents a positive voxel for gastric cancer. p i q is the i-th voxel result predicted by the model. i The annotation results drawn by the image workers in step 2 are as follows. The calculation methods for dice, recall, and precision are as follows:

[0088]

[0089] The formula for calculating ASD is as follows:

[0090]

[0091] Where n is the number of positive voxels in the model output, m is the number of positive voxels in the annotation results, and p i Let q represent the i-th voxel of the model result. j d(p) represents the j-th voxel in the annotation result. i ,q) represents point p in the model output. i The distance between d(p,q) and the nearest point q in the labeled results. j ) represents point q in the annotation result. j The distance to the nearest point p in the model output. The calculation of ASD involves finding the nearest point to each point in the model, calculating the distances between these nearest points, and finally taking the average.

[0092] Table 1 Comparison of segmentation results of the present invention and other methods on gastric cancer CT images.

[0093] U-Net 0.7550 0.7356 0.8307 6.307 UNet++ 0.7850 0.7621 0.8648 4.303 Attention U-Net 0.6070 0.5988 0.6876 17.58 Swin UNETR 0.7346 0.7096 0.8437 6.738 nnU-Net 0.8294 0.8195 0.8658 4.013 MA-MTLN 0.7643 0.7580 0.8562 6.804 This invention 0.8456 0.8339 0.8836 3.208

[0094] As can be seen from Table 1, the method of the present invention is optimal in all four evaluation indicators of gastric cancer CT image lesion segmentation: dice, recall, precision, and average surface distance (ASD).

[0095] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0096] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for lesion segmentation in three-dimensional CT images based on a self-attention feedback mechanism, characterized in that, Includes the following steps: S1. Acquire patient's three-dimensional CT scan image data; S2. Mark the location of lesion areas in the image and perform data standardization and enhancement processing; S3. Construct a self-attention module and determine voxel energy; S4. Construct a self-attention feedback mechanism; S5. Construct an encoder-decoder deep learning network model; S6. Train the deep learning network model using five-fold cross-validation; S7. Use the trained model to segment lesions in CT images; The method for calculating voxel energy is as follows: in, for Number of mid-voxels For model feature maps, Indicates the first Individual elements and the first The degree of correlation between individual elements Indicates the first The amount of energy possessed by an individual element Indicates will Projected onto In a space of the same size, the voxel energy matrix is ​​finally obtained. ; The specific steps of S4 are as follows: The self-attention feedback method is as follows: in For activation functions; For the first Layer network feature map; Input CT images into the network and perform... Scaling by a factor of 1; For voxel energy, and to perform Scaling by a factor of 1; It consists of modules for convolution, normalization, and activation functions; To introduce self-attention feedback information Layer network feature map; The specific steps of S5 are as follows: The deep learning network model consists of two modules: an encoder and a decoder. A six-layer cascaded convolutional network is used as the encoder, and a self-attention module is added to the deep encoder network to learn self-attention information. The encoder's feature extraction process based on the input CT is as follows: in, The CT images received by the model are used as input. This represents the encoder and corresponding feature output of the shallowest layer network. For the first Cascade encoder, For feature map After encoder The characteristic output; The decoder uses a cascaded pure convolutional network to decode the features learned from each level of the encoder, ultimately obtaining the lesion region segmentation result from the 3D CT image: in, For the first Cascaded decoder The decoder for the shallowest layer network. For network model pairs Results of lesion region segmentation.

2. The method for lesion segmentation in three-dimensional CT images based on a self-attention feedback mechanism according to claim 1, characterized in that, The CT scan image data in the S1 includes CT enhanced scan and CT plain scan data, the data layer thickness requirement is below 2mm, and the layer resolution requirement is 1.5*1.5mm 2 The following; Data samples at least 50 cases.

3. The method for lesion segmentation in three-dimensional CT images based on a self-attention feedback mechanism according to claim 1, characterized in that, The specific steps of S2 are as follows: Using open-source tools, the lesion region is marked layer by layer in each scan data, and three-dimensional correction is performed in the transverse, coronal, and sagittal planes. The marking results are then cross-validated. The CT data of the lesion region are sorted in ascending order, and the 0.5% value is taken as the lower bound (lower), and the 99.5% value as the upper bound (upper). The entire CT data is then processed. Standardize the cropping to the upper and lower bounds. The cropping method is as follows: In the above formula, each CT data Li is greater than The value becomes less than The value becomes lower.

4. The method for lesion segmentation in three-dimensional CT images based on a self-attention feedback mechanism according to claim 1, characterized in that, The specific steps of S3 are as follows: Constructing a self-attention module: in, For model feature maps, For linear change, Transformed into three features of the same size: query matrix Key matrix Sum matrix , Here is the self-attention weight matrix, recording... The correlation between each voxel in the data. This is the scaling factor. The function projects values ​​onto the interval 0 to 1; This is the output of the self-attention module.

5. The method for lesion segmentation in three-dimensional CT images based on a self-attention feedback mechanism according to claim 1, characterized in that, The self-attention module includes... In cascade encoders.

6. The method for lesion segmentation in three-dimensional CT images based on a self-attention feedback mechanism according to claim 1, characterized in that, The specific steps of S6 are as follows: The standardized data and corresponding labeled data are divided into five equal parts. Four parts are used as the training set each time, and the remaining part is used as the test set. This is used during the network model training process. The loss and cross-entropy loss are used to optimize the model parameters: in, for Loss calculation process, This is the process of calculating cross-entropy loss. For model output, representing the first... The probability that an individual element represents a lesion area; For labeled data, 0 indicates the first... Individual elements are not lesions, 1 indicates the first Individual factors are the lesions.

7. The method for lesion segmentation in three-dimensional CT images based on a self-attention feedback mechanism according to claim 1, characterized in that, The specific steps of S7 are as follows: The scanned CT image data, after being standardized using the lower and upper values ​​of S2, is directly input into the five network models trained by S6. The models output five segmentation probability maps. The average of these five probability maps is taken to obtain the final probability map. The following process is then used to transform the probability map into a segmentation result: In the above formula, The average segmentation result of the five trained network models is used. This formula treats voxels with a probability greater than 0.5 as lesions and voxels with a probability less than or equal to 0.5 as non-lesion areas.

Images