Tumor image segmentation system and method based on dynamic threshold
By introducing dynamic threshold technology into the tumor image segmentation system, the missegment problem caused by fixed thresholds is solved, and higher segmentation accuracy and more suitable threshold selection are achieved.
Patent Information
- Application Number
- CN202311357729.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-18
- Publication Date
- 2025-04-18
AI Technical Summary
In the prior art, during the tumor segmentation process of three-dimensional CT images, the use of fixed thresholds leads to missegment of some pixels or regions, affecting the segmentation accuracy.
A tumor image segmentation system and method based on dynamic thresholds is proposed. Through the lesion segmentation module, threshold coding generation module and dynamic threshold prediction module, dynamic threshold values are generated more suitable for different regions to segment tumor lesions.
Through the use of dynamic thresholds, more suitable thresholds can be generated for different image areas, improve the accuracy of tumor lesions segmentation, and obtain a relatively high segmentation effect.
Smart Images
Figure CN119850943A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of data processing, and in particular to a tumor image segmentation system and method based on dynamic threshold. Background Art
[0002] Lesion segmentation of three-dimensional CT images refers to dividing each pixel of a three-dimensional CT image into tumor lesion area / non-tumor lesion area, represented by 1 / 0. The goal of the task is to predict each pixel in the image as 0 or 1, and finally obtain the lesion of the image to assist doctors in judging the patient's condition.
[0003] For the tumor segmentation problem of CT images, existing deep learning usually provides a certain threshold. During the entire segmentation process, this threshold is used for semantic prediction for all images. For example, if the output prediction value of a certain pixel after the network is 0.8, then the pixel is finally classified as a tumor pixel (1). If the output prediction value of a certain pixel after the network is 0.4, then the pixel is finally classified as a non-tumor pixel (0). However, the optimal threshold for different images, or even different areas of the same image, is often not a unified value. Having only one threshold will cause fluctuations in segmentation accuracy, thereby affecting the final lesion segmentation effect. Summary of the invention:
[0004] The present invention aims to solve the problem of mis-segmentation of some pixels or regions caused by using a fixed threshold for all images during the segmentation process.
[0005] The invention provides a tumor image segmentation system and method based on dynamic threshold.
[0006] A tumor image segmentation system based on dynamic threshold, comprising:
[0007] A lesion segmentation module, which is used to predict the input image, obtain a pixel-by-pixel prediction probability value, and output a lesion prediction probability map of the same size as the input image;
[0008] A threshold code generation module, wherein the threshold code generation module is used to divide the input lesion prediction probability map and the lesion segmentation label, compare and assign values to the slices of the lesion prediction probability map and the slices of the lesion segmentation label, and output the true value of the threshold code;
[0009] A dynamic threshold prediction module, the dynamic threshold prediction module includes a partitioning module, a slice embedding module and a dynamic threshold prediction network; the partitioning module is used to partition the lesion prediction probability map and the input image; the slice embedding module is used to flatten the pixels inside the slice to generate a one-dimensional vector; the input of the dynamic threshold prediction network is a feature map formed by the lesion prediction probability map after partitioning and slice embedding and the input image, and the output is a threshold-coded prediction map;
[0010] The dynamic threshold prediction module cooperates with the lesion segmentation module to generate the lesion segmentation results of the image.
[0011] Preferably, the lesion segmentation module includes an encoder and a decoder, the encoder consists of four layers, each of which is a three-dimensional convolution operation, a group normalization operation, a ReLU activation function and a pooling operation; the decoder consists of four layers, each of which is an upsampling operation, a three-dimensional convolution operation, a group normalization operation, a ReLU activation function and a 1×1×1 convolution layer.
[0012] Preferably, the dynamic threshold prediction network includes an encoder and a decoder, the encoder consists of three layers, namely three-dimensional convolution operation, group standardization operation, ReLU activation function operation, self-attention module operation, group standardization operation, ReLU activation function and pooling operation, and three-dimensional convolution operation, group standardization operation, ReLU activation function and pooling operation, and three-dimensional convolution operation, group standardization operation, ReLU activation function, self-attention module operation, group standardization operation, ReLU activation function and pooling operation; the decoder consists of four layers, namely three-dimensional convolution operation, group standardization operation, ReLU activation function, self-attention module operation, group standardization operation, ReLU activation function and pooling operation, and bilinear interpolation upsampling operation, three-dimensional convolution operation, group standardization operation and ReLU activation function, and three-dimensional convolution operation, group standardization operation, ReLU activation function, self-attention module operation, group standardization operation, ReLU activation function, pooling operation and fully connected layer.
[0013] Preferably, for the lesion segmentation module, its decoder fuses features from the same layer of its encoder into features obtained by its decoder through skip connections;
[0014] Alternatively, for the dynamic threshold prediction network, its decoder fuses the features from the same layer of its encoder into the features obtained by its decoder through skip connections.
[0015] A tumor image segmentation method based on dynamic threshold, comprising:
[0016] Step 1, input the image to be detected into the lesion segmentation network for training, and use the trained lesion segmentation network to output a lesion prediction probability map;
[0017] Step 2: divide and convert the lesion prediction probability map and the lesion segmentation label. For the slices of the lesion prediction probability map, preset multiple thresholds as the prediction probability values of the pixels of each slice, obtain the prediction segmentation result of the current slice under the corresponding threshold, calculate the IoU between the result and the lesion segmentation label of the corresponding slice, select a specific value as the threshold code of the current slice, and obtain the true value of the threshold code after all slices are calculated;
[0018] Step 3, the image to be detected is divided and converted, and the slice embedding operation is performed on the image to be detected and the lesion prediction probability map after the division and conversion, and then feature fusion is performed to obtain a fused feature map, and the fused feature map is input into a dynamic threshold prediction network for training, and the dynamic threshold prediction network is used to output a threshold coding prediction map;
[0019] Step 4: Use the trained lesion segmentation network and the trained dynamic threshold prediction network to perform inference segmentation on the image of the lesion area to generate a lesion segmentation result.
[0020] Preferably, in step 2, for the slice of the lesion prediction probability map, the method of presetting multiple thresholds is: select n thresholds, and take the thresholds as in Represents rounding down, where n is an integer greater than 1.
[0021] Preferably, in step 2, the predicted probability value of each pixel inside the slice is compared with the corresponding threshold. If the predicted probability value is greater than the threshold, the pixel is considered to be a tumor lesion area, and the predicted segmentation result is 1; otherwise, the pixel is considered to be a non-tumor lesion area, and the predicted segmentation result is 0.
[0022] Preferably, in step 2, the code of the threshold corresponding to the maximum IoU calculation result is taken as the threshold code of the current slice, and the threshold codes are numbered in ascending order according to the threshold.
[0023] Preferably, in step 1, the parameters of the lesion segmentation network are trained using the stochastic gradient descent method; in step 3, the parameters of the dynamic threshold prediction network are trained using the stochastic gradient descent method.
[0024] Preferably, in step 1, the parameters of the lesion segmentation network are trained using a cross entropy loss function and a back propagation method; in step 3, the parameters of the dynamic threshold prediction network are trained using a binary cross entropy loss function and a back propagation method.
[0025] Compared with the prior art, the present invention has the following advantages and effects:
[0026] The present invention can generate dynamic thresholds that are more suitable for the current area for different areas of different images, and in addition, train a lesion segmentation module to output the initial lesion prediction probability map of the detection image. Based on the above dynamic thresholds and the initial lesion prediction probability map, the final tumor lesion segmentation result is obtained, and a relatively high segmentation accuracy is obtained. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0028] Figure 1 It is a flowchart of the threshold coding generation module.
[0029] Figure 2 It is the training flow chart of the dynamic threshold prediction module.
[0030] Figure 3 It is the prediction flow chart of the dynamic threshold prediction network. DETAILED DESCRIPTION
[0031] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0032] Embodiment 1:
[0033] A tumor image segmentation system based on dynamic threshold, comprising:
[0034] A lesion segmentation module, which is composed of a lesion segmentation network and is used to obtain a segmentation prediction probability for each pixel of the image to be detected. All pixels in the image are predicted to obtain a probability value, and finally a lesion prediction probability map of the same size as the image to be detected is output;
[0035] A threshold code generation module, wherein the threshold code generation module is used to divide the input lesion prediction probability map and the lesion segmentation label, compare and assign values to the slices of the lesion prediction probability map and the slices of the lesion segmentation label, and output the true value of the threshold code;
[0036] The dynamic threshold prediction module includes a partitioning module, a slice embedding module and a dynamic threshold prediction network. The partitioning module is used to partition the input image and generate an image in slices, and the content of the image itself remains unchanged. The slice embedding module is used to map each slice to a high-dimensional one-dimensional vector, flatten all pixels inside the slice to generate a one-dimensional vector. The dynamic threshold prediction network is used to predict the feature map formed after partitioning and slice embedding, and output the threshold-coded prediction map.
[0037] In a specific embodiment, the lesion segmentation module includes an encoder and a decoder, the encoder is composed of four layers, each layer is a three-dimensional convolution operation, a group normalization operation, a ReLU activation function and a pooling operation; the decoder is composed of four layers, each layer is an upsampling operation, a three-dimensional convolution operation, a group normalization operation, a ReLU activation function and a 1×1×1 convolution layer. For example, after the data layer, there are first four modules consisting of "three-dimensional convolution-group normalization-ReLU activation function-pooling", where the convolution kernel size of the convolution is 3×3×3, the padding value is set to 1, the pooling kernel size is 2×2×2, and the step size is 2. Secondly, there are four modules consisting of "upsampling-three-dimensional convolution-group normalization-ReLU activation function", where the upsampling ratio is 2, the convolution kernel size of the convolution is 3×3×3, and the padding value is set to 1. Finally, there is a convolution module with a convolution kernel size of 1×1×1, which finally outputs a lesion prediction probability map. The last channel represents the lesion prediction probability value of the corresponding pixel. The probability value is a value in the interval [0,1]. The closer the value is to 0, the higher the probability that the pixel is a non-tumor lesion area. The closer the value is to 1, the higher the probability that the pixel is a tumor lesion area. In a specific embodiment, the parameters of the lesion segmentation network are trained and optimized using the stochastic gradient descent method, and the parameters are set as follows: the number of batches for each training is 4, and the learning rate is set to 0.0004. Set random data expansion, including flipping, affine transformation, etc. Use the cross entropy loss function and back propagation method to train the parameters of the basic lesion segmentation network until the model converges.
[0038] In a specific implementation, a threshold encoding true value in slice units is generated, such as Figure 1 As shown in FIG. 1 , a lesion prediction segmentation result map under a specific threshold is generated based on the set multiple thresholds. Figure 1-3 All slices in are 3D.
[0039] In a specific embodiment, the dynamic threshold prediction network, the network structure selected by the present invention is a network composed of three layers of encoders and four layers of decoders. Specifically, at the data layer, the two feature maps are fully connected and mapped respectively, and then the two feature maps are sliced and fused, that is, the feature vectors of the slices at the same position are spliced, and finally a fused feature map is generated. Next, the fused feature map is sent to the U-shaped network. The U-shaped network first has three encoder modules composed of "three-dimensional convolution-group normalization-ReLU activation function-self-attention module-group normalization-ReLU activation function-pooling", "three-dimensional convolution-group normalization-ReLU activation function-pooling", and "three-dimensional convolution-group normalization-ReLU activation function-self-attention module-group normalization-ReLU activation function-pooling", wherein the convolution kernel size of the convolution is 3×3×3, the padding value is set to 1, the pooling kernel size is 2×2×2, and the step size is 2. Secondly, there are four decoder modules consisting of "3D convolution-group normalization-ReLU activation function-self-attention module-group normalization-ReLU activation function-bilinear interpolation upsampling", "3D convolution-group normalization-ReLU activation function-bilinear interpolation upsampling", "3D convolution-group normalization-ReLU activation function-self-attention module-group normalization-ReLU activation function-bilinear interpolation upsampling", and "fully connected layer", where the upsampling factor is 2, the convolution kernel size is 3×3×3, and the padding value is set to 1. The final output is a threshold coding prediction map, and the last channel represents the threshold coding prediction value of the corresponding slice. In addition, in addition to the features obtained by stepwise upsampling, the decoder also fuses the features from the same layer of the encoder into the features obtained by the decoder through skip connections. In a specific embodiment, the dynamic threshold prediction network parameters are trained and optimized using the stochastic gradient descent method. The training parameters are set as follows: the number of batches for each training is 4, and the learning rate is set to 0.0004. No form of data augmentation is set, including cropping, flipping, etc. The parameters of the dynamic threshold prediction network are trained using the binary cross entropy loss function and back propagation until the model converges.
[0040] A tumor image segmentation method based on dynamic threshold includes the following steps:
[0041] Step 1, input the image to be detected into the lesion segmentation network for training, and use the trained lesion segmentation network to output a lesion prediction probability map;
[0042] Step 2: divide and convert the lesion prediction probability map and the lesion segmentation label. For the slices of the lesion prediction probability map, preset multiple thresholds as the prediction probability values of the pixels of each slice, obtain the prediction segmentation result of the current slice under the corresponding threshold, calculate the IoU between the result and the corresponding slice of the lesion segmentation label, select a specific value as the threshold code of the current slice, and obtain the true value of the threshold code after all slices are calculated;
[0043] Step 3, the image to be detected is divided and converted, and the slice embedding operation is performed on the image to be detected and the lesion prediction probability map after the division and conversion, and then feature fusion is performed to obtain a fused feature map, and the fused feature map is input into a dynamic threshold prediction network for training, and the dynamic threshold prediction network is used to output a threshold coding prediction map;
[0044] Step 4: Use the trained lesion segmentation network and the trained dynamic threshold prediction network to perform inference segmentation on the image of the lesion area to generate a lesion segmentation result.
[0045] In one specific embodiment:
[0046] Step 1: Input the image to be detected into the lesion segmentation network. The size of the image to be detected is 512×512×512×1. The lesion segmentation network predicts the image to be detected pixel by pixel. For each pixel position, the lesion segmentation network will output a segmentation prediction probability. The calculation process for obtaining this probability value is learned by the network when it is trained using existing methods. All pixels in the image are predicted to obtain a probability value, and finally a probability map of the same size as the image to be detected is output.
[0047] Step 2, divide and transform the lesion prediction probability map (512×512×512×1) with pixels as the basic unit and the lesion segmentation label obtained in step 1. After the division, the size of each slice is 32×32×32×1. Therefore, after the division and transformation, the size of the lesion prediction probability map with slices as the basic unit is 16×16×16, and the size of the slice with pixels as the basic unit is 32×32×32×1. Similarly, for the lesion segmentation label (the value is 0, which means that the pixel is a non-tumor lesion area. The value is 1, which means that the pixel is a tumor lesion area), the division and transformation are also performed in the same way as above. For each slice in the lesion prediction probability map with slices as the basic unit, select n thresholds, and take the thresholds as in Represents rounding down, and the codes corresponding to the subsequent thresholds are 0, 1, 2, ..., n-1. For example, if 4 thresholds are selected, the thresholds are 0.2, 0.4, 0.6, and 0.8 respectively. The threshold is applied to the predicted probability value of each pixel in the slice, and the predicted probability value of each pixel in the slice is compared with the corresponding threshold. If the predicted probability value is greater than the threshold, the pixel is considered to be a tumor lesion area, and the predicted segmentation result is 1; otherwise, the pixel is considered to be a non-tumor lesion area, and the predicted segmentation result is 0. The predicted segmentation result of the current slice under the corresponding threshold is obtained, and the IoU comparison calculation is performed on this result and the real segmentation label of the current slice. The real segmentation label is annotated according to the actual situation (for example, the segmentation label of the existing tumor image can be used). For 4 thresholds, 4 IoU values are obtained in any slice, and the maximum value is numbered as the threshold code of the current slice (for example, the preset threshold 0.2 corresponds to code 0, the threshold 0.4 corresponds to code 1, the threshold 0.6 corresponds to code 2, and the threshold 0.8 corresponds to code 3; that is, if the IoU of the current slice is the largest when the threshold is 0.6, the number of the slice is 2, and the IoU is the largest when the threshold is 0.2, then the number is 0). At this point, after all slices are compared and calculated with IoU of all thresholds, the threshold code corresponding to each slice is obtained, and the generation process of the true value of the threshold code is completed. For an image with a size of 512×512×512×1, the size of the true value of the threshold code is 16×16×16. In addition, the size of the slice can be flexibly adjusted. After the slice size is adjusted, the size of the corresponding threshold-coded true value is also flexibly adjusted. For example, if the slice size is 64×64×64, the size of the threshold-coded true value is correspondingly adjusted to 8×8×8.
[0048] Step 3, such as Figure 2 As shown, the lesion prediction probability map (512×512×512×1) with pixels as the basic unit obtained in step 1 is divided and transformed. After the division, the size of each slice is 32×32×32×1. Therefore, the size of the lesion prediction probability map with slices as the basic unit after the division and transformation is 16×16×16. Similarly, for the image to be detected, the division and transformation are also performed in the same way as above. The method here is consistent with that in step 2. The size of the slice can be flexibly adjusted. Similarly, the size of the lesion prediction probability map with slices as the basic unit and the size of the image to be detected are also adjusted accordingly. The slice embedding operation is used to map each slice to a high-dimensional one-dimensional vector, and flatten all the pixels inside the slice to generate a one-dimensional vector. ， For example, input slice by slice, the size of which is 32*32*32*1, and complete linear mapping of the pixels contained in it through convolution operation, and output a one-dimensional vector of 1*96. Figure 3As shown, the feature map formed after the image to be detected and the lesion prediction probability map are divided and sliced and embedded is input into the dynamic threshold prediction network, and the threshold-coded prediction map is output. The size of the threshold-coded prediction map obtained after the dynamic threshold prediction network is: 16×16×16×1.
[0049] Step 4: Use the trained lesion segmentation network and the trained dynamic threshold prediction network to perform inference segmentation on the image of the lesion area to generate a lesion segmentation result.
[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A tumor image segmentation system based on dynamic threshold, characterized in that: include: A lesion segmentation module, which is used to predict the input image, obtain a pixel-by-pixel prediction probability value, and output a lesion prediction probability map of the same size as the input image; A threshold code generation module, wherein the threshold code generation module is used to divide the input lesion prediction probability map and the lesion segmentation label, compare and assign values to the slices of the lesion prediction probability map and the slices of the lesion segmentation label, and output the true value of the threshold code; A dynamic threshold prediction module, the dynamic threshold prediction module includes a partitioning module, a slice embedding module and a dynamic threshold prediction network; the partitioning module is used to partition the lesion prediction probability map and the input image; the slice embedding module is used to flatten the pixels inside the slice to generate a one-dimensional vector; the input of the dynamic threshold prediction network is a feature map formed by the lesion prediction probability map after partitioning and slice embedding and the input image, and the output is a threshold-coded prediction map; The dynamic threshold prediction module cooperates with the lesion segmentation module to generate the lesion segmentation results of the image.
2. The tumor image segmentation system based on dynamic threshold according to claim 1, characterized in that: The lesion segmentation module includes an encoder and a decoder, the encoder consists of four layers, each layer is a three-dimensional convolution operation, a group normalization operation, a ReLU activation function and a pooling operation; The decoder consists of four layers, each of which is an upsampling operation, a three-dimensional convolution operation, a group normalization operation, a ReLU activation function, and a 1×1×1 convolution layer.
3. The tumor image segmentation system based on dynamic threshold according to claim 1, characterized in that: The dynamic threshold prediction network includes an encoder and a decoder. The encoder consists of three layers, namely three-dimensional convolution operation, group standardization operation, ReLU activation function operation, self-attention module operation, group standardization operation, ReLU activation function and pooling operation, and three-dimensional convolution operation, group standardization operation, ReLU activation function and pooling operation, and three-dimensional convolution operation, group standardization operation, ReLU activation function, self-attention module operation, group standardization operation, ReLU activation function and pooling operation; the decoder consists of four layers, namely three-dimensional convolution operation, group standardization operation, ReLU activation function, self-attention module operation, group standardization operation, ReLU activation function and pooling operation, and bilinear interpolation upsampling operation, three-dimensional convolution operation, group standardization operation and ReLU activation function, and three-dimensional convolution operation, group standardization operation, ReLU activation function, self-attention module operation, group standardization operation, ReLU activation function, pooling operation and fully connected layer.
4. The tumor image segmentation system based on dynamic threshold according to claim 2 or 3, characterized in that: For the lesion segmentation module, its decoder fuses the features from the same layer of its encoder into the features obtained by its decoder through skip connections; Alternatively, for the dynamic threshold prediction network, its decoder fuses the features from the same layer of its encoder into the features obtained by its decoder through skip connections.
5. A tumor image segmentation method based on dynamic threshold, characterized in that: include: Step 1, input the image to be detected into the lesion segmentation network for training, and use the trained lesion segmentation network to output a lesion prediction probability map; Step 2: divide and convert the lesion prediction probability map and the lesion segmentation label. For the slices of the lesion prediction probability map, preset multiple thresholds as the prediction probability values of the pixels of each slice, obtain the prediction segmentation result of the current slice under the corresponding threshold, calculate the IoU between the result and the lesion segmentation label of the corresponding slice, select a specific value as the threshold code of the current slice, and obtain the true value of the threshold code after all slices are calculated; Step 3, the image to be detected is divided and converted, and the slice embedding operation is performed on the image to be detected and the lesion prediction probability map after the division and conversion, and then feature fusion is performed to obtain a fused feature map, and the fused feature map is input into a dynamic threshold prediction network for training, and the dynamic threshold prediction network is used to output a threshold coding prediction map; Step 4: Use the trained lesion segmentation network and the trained dynamic threshold prediction network to perform inference segmentation on the image of the lesion area to generate a lesion segmentation result.
6. The tumor image segmentation method based on dynamic threshold according to claim 5, characterized in that: In step 2, for the slice of the lesion prediction probability map, the method of presetting multiple thresholds is: select n thresholds, and the thresholds are (i=1,2,3,…,n), where Represents rounding down, where n is an integer greater than 1.
7. The tumor image segmentation method based on dynamic threshold according to claim 5, characterized in that: In step 2, the predicted probability value of each pixel inside the slice is compared with the corresponding threshold. If the predicted probability value is greater than the threshold, the predicted segmentation result is 1; otherwise, the predicted segmentation result is 0.
8. The tumor image segmentation method based on dynamic threshold according to claim 5, characterized in that: In step 2, the code of the threshold corresponding to the maximum IoU calculation result is taken as the threshold code of the current slice, and the codes of the thresholds are numbered in ascending order according to the thresholds.
9. The tumor image segmentation method based on dynamic threshold according to claim 5, characterized in that: In step 1, the stochastic gradient descent method is used to train the lesion segmentation network parameters; in step 3, the stochastic gradient descent method is used to train the dynamic threshold prediction network parameters.
10. The tumor image segmentation method based on dynamic threshold according to claim 5, characterized in that: In the step 1, the parameters of the lesion segmentation network are trained using a cross entropy loss function and a back propagation method; in the step 3, the parameters of the dynamic threshold prediction network are trained using a binary cross entropy loss function and a back propagation method.