A three-dimensional image segmentation method, device and electronic device based on dynamic threshold
The dynamic threshold prediction model generates suitable thresholds for different regions of the three-dimensional image, which solves the problem of segmentation accuracy fluctuations caused by fixed threshold selection in three-dimensional image segmentation, and improves the accuracy and stability of target segmentation.
Patent Information
- Application Number
- CN202410924049.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-07-11
AI Technical Summary
In three-dimensional image segmentation, the selection of fixed thresholds leads to fluctuations in segmentation accuracy, especially the inconsistent optimal thresholds for different images or different regions of the same image, affecting the final target segmentation effect.
Using a dynamic threshold-based method, the basic target segmentation network model and dynamic threshold prediction model are used to predict dynamic thresholds for different regions in the three-dimensional image, generate thresholds suitable for the current region, and final image division is performed based on the predicted probability value.
It improves the accuracy of target segmentation in three-dimensional images, reduces pixel missegment, and enhances the accuracy of image segmentation, especially in the absence of real segmentation result labels, which can assist model training and inference segmentation.
Smart Images

Figure CN118470057B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a three-dimensional image segmentation method, device and electronic equipment based on dynamic threshold. Background Art
[0002] Image segmentation technology is one of the core tasks in the field of computer vision and image processing. It refers to the process of dividing all pixels in an image into several regions with similar semantics. These regions are non-overlapping. Image processing purposes are achieved through region division, such as segmenting defective areas from normal areas in industrial product defect detection images to distinguish them.
[0003] In recent years, image segmentation technology based on deep learning has achieved good performance. Deep learning technology adopts the paradigm of "input-deep learning network-output". The image to be segmented is used as input, and a deep learning network with learning ability, such as high-order function fitting ability, outputs the expected segmentation result (i.e., mask). This process is called the inference process. At the same time, in order for the deep learning network to output the correct expected segmentation result, the network needs to go through a training process. In this process, the user provides a real segmentation result that matches the input image, and the predicted segmentation result output by the deep learning network is supervised by the real segmentation result. The learning ability of the deep learning network is corrected by the back propagation method based on gradient descent. This process is called the training process. The above training process and inference process constitute the learning stage and inference application stage of the deep learning network. Regardless of the training process or the inference process, the predicted segmentation result output by the network is the probability value of each pixel being divided into a certain category, and the pixel is finally predicted to be a certain semantic category. A threshold needs to be provided for the predicted probability value. The predicted probability value of the pixel is combined with the provided threshold to obtain the predicted semantic category of the pixel.
[0004] The segmentation of a three-dimensional image refers to dividing each pixel in the image into target area / non-target area, represented by 1 / 0. The goal of the task is to predict each pixel in the image as 0 or 1, and finally obtain the target of the image to assist industrial production in making judgments. During the entire segmentation process, the selection of thresholds for different images or even different areas in the same image often leads to fluctuations in segmentation accuracy, thereby affecting the effect of the final target segmentation. Summary of the invention
[0005] The purpose of the present invention is to provide a three-dimensional image segmentation method, device and electronic device based on dynamic threshold. Different thresholds are predicted for different areas in the three-dimensional image through a basic target segmentation network model and a dynamic threshold prediction model. Finally, the predicted probability values of different areas will be combined with the dynamic threshold to divide the final image, forming a new three-dimensional image processing solution.
[0006] To achieve the above object, the present invention proposes the following technical solutions:
[0007] In the first aspect, a three-dimensional image segmentation method based on dynamic threshold is proposed, comprising:
[0008] Obtain three-dimensional images of several known entities, determine training data and standard data, and construct a training data set;
[0009] According to the training data set, a basic target segmentation network model is trained; wherein the basic target segmentation network model is a convolutional neural network model, which is used to perform pixel-by-pixel segmentation prediction probability determination on the input training data to obtain a pixel-by-pixel segmentation prediction probability map of the three-dimensional image;
[0010] According to the standard data of the training data set, a standard segmentation prediction probability map is obtained based on the basic target segmentation network model and sub-region division is performed;
[0011] Determine the target prediction segmentation result of each sub-region under each preset threshold value, and determine the threshold coding of each sub-region according to the target prediction segmentation result and the actual segmentation result of the standard data, and generate a threshold coding standard with sub-region as the unit;
[0012] According to the training data, the segmentation prediction probability map output by the basic target segmentation network model according to any of the training data and the corresponding threshold coding standard, a dynamic threshold prediction model is trained; wherein the network structure of the dynamic threshold prediction model consists of a three-layer encoder and a four-layer decoder;
[0013] A three-dimensional image to be segmented is obtained, and the basic target segmentation network model and the dynamic threshold prediction model are used to perform inference segmentation.
[0014] Furthermore, the process of generating the threshold coding standard in sub-region units is as follows:
[0015] For each of the sub-regions of the standard segmentation prediction probability map, respectively calculating the prediction probability value of each pixel in the sub-region under a preset threshold based on a plurality of set thresholds, and obtaining a target prediction segmentation result of the sub-region under each preset threshold;
[0016] For any of the sub-regions, the IoU value of the sub-region under each preset threshold is calculated according to the target predicted segmentation result under each preset threshold and the actual segmentation result of the standard data;
[0017] Compare the IoU values of the sub-regions under each preset threshold, and determine the threshold code corresponding to the preset threshold corresponding to the maximum IoU value as the threshold code of the sub-region;
[0018] According to the threshold coding of each sub-region, a threshold coding standard with the sub-region as a unit is generated.
[0019] Furthermore, the process of using the basic target segmentation network model and the dynamic threshold prediction model to perform reasoning segmentation is:
[0020] Acquire a three-dimensional image to be segmented, and use the basic target segmentation network model to obtain a pixel-by-pixel segmentation prediction probability map of the three-dimensional image to be segmented;
[0021] According to the three-dimensional image to be segmented and the corresponding segmentation prediction probability map, the dynamic threshold prediction model is used to generate a corresponding threshold coding prediction map in units of sub-regions, and a pixel-by-pixel prediction probability threshold of the three-dimensional image to be segmented is determined;
[0022] The pixel-by-pixel prediction probability threshold of the three-dimensional image to be segmented is applied to the pixel-by-pixel segmentation prediction probability map of the three-dimensional image to be segmented to perform inference segmentation, so as to generate a target segmentation result of the three-dimensional image.
[0023] Furthermore, the basic object segmentation network model includes a four-layer encoder, a four-layer decoder and an output layer;
[0024] The four-layer structure of the encoder is a first three-dimensional convolution layer, a first group of normalization layers, a first ReLU activation function layer and a first pooling layer, wherein the convolution kernel size of the first three-dimensional convolution layer is 3×3×3, the padding value is set to 1, and the kernel size of the first pooling layer is 2×2×2, and the stride is 2;
[0025] The four-layer structure of the decoder is an upsampling layer, a second three-dimensional convolution layer, a second group of normalization layers, and a second ReLU activation function layer. The upsampling layer has a magnification of 2, and the convolution kernel size of the second three-dimensional convolution layer is 3×3×3, and the padding value is set to 1.
[0026] The output layer includes a third three-dimensional convolutional layer and a third group of normalization layers, and the convolution kernel size of the third three-dimensional convolutional layer is 1×1×1.
[0027] Furthermore, the training process of the basic target segmentation network model includes:
[0028] The parameters of the basic target segmentation network model are trained and optimized using the stochastic gradient descent method, and the parameters are set as follows: the number of batches for each training is 4, the learning rate is set to 0.0004, and random data expansion is set, including flipping and affine transformation;
[0029] The parameters of the basic target segmentation network model are trained using a cross entropy loss function and a back propagation method until the model converges.
[0030] Furthermore, the three-layer encoder of the dynamic threshold prediction model is composed as follows: the first layer includes: a fourth three-dimensional convolution layer, a fourth group of standardization layers, a third ReLU activation function layer, a first self-attention module, a fifth group of standardization layers, a fourth ReLU activation function layer, and a second pooling layer; the second layer includes: a fifth three-dimensional convolution layer, a sixth group of standardization layers, a fifth ReLU activation function layer, and a third pooling layer; the third layer includes: a sixth three-dimensional convolution layer, a seventh group of standardization layers, a sixth ReLU activation function layer, a second self-attention module, a seventh group of standardization layers, a seventh ReLU activation function layer, and a fourth pooling layer; wherein the convolution kernel size of each convolution layer of the encoder is 3×3×3, the padding value is set to 1, and the kernel size of each pooling layer is 2×2×2, and the step size is 2;
[0031] The four-layer decoder of the dynamic threshold prediction model is composed as follows: the first layer includes: the seventh three-dimensional convolution layer, the eighth group of standardization layers, the eighth ReLU activation function layer, the third self-attention module, the ninth group of standardization layers, the ninth ReLU activation function layer and the first bilinear interpolation upsampling layer; the second layer includes: the eighth three-dimensional convolution layer, the tenth group of standardization layers, the tenth ReLU activation function layer and the second bilinear interpolation upsampling layer; the third layer includes: the ninth three-dimensional convolution layer, the eleventh group of standardization layers, the eleventh ReLU activation function layer, the fourth self-attention module, the twelfth group of standardization layers, the twelfth ReLU activation function layer and the third bilinear interpolation upsampling layer; the fourth layer includes: a fully connected layer; wherein the magnification of the bilinear interpolation upsampling layer of the decoder is 2, the convolution kernel size of the convolution layer is 3×3×3, and the padding value is set to 1.
[0032] Furthermore, the training process of the dynamic threshold prediction model includes: using the stochastic gradient descent method to train and optimize the parameters of the dynamic threshold prediction model, and the parameters are set as follows: the number of batches for each training is 4, the learning rate is set to 0.0004, and data expansion is not set;
[0033] The parameters of the dynamic threshold prediction model are trained using a binary cross entropy loss function and a back propagation method until the model converges.
[0034] In a second aspect, a three-dimensional image segmentation device based on a dynamic threshold is proposed, comprising:
[0035] An acquisition and construction module is used to acquire three-dimensional images of several known entities, determine training data and standard data, and construct a training data set;
[0036] A first training module is used to train a basic target segmentation network model according to the training data set; wherein the basic target segmentation network model is a convolutional neural network model, which is used to perform pixel-by-pixel segmentation prediction probability determination on the input training data to obtain a pixel-by-pixel segmentation prediction probability map of the three-dimensional image;
[0037] A partitioning module, used to obtain a standard segmentation prediction probability map and perform sub-region partitioning based on the standard data of the training data set and the basic target segmentation network model;
[0038] A determination generation module is used to determine the target prediction segmentation result of each sub-region under each preset threshold value, and determine the threshold coding of each sub-region according to the target prediction segmentation result and the actual segmentation result of the standard data, and generate a threshold coding standard with sub-region as a unit;
[0039] A second training module is used to train a dynamic threshold prediction model according to the training data, the segmentation prediction probability map output by the basic target segmentation network model according to any of the training data, and the corresponding threshold coding standard; wherein the network structure of the dynamic threshold prediction model consists of a three-layer encoder and a four-layer decoder;
[0040] The segmentation module is used to obtain the three-dimensional image to be segmented, and use the basic target segmentation network model and the dynamic threshold prediction model to perform inference segmentation.
[0041] Furthermore, the determination generating module generates an execution unit of the threshold coding standard in units of sub-regions, including:
[0042] A first calculation unit is used to calculate the prediction probability value of each pixel in the sub-region under the preset threshold value for each sub-region of the standard segmentation prediction probability map based on the set multiple thresholds, so as to obtain the target prediction segmentation result of the sub-region under each preset threshold value;
[0043] A second calculation unit is used to calculate, for any of the sub-regions, an IoU value of the sub-region under each preset threshold value according to the target prediction segmentation result under each preset threshold value and the actual segmentation result of the standard data;
[0044] A comparing and determining unit, configured to compare the IoU values of the sub-regions under various preset thresholds, and determine a threshold code corresponding to the preset threshold corresponding to the maximum IoU value as the threshold code of the sub-region;
[0045] The generating unit is used to generate a threshold coding standard with sub-regions as units according to the threshold coding of each sub-region.
[0046] In the third aspect, an electronic device is proposed, comprising a computer program, wherein the computer program is stored in a computer-readable storage medium; when a processor of the electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of the above-mentioned dynamic threshold-based three-dimensional image segmentation method.
[0047] It can be seen from the above technical solutions that the technical solutions of the present invention have the following beneficial effects:
[0048] The present invention discloses a three-dimensional image segmentation method, device and electronic device based on dynamic threshold, and the method includes: obtaining a three-dimensional image of a known entity, determining training data and standard data, and constructing a training data set; training a basic target segmentation network model according to the training data set; using the basic target segmentation network model to obtain a standard segmentation prediction probability map and perform sub-region division according to the standard data of the training data set, and determining the target prediction segmentation results of each sub-region under each preset threshold; determining the threshold coding of each sub-region according to the real segmentation result of the standard data, and generating a threshold coding standard with sub-regions as units; training a dynamic threshold prediction model according to the training data, the segmentation prediction probability map and the threshold coding standard; and using the basic target segmentation network model and the dynamic threshold prediction model to perform reasoning segmentation on the three-dimensional image to be segmented. The present invention aims at the problem of partial pixel mis-segmentation when the three-dimensional image is segmented by fixed threshold segmentation, and proposes to use a dynamic threshold prediction model to attach different dynamic thresholds to different regions of the three-dimensional image, so that the generated threshold is more suitable for the current region, and finally realizes the accurate segmentation of the target in the image, so as to achieve the technical effect of improving the accuracy of target segmentation in the three-dimensional image.
[0049] At the same time, the scheme assists model training by learning the targets of existing real standard data, and then applies this scheme to inference segmentation of three-dimensional images without real segmentation result labels, helping staff to find target areas in the image.
[0050] It should be appreciated that all combinations of the foregoing concepts, as well as additional concepts described in greater detail below, may be considered to be part of the inventive subject matter of the present disclosure, provided such concepts are not mutually inconsistent.
[0051] The foregoing and other aspects, embodiments and features of the present invention can be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the present invention, such as the features and / or beneficial effects of the exemplary embodiments, will be apparent from the following description or learned from the practice of the specific embodiments according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] The accompanying drawings are not drawn to scale according to actual reference objects. In the accompanying drawings, each identical or nearly identical component shown in various figures may be represented by the same reference numeral. For the sake of clarity, not every component is labeled in each figure. Now, embodiments of various aspects of the present invention will be described by way of example and with reference to the accompanying drawings, in which:
[0053] Figure 1 A flow chart of a three-dimensional image segmentation method based on dynamic threshold value disclosed in the present invention;
[0054] Figure 2 A flow chart of a threshold coding standard based on sub-regions disclosed in the present invention;
[0055] Figure 3 A flowchart of performing reasoning segmentation based on a basic target segmentation network model and a dynamic threshold prediction model disclosed in the present invention;
[0056] Figure 4 A schematic diagram of an example of lesion segmentation disclosed in the embodiment;
[0057] Figure 5 It is a structural block diagram of a three-dimensional image segmentation device based on dynamic threshold value disclosed in the present invention;
[0058] Figure 6 A structural block diagram of the electronic device disclosed in this embodiment. DETAILED DESCRIPTION
[0059] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meaning understood by people with general skills in the field to which the present invention belongs.
[0060] The words "first", "second" and similar words used in the specification and claims of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular forms of "a", "an" or "the" and similar words do not indicate a quantitative limitation, but indicate the presence of at least one. The words "include" or "comprises" and similar words mean that the elements or objects appearing before "include" or "comprises" include the features, wholes, steps, operations, elements and / or components listed after "include" or "comprises", and do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.
[0061] When performing target segmentation based on current three-dimensional images, a fixed threshold is used for semantic prediction for all images, but the optimal threshold for different images or even different areas of the same image is often not a unified value, which leads to low segmentation accuracy of the target by the three-dimensional image; therefore, the present invention proposes a three-dimensional image segmentation method, device and electronic device based on dynamic threshold for three-dimensional images, performs dynamic threshold prediction on different areas in the three-dimensional image, obtains the optimal threshold selection for different areas, and finally improves the target segmentation accuracy.
[0062] Combination Figure 1 As shown, the three-dimensional image segmentation method based on dynamic threshold proposed by the present invention comprises the following steps:
[0063] Step S102, obtaining three-dimensional images of several known entities, determining training data and standard data, and constructing a training data set; wherein the training data includes three-dimensional images of known real target segmentation results and three-dimensional images of unknown real target segmentation results, and the standard data is a three-dimensional image of known real target segmentation results, and the two together constitute the training data set;
[0064] Step S104, training a basic target segmentation network model according to the training data set; wherein the basic target segmentation network model is a convolutional neural network model, which is used to perform pixel-by-pixel segmentation prediction probability determination on the input training data to obtain a pixel-by-pixel segmentation prediction probability map of the three-dimensional image; during training, a standard data may be used to form a validation set to verify the accuracy of the basic target segmentation network model, and then perform model optimization;
[0065] Step S106, according to the standard data of the training data set, based on the basic target segmentation network model, a standard segmentation prediction probability map is obtained and sub-region division is performed; for example, the size of the segmentation prediction probability map with pixels as the basic unit is 512×512×512×1, and the size of the sub-region is 32×32×32×1, then the size of the segmentation prediction probability map with sub-regions as the basic unit after the division conversion is 16×16×16;
[0066] Step S108, determining the target prediction segmentation results of each sub-region under each preset threshold, and determining the threshold coding of each sub-region according to the target prediction segmentation results and the actual segmentation results of the standard data, and generating a threshold coding standard with sub-regions as units;
[0067] The specific process of generating the threshold coding standard in sub-region units is as follows: Figure 2The method includes: step S1081, for each of the sub-regions of the standard segmentation prediction probability map, respectively calculating the prediction probability value of each pixel in the sub-region under the preset threshold value based on the set multiple threshold values, and obtaining the target prediction segmentation result of the sub-region under each preset threshold value; wherein the multiple preset threshold values are such as 0.2, 0.4, 0.6 and 0.8; step S1082, for any of the sub-regions, calculating the IoU (intersection over union) value of the sub-region under each preset threshold value according to its target prediction segmentation result under each preset threshold value and the true segmentation result of the standard data, that is, the pixel-by-pixel prediction segmentation result is the same as the pixel-by-pixel true segmentation result. The proportion of the pixels that are consistent with each other to the total number of pixels in the sub-region; step S1083, compare the IoU values of the sub-region under each preset threshold, and determine the threshold code corresponding to the preset threshold corresponding to the maximum IoU value as the threshold code of the sub-region; for example, the threshold code corresponding to the threshold 0.2 is 0, the threshold code corresponding to the threshold 0.4 is 1, the threshold code corresponding to the threshold 0.6 is 2, and the threshold code corresponding to the threshold 0.8 is 3; when the threshold is 0.6, the IoU value calculated by the current sub-region is the largest, and the threshold number of the sub-region is 2; when the threshold is 0.2, the IoU value calculated by the current sub-region is the largest, and the threshold number of the sub-region is 0. Step S1084, according to the threshold codes of each of the sub-regions, a threshold coding standard in units of sub-regions is generated. After all the sub-regions have been compared and calculated for their IoU values of all thresholds, the threshold coding corresponding to each sub-region is obtained, and the generation process of the threshold coding standard is completed. For a three-dimensional image with a size of 512×512×512×1, the size of the threshold coding standard is 16×16×16; in addition, the size of the sub-region can be flexibly adjusted. After the sub-region size is adjusted, the size of the corresponding threshold coding standard is also flexibly adjusted. For example, if the sub-region size is 64×64×64, the size of the threshold coding standard is adjusted to 8×8×8 accordingly. This scheme is used for the threshold coding standard of three-dimensional image segmentation. When performing specific image segmentation, standard data with the same content as the image to be segmented should be selected to maintain correspondence.
[0068] Step S110, training a dynamic threshold prediction model according to the training data, the segmentation prediction probability map output by the basic target segmentation network model of any of the training data, and the corresponding threshold coding standard; wherein the network structure of the dynamic threshold prediction model consists of a three-layer encoder and a four-layer decoder; the dynamic threshold prediction model uses the three-dimensional image of the training data, the segmentation prediction probability map obtained in the above steps, and the threshold coding standard for model training.
[0069] Step S112, obtaining a three-dimensional image to be segmented, and performing inference segmentation using the basic target segmentation network model and the dynamic threshold prediction model.
[0070] Based on the above steps, the present invention proposes a dynamic threshold post-processing mechanism. For the three-dimensional image to be detected, the image is taken as input and sent to the basic target segmentation network model, and a segmentation prediction probability map is output; next, the segmentation prediction probability map and the three-dimensional image are taken as new inputs and sent to the dynamic threshold prediction model, and the predicted pixel-by-pixel probability threshold is output; finally, the predicted pixel-by-pixel probability threshold is applied to the segmentation prediction probability map, and finally the target segmentation result of the three-dimensional image is generated, that is, the image is divided into the target area and the non-target area. The accuracy of lesion segmentation is improved through this scheme.
[0071] Combination Figure 3 As shown, the process of performing inference segmentation using the basic target segmentation network model and the dynamic threshold prediction model in the above step S112 includes: step S1121, obtaining a three-dimensional image to be segmented, and using the basic target segmentation network model to obtain a pixel-by-pixel segmentation prediction probability map of the three-dimensional image to be segmented; step S1122, according to the three-dimensional image to be segmented and the corresponding segmentation prediction probability map, using the dynamic threshold prediction model to generate a corresponding threshold coding prediction map in units of sub-regions, and determining a pixel-by-pixel prediction probability threshold of the three-dimensional image to be segmented; step S1123, applying the pixel-by-pixel prediction probability threshold of the three-dimensional image to be segmented to the pixel-by-pixel segmentation prediction probability map of the three-dimensional image to be segmented to perform inference segmentation, and generate a target segmentation result of the three-dimensional image.
[0072] As an optional implementation, in the above step S104, the structure of the basic target segmentation network model includes a four-layer encoder, a four-layer decoder and an output layer; wherein the four-layer structure of the encoder is a first three-dimensional convolution layer, a first group of normalization layers, a first ReLU activation function layer and a first pooling layer, the convolution kernel size of the first three-dimensional convolution layer is 3×3×3, the padding value is set to 1, the kernel size of the first pooling layer is 2×2×2, and the step size is 2; the four-layer structure of the decoder is an upsampling layer, a second three-dimensional convolution layer, a second group of normalization layers and a second ReLU activation function layer, the magnification of the upsampling layer is 2, the convolution kernel size of the second three-dimensional convolution layer is 3×3×3, and the padding value is set to 1; in addition, in addition to the features obtained by stepwise upsampling, the decoder also fuses the features from the same layer of the encoder into the features obtained by the decoder through jump connections; the output layer includes a third three-dimensional convolution layer and a third group of normalization layers, the convolution kernel size of the third three-dimensional convolution layer is 1×1×1, and outputs a pixel-by-pixel segmentation prediction probability map.
[0073] The training process of the basic target segmentation network model includes: using the stochastic gradient descent method to train and optimize the parameters of the basic target segmentation network model, and the parameters are set as follows: the number of batches for each training is 4, the learning rate is set to 0.0004, and random data expansion is set, including flipping and affine transformation; the parameters of the basic target segmentation network model are trained using the cross entropy loss function and the back propagation method until the model converges. When the basic target segmentation network model is working, the number of image channels of the three-dimensional image to be detected is 1, the size of the input image is 512×512×512×1, and the size of the segmentation prediction probability map output by the final output layer is 512×512×512×1. The probability value of each pixel prediction is a value in the interval [0, 1]. The closer the value is to 0, the higher the probability that the pixel is a non-target area, and the closer the value is to 1, the higher the probability that the pixel is a target area.
[0074] As an optional implementation, in the above step S110, the three-layer encoder of the dynamic threshold prediction model is composed as follows: the first layer includes: a fourth three-dimensional convolution layer, a fourth group of standardization layers, a third ReLU activation function layer, a first self-attention module, a fifth group of standardization layers and a fourth ReLU activation function layer and a second pooling layer; the second layer includes: a fifth three-dimensional convolution layer, a sixth group of standardization layers, a fifth ReLU activation function layer and a third pooling layer; the third layer includes: a sixth three-dimensional convolution layer, a seventh group of standardization layers, a sixth ReLU activation function layer, a second self-attention module, a seventh group of standardization layers, a seventh ReLU activation function layer and a fourth pooling layer; wherein the convolution kernel size of each convolution layer of the encoder is 3×3×3, the padding value is set to 1, the kernel size of each pooling layer is 2×2×2, and the step size is 2; The four-layer decoder of the state threshold prediction model is composed as follows: the first layer includes: the seventh three-dimensional convolution layer, the eighth group of standardization layers, the eighth ReLU activation function layer, the third self-attention module, the ninth group of standardization layers, the ninth ReLU activation function layer and the first bilinear interpolation upsampling layer; the second layer includes: the eighth three-dimensional convolution layer, the tenth group of standardization layers, the tenth ReLU activation function layer and the second bilinear interpolation upsampling layer; the third layer includes: the ninth three-dimensional convolution layer, the eleventh group of standardization layers, the eleventh ReLU activation function layer, the fourth self-attention module, the twelfth group of standardization layers, the twelfth ReLU activation function layer and the third bilinear interpolation upsampling layer; the fourth layer includes: the fully connected layer; wherein the multiplier of the bilinear interpolation upsampling layer of the decoder is 2, the convolution kernel size of the convolution layer is 3×3×3, and the padding value is set to 1. In addition to the features obtained by stepwise bilinear interpolation upsampling, the decoder also fuses the features from the same layer of the encoder into the features obtained by the decoder through skip connections.
[0075] The training process of the dynamic threshold prediction model includes: using the stochastic gradient descent method to train and optimize the parameters of the dynamic threshold prediction model, and the parameters are set as follows: the number of batches for each training is 4, the learning rate is set to 0.0004, and no data expansion is set; using the binary cross entropy loss function and the back propagation method to train the parameters of the dynamic threshold prediction model until the model converges.
[0076] Based on the above design of the dynamic threshold prediction model, the process of obtaining the threshold coding prediction map in sub-region units in the above step S1122 can be composed of three parts: sub-region division operation, sub-region mapping operation and model prediction operation, and finally the dynamic threshold of each sub-region of the three-dimensional image to be detected is obtained. Specifically, the sub-region division operation includes dividing the input image to generate a sub-region image in sub-region units, and the content of the image itself remains unchanged; including dividing and converting the pixel-based segmentation prediction probability map 512×512×512×1 obtained by the basic target segmentation network model, dividing to obtain a sub-region segmentation prediction probability map 16×16×16 in sub-region units of 32×32×32×1, and the unit image to be detected is also divided in the same way; here, it is consistent with the threshold coding standard generation process, the size of the sub-region can be flexibly adjusted, and the size of the segmentation prediction probability map in sub-region units and the three-dimensional image to be detected are also adjusted accordingly. The sub-region mapping operation is used to map each sub-region to a high-dimensional one-dimensional vector, flatten all pixels inside the sub-region to generate a one-dimensional vector for prediction.
[0077] Furthermore, when using the dynamic threshold prediction model for prediction operations, the three-dimensional image to be detected with a size of 16×16×16×132768 and the segmentation prediction probability map with a size of 16×16×16×132768 are first fully connected and mapped in the data input layer, and the size of the mapped feature map is 16×16×16×16; then the two images are sub-region fused, that is, the feature vectors of the sub-regions at the same position are spliced, and finally a fused feature map with a size of 16×16×16×32 is generated. The fused feature map is input into the dynamic threshold prediction model for prediction to output a threshold encoding prediction map with a size of 16×16×16×1.
[0078] The following is a further detailed introduction to the three-dimensional image segmentation method based on dynamic threshold value disclosed in the present invention in conjunction with specific embodiments.
[0079] Aiming at the problem of partial pixel or area mis-segmentation caused by fixed thresholds in existing deep learning-based three-dimensional image tumor segmentation methods, a method for accurate segmentation of liver tumor lesions in three-dimensional CT images is proposed. The above method is applied to propose a dynamic threshold post-processing process, and different thresholds are predicted for different areas in the image. Finally, the predicted probability values of different areas will be combined with the dynamic threshold for final semantic division. In this embodiment, the target is a lesion tumor.
[0080] like Figure 4 As shown in the figure, the image to be detected is first taken as input and sent to the basic target segmentation network model, and a pixel-by-pixel segmentation prediction probability map is output. Each square in the map represents a pixel. Next, the segmentation prediction probability map and the image to be detected are taken as new inputs and sent to the dynamic threshold prediction model to output a predicted pixel-by-pixel threshold coding prediction map. The prediction map has the same size as the segmentation prediction probability map. Finally, the predicted pixel-by-pixel threshold coding prediction map is applied to the segmentation prediction probability map. For example, if the threshold of a certain pixel is 0.7 and the probability value predicted by the segmentation prediction probability map is 0.8, the pixel is counted as a pixel in the target area, and finally the lesion segmentation result of the image is generated, and the image to be detected is divided into tumor area and non-tumor area.
[0081] To address the problem of partial pixel mis-segmentation caused by fixed threshold segmentation of three-dimensional images, the present invention proposes to use a dynamic threshold prediction surface model to attach different dynamic thresholds to different areas of the image, obtain segmentation results under the dynamic threshold, and significantly improve the target segmentation accuracy of the three-dimensional image.
[0082] In an embodiment of the present application, an electronic device is also provided, including a computer program, the computer program being stored in a computer-readable storage medium; when a processor of the electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of the above-mentioned three-dimensional image segmentation method based on dynamic threshold. Taking an electronic device running on a computer as an example, Figure 6 As shown, the electronic device may include one or more (only one is shown in the figure) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, and a transmission device for communication functions. A person skilled in the art can understand that Figure 6 The structure shown is for illustration only and does not limit the structure of the above-mentioned electronic device.
[0083] The above program can be run in the processor, or it can be stored in the memory, that is, in the computer-readable medium, which includes permanent and non-permanent, removable and non-removable media. Information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include temporary computer-readable media, such as modulated data signals and carrier waves. These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in a block or multiple blocks may be implemented by different modules corresponding to different method steps.
[0084] In this embodiment, such a device or system is provided. The device can be called a three-dimensional image segmentation device based on dynamic threshold. Figure 5As shown, it includes: an acquisition construction module, which is used to acquire three-dimensional images of several known entities, determine training data and standard data, and construct a training data set; a first training module, which is used to train a basic target segmentation network model according to the training data set; wherein the basic target segmentation network model is a convolutional neural network model, which is used to perform pixel-by-pixel segmentation prediction probability judgment on the input training data to obtain a pixel-by-pixel segmentation prediction probability map of the three-dimensional image; a division module, which is used to obtain a standard segmentation prediction probability map based on the basic target segmentation network model according to the standard data of the training data set and perform sub-region division; a determination generation module, which is used to determine the probability of each sub-region within each preset threshold The target prediction segmentation result under the value, and the threshold coding of each sub-region is determined according to the target prediction segmentation result and the actual segmentation result of the standard data, and the threshold coding standard in units of sub-regions is generated; the second training module is used to train the dynamic threshold prediction model according to the training data, the segmentation prediction probability map output by the basic target segmentation network model of any training data and the corresponding threshold coding standard; wherein the network structure of the dynamic threshold prediction model consists of a three-layer encoder and a four-layer decoder; an acquisition segmentation module is used to acquire the three-dimensional image to be segmented, and use the basic target segmentation network model and the dynamic threshold prediction model for inference segmentation.
[0085] The steps of the device for implementing the three-dimensional image segmentation method based on dynamic threshold value disclosed in the above embodiment have been described and will not be repeated here.
[0086] For example, the determination generation module generates an execution unit for the threshold coding standard in units of sub-regions, including: a first calculation unit, used to calculate the prediction probability value of each pixel in the sub-region under the preset threshold for each sub-region of the standard segmentation prediction probability map based on the set multiple thresholds, and obtain the target prediction segmentation result of the sub-region under each preset threshold; a second calculation unit, used to calculate the IoU value of the sub-region under each preset threshold for any sub-region according to its target prediction segmentation result under each preset threshold and the actual segmentation result of the standard data; a comparison and determination unit, used to compare the IoU values of the sub-region under each preset threshold, and determine the threshold code corresponding to the preset threshold corresponding to the maximum IoU value as the threshold code of the sub-region; a generation unit, used to generate a threshold coding standard in units of sub-regions according to the threshold code of each sub-region.
[0087] For another example, the execution unit of the acquisition segmentation module that uses the basic target segmentation network model and the dynamic threshold prediction model to perform inference segmentation includes: an acquisition unit, used to acquire the three-dimensional image to be segmented, and use the basic target segmentation network model to obtain a pixel-by-pixel segmentation prediction probability map of the three-dimensional image to be segmented; a prediction generation unit, used to generate a corresponding threshold coding prediction map in units of sub-regions based on the three-dimensional image to be segmented and the corresponding segmentation prediction probability map, and determine the pixel-by-pixel prediction probability threshold of the three-dimensional image to be segmented; a segmentation generation unit, used to apply the pixel-by-pixel prediction probability threshold of the three-dimensional image to be segmented to the pixel-by-pixel segmentation prediction probability map of the three-dimensional image to be segmented for inference segmentation, and generate a target segmentation result of the three-dimensional image.
[0088] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. A person with ordinary knowledge in the technical field to which the present invention belongs may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.
Claims
1. A three-dimensional image segmentation method based on dynamic threshold, characterized in that: include: Obtain three-dimensional images of several known entities, determine training data and standard data, and construct a training data set; According to the training data set, a basic target segmentation network model is trained; wherein the basic target segmentation network model is a convolutional neural network model, which is used to perform pixel-by-pixel segmentation prediction probability determination on the input training data to obtain a pixel-by-pixel segmentation prediction probability map of the three-dimensional image; According to the standard data of the training data set, a standard segmentation prediction probability map is obtained based on the basic target segmentation network model and sub-region division is performed; Determine the target prediction segmentation result of each sub-region under each preset threshold value, and determine the threshold coding of each sub-region according to the target prediction segmentation result and the actual segmentation result of the standard data, and generate a threshold coding standard with sub-region as the unit; According to the training data, the segmentation prediction probability map output by the basic target segmentation network model according to any of the training data and the corresponding threshold coding standard, a dynamic threshold prediction model is trained; wherein the network structure of the dynamic threshold prediction model consists of a three-layer encoder and a four-layer decoder; Acquire a three-dimensional image to be segmented, and perform inference segmentation using the basic target segmentation network model and the dynamic threshold prediction model; The process of generating the threshold coding standard in sub-region units is as follows: For each of the sub-regions of the standard segmentation prediction probability map, respectively calculating the prediction probability value of each pixel in the sub-region under a preset threshold based on a plurality of set thresholds, and obtaining a target prediction segmentation result of the sub-region under each preset threshold; For any of the sub-regions, the IoU value of the sub-region under each preset threshold is calculated according to the target predicted segmentation result under each preset threshold and the actual segmentation result of the standard data; Compare the IoU values of the sub-regions under each preset threshold, and determine the threshold code corresponding to the preset threshold corresponding to the maximum IoU value as the threshold code of the sub-region; According to the threshold coding of each sub-region, a threshold coding standard with the sub-region as a unit is generated.
2. The three-dimensional image segmentation method based on dynamic threshold according to claim 1, characterized in that: The process of using the basic target segmentation network model and the dynamic threshold prediction model to perform reasoning segmentation is: Acquire a three-dimensional image to be segmented, and use the basic target segmentation network model to obtain a pixel-by-pixel segmentation prediction probability map of the three-dimensional image to be segmented; According to the three-dimensional image to be segmented and the corresponding segmentation prediction probability map, the dynamic threshold prediction model is used to generate a corresponding threshold coding prediction map in units of sub-regions, and a pixel-by-pixel prediction probability threshold of the three-dimensional image to be segmented is determined; The pixel-by-pixel prediction probability threshold of the three-dimensional image to be segmented is applied to the pixel-by-pixel segmentation prediction probability map of the three-dimensional image to be segmented to perform inference segmentation, so as to generate a target segmentation result of the three-dimensional image.
3. The three-dimensional image segmentation method based on dynamic threshold according to claim 1, characterized in that: The basic object segmentation network model includes a four-layer encoder, a four-layer decoder and an output layer; The four-layer structure of the encoder is a first three-dimensional convolution layer, a first group of normalization layers, a first ReLU activation function layer and a first pooling layer, wherein the convolution kernel size of the first three-dimensional convolution layer is 3×3×3, the padding value is set to 1, and the kernel size of the first pooling layer is 2×2×2, and the stride is 2; The four-layer structure of the decoder is an upsampling layer, a second three-dimensional convolution layer, a second group of normalization layers, and a second ReLU activation function layer. The upsampling layer has a magnification of 2, and the convolution kernel size of the second three-dimensional convolution layer is 3×3×3, and the padding value is set to 1. The output layer includes a third three-dimensional convolutional layer and a third group of normalization layers, and the convolution kernel size of the third three-dimensional convolutional layer is 1×1×1.
4. The three-dimensional image segmentation method based on dynamic threshold according to claim 3, characterized in that: The training process of the basic target segmentation network model includes: The parameters of the basic target segmentation network model are trained and optimized using the stochastic gradient descent method, and the parameters are set as follows: the number of batches for each training is 4, the learning rate is set to 0.0004, and random data expansion is set, including flipping and affine transformation; The parameters of the basic target segmentation network model are trained using a cross entropy loss function and a back propagation method until the model converges.
5. The three-dimensional image segmentation method based on dynamic threshold according to claim 1, characterized in that: The three-layer encoder of the dynamic threshold prediction model is composed as follows: The first layer includes: a fourth three-dimensional convolution layer, a fourth group of normalization layers, a third ReLU activation function layer, a first self-attention module, a fifth group of normalization layers, a fourth ReLU activation function layer, and a second pooling layer; The second layer includes: the fifth three-dimensional convolution layer, the sixth group of normalization layers, the fifth ReLU activation function layer and the third pooling layer; The third layer includes: a sixth three-dimensional convolution layer, a seventh group of normalization layers, a sixth ReLU activation function layer, a second self-attention module, a seventh group of normalization layers, a seventh ReLU activation function layer, and a fourth pooling layer; wherein the convolution kernel size of each convolution layer of the encoder is 3×3×3, the padding value is set to 1, and the kernel size of each pooling layer is 2×2×2, and the step size is 2; The four-layer decoder of the dynamic threshold prediction model is composed as follows: the first layer includes: the seventh three-dimensional convolution layer, the eighth group of standardization layers, the eighth ReLU activation function layer, the third self-attention module, the ninth group of standardization layers, the ninth ReLU activation function layer and the first bilinear interpolation upsampling layer; the second layer includes: the eighth three-dimensional convolution layer, the tenth group of standardization layers, the tenth ReLU activation function layer and the second bilinear interpolation upsampling layer; the third layer includes: the ninth three-dimensional convolution layer, the eleventh group of standardization layers, the eleventh ReLU activation function layer, the fourth self-attention module, the twelfth group of standardization layers, the twelfth ReLU activation function layer and the third bilinear interpolation upsampling layer; the fourth layer includes: a fully connected layer; wherein the magnification of the bilinear interpolation upsampling layer of the decoder is 2, the convolution kernel size of the convolution layer is 3×3×3, and the padding value is set to 1.
6. The three-dimensional image segmentation method based on dynamic threshold according to claim 5, characterized in that: The training process of the dynamic threshold prediction model includes: using the stochastic gradient descent method to train and optimize the parameters of the dynamic threshold prediction model, and the parameters are set as follows: the number of batches for each training is 4, the learning rate is set to 0.0004, and data expansion is not set; The parameters of the dynamic threshold prediction model are trained using a binary cross entropy loss function and back propagation until the model converges.
7. A three-dimensional image segmentation device based on dynamic threshold, characterized in that: include: An acquisition and construction module is used to acquire three-dimensional images of several known entities, determine training data and standard data, and construct a training data set; A first training module is used to train a basic target segmentation network model according to the training data set; wherein the basic target segmentation network model is a convolutional neural network model, which is used to perform pixel-by-pixel segmentation prediction probability determination on the input training data to obtain a pixel-by-pixel segmentation prediction probability map of the three-dimensional image; A partitioning module, used to obtain a standard segmentation prediction probability map and perform sub-region partitioning based on the standard data of the training data set and the basic target segmentation network model; A determination generation module is used to determine the target prediction segmentation result of each sub-region under each preset threshold value, and determine the threshold coding of each sub-region according to the target prediction segmentation result and the actual segmentation result of the standard data, and generate a threshold coding standard with sub-region as a unit; A second training module is used to train a dynamic threshold prediction model according to the training data, the segmentation prediction probability map output by the basic target segmentation network model according to any of the training data, and the corresponding threshold coding standard; wherein the network structure of the dynamic threshold prediction model consists of a three-layer encoder and a four-layer decoder; An acquisition segmentation module is used to acquire a three-dimensional image to be segmented, and perform reasoning segmentation using the basic target segmentation network model and the dynamic threshold prediction model; The determination generating module generates an execution unit of a threshold coding standard in units of sub-regions, including: A first calculation unit is used to calculate the prediction probability value of each pixel in the sub-region under the preset threshold value for each sub-region of the standard segmentation prediction probability map based on the set multiple thresholds, so as to obtain the target prediction segmentation result of the sub-region under each preset threshold value; A second calculation unit is used to calculate, for any of the sub-regions, an IoU value of the sub-region under each preset threshold value according to the target prediction segmentation result under each preset threshold value and the actual segmentation result of the standard data; A comparing and determining unit, configured to compare the IoU values of the sub-regions under various preset thresholds, and determine a threshold code corresponding to the preset threshold corresponding to the maximum IoU value as the threshold code of the sub-region; The generating unit is used to generate a threshold coding standard with sub-regions as units according to the threshold coding of each sub-region.
8. An electronic device, characterized in that: It includes a computer program, which is stored in a computer-readable storage medium; when a processor of an electronic device reads the computer program from the computer-readable storage medium, the processor executes the computer program, so that the electronic device performs the steps of the dynamic threshold-based three-dimensional image segmentation method according to any one of claims 1 to 6.
