Medical image segmentation method, device and storage medium fusing structural prior knowledge

Abstract

The application relates to a medical image segmentation method and device fusing structural prior knowledge and a storage medium. The method uses a pre-trained Swin-Unet-based image segmentation model to segment an acquired medical image. The training process of the image segmentation model comprises the following steps: acquiring a labeled medical image dataset; based on the medical image dataset, using a loss function to train the Swin-Unet-based image segmentation model to obtain a pre-trained image segmentation model, wherein the loss function comprises a probability prior loss function. Compared with the prior art, the application has the advantages of improving the model learning effect and wide application scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image segmentation, in particular to a medical image segmentation method, device and storage medium fusing structure priori knowledge. BACKGROUND

[0002] Medical auxiliary systems are gradually becoming a reality, which can provide help for doctors to diagnose and treat by processing and analyzing medical images. In medical images, the distribution of organs of the human body has certain regularity, so the distribution of organs in medical images has certain characteristics, which are called structure priori information of medical images. The probability atlas is a form of medical image structure priori knowledge, and the topological structure information of organs in the image can be reflected through the probability atlas. The probability atlas is usually obtained by processing the standard medical images labeled by relevant medical experts. In order to better describe the structure information of the human body, these priori information can be fully utilized to guide the optimization direction of the network model in medical image segmentation, but many existing segmentation methods ignore these important priori information.

[0003] In summary, there is currently a lack of a medical image segmentation method combining priori information. SUMMARY

[0004] The present application is to overcome the defects of the prior art and provide a medical image segmentation method, device and storage medium fusing structure priori knowledge, which introduces priori probability in the image segmentation model training process to improve the learning effect.

[0005] The object of the present application can be achieved by the following technical solutions:

[0006] In one aspect of the present application, a medical image segmentation method fusing structure priori knowledge is provided, which uses a pre-trained image segmentation model based on Swin-Unet to segment the obtained medical image. The training process of the image segmentation model includes the following steps:

[0007] Obtain the labeled medical image dataset;

[0008] Based on the medical image dataset, use a loss function to train the image segmentation model based on Swin-Unet to obtain a pre-trained image segmentation model, wherein the loss function includes a probability priori loss function.

[0009] As a preferred technical solution, the calculation of the probability priori loss function value includes the following steps:

[0010] For the organ class probability atlas output by the image segmentation model, convert it into a MAP weight matrix using a Gaussian function;

[0011] Calculate a probability prior loss function value of the current training round based on the MAP weight matrix and a cross-entropy function.

[0012] As a preferred technical solution, the MAP weight matrix is:

[0013]

[0014] Wherein, σ1, σ2 respectively represent two hyperparameters, p i represents a probability map set of the i-th organ class, represents a probability weight matrix of the i-th organ, p i The value range of p i is 0, which means that the corresponding position has never shown as the target organ class; when p i is 1, it means that all the corresponding positions show the target organ class; when p i is 0.5, it means that the corresponding position has no enough confidence to determine whether it belongs to the target organ class.

[0015] As a preferred technical solution, the probability prior loss function value is calculated by the following formula:

[0016]

[0017]

[0018] Wherein, represents the probability prior loss function of the i-th organ class, Loss represents the final probability prior loss function value, y represents the segmentation label, represents the prediction result, C represents the number of organ classifications, represents the probability weight matrix of the i-th organ, h and w respectively represent the height and width of the image, ωi is a parameter, and N represents the number of samples.

[0019] As a preferred technical solution, the image segmentation model based on Swin-Unet includes:

[0020] An encoder including at least one encoding block;

[0021] An intermediate layer;

[0022] A decoder including at least one encoding block.

[0023] As a preferred technical solution, each of the encoding blocks includes at least one Swin Transformer Block.

[0024] Preferably, the intermediate layer comprises a convolution layer and a ReLU activation function.

[0025] Preferably, after obtaining the medical image dataset, the method further comprises:

[0026] The medical image dataset is divided into a training set and a test set according to a preset ratio.

[0027] In another aspect of the present application, an electronic device is provided, comprising one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs comprise instructions for executing the medical image segmentation method with fused structural prior knowledge.

[0028] In another aspect of the present application, a computer-readable storage medium is provided, comprising one or more programs for execution by one or more processors of an electronic device, and the one or more programs comprise instructions for executing the medical image segmentation method with fused structural prior knowledge.

[0029] Compared with the prior art, the present application has the following advantages:

[0030] (1) Improve the learning effect of the model: unlike some prior art which assigns the same weight to different pixels, the present application trains the model based on a loss function comprising a probability prior in the training process of the model, balances the weights between difficult-to-segment samples, guides the network to perform more balanced feature learning, and thus improves the learning effect of the model.

[0031] (2) Wide application scenarios: the probability prior loss function provided by the present application can be combined with different network structure models to improve the segmentation effect on different model multi-organ medical image segmentation tasks. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 FIG. 1 is a schematic diagram of a medical image segmentation model framework with fused structural prior knowledge;

[0033] Figure 2 FIG. 2 is a schematic diagram of a Swin-Unet structure in a medical image segmentation model with fused structural prior knowledge;

[0034] Figure 3 FIG. 3 is a schematic diagram of a Swin Transformer block structure in a medical image segmentation model with fused structural prior knowledge;

[0035] Figure 4 FIG. 4 is a visualization diagram of a probability weight matrix in a medical image segmentation model with fused structural prior knowledge;

[0036] Figure 5Visualization of prostate segmentation results of different functions for a medical image segmentation model using structural prior knowledge. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work should fall within the protection scope of the present application.

[0038] Embodiment 1

[0039] To solve the problems in the prior art, the embodiment provides a medical image segmentation method using structural prior knowledge. In the training process, the weight of a pixel point is adjusted according to a probability map set for different organs, the learning focus of a network is adjusted, and then the segmentation effect is optimized by using a probability prior loss function.

[0040] The method comprises the following steps:

[0041] S1, obtaining a medical image to be segmented, and labeling a target region.

[0042] S2, preprocessing the original data.

[0043] S3, dividing the image data set into a training set and a test set according to a preset proportion.

[0044] S4, generating a medical image segmentation model, inputting the training set of the data set into the medical image segmentation model, and training the medical image segmentation model.

[0045] S5, inputting the test set of the data set into the trained image segmentation model, and obtaining a segmentation image.

[0046] Wherein, referring to Figure 2 The model framework is mainly designed based on Swin-Unet, and comprises the following structures.

[0047] Encoder block: in the encoding structure of the network, three encoding blocks and one Patch Merging layer are adopted, and each encoding block is composed of 2 Swin Transformer blocks.

[0048] Intermediate layer: mainly composed of a convolution layer and a ReLU activation function.

[0049] Decoder block: in the decoding structure of the network, three layers of decoder blocks corresponding to the encoder and a PatchExpanding layer are used, and each decoder block is composed of 2 Swin Transformer Blocks. As shown in Figure 3 The diagram of the SwinTransformer Block is shown, and each Block first performs LayerNorm operation on the feature map, and then determines whether the window shift operation is needed for the feature map. As shown in Figure 3 As shown in the left side of the figure, the specific shift operation is to shift the feature map, and to indirectly realize it by setting mask for attention, so as to realize the equivalence of the calculation result under the condition that the window number is unchanged. After the calculation is completed, the window needs to be merged and restored. After attention calculation, dropout and residual connection are performed, and finally a LayerNorm and full connection layer, as well as dropout and residual connection, are performed.

[0050] Referring to Figure 1 In the training process, the loss function is defined based on the cross-entropy function, and the probability map set fusion is taken as the prior weight. A probability prior loss function for multi-organ medical image segmentation, namely the MAP loss function, is proposed. The specific implementation of the MAP loss function is as follows: first, the probability prior loss of each class is calculated respectively, and then all the probability prior loss functions of the classes are fused into the MAP loss function. The probability prior loss function of the i-th class is loss pi The expression is as follows:

[0051]

[0052] Where y represents the segmentation label, represents the prediction result, and C represents the classification category.

[0053] Since the sample numbers of different organs are different, different weight relationships are allocated to different organs by different parameter configurations when calculating the loss function. The expression is as follows:

[0054]

[0055] Where ωi is a parameter used to handle the loss imbalance of different organs.

[0056] For the weight matrix The probability weight is converted into the MAP weight matrix by using the Gaussian function, and the probability weight matrix The expression is as follows:

[0057]

[0058] wherein σ1 and σ2 represent two hyperparameters respectively, p i denotes the probability map set of the i-th organ class, denotes the probability weight matrix of the i-th organ. i The value ranges from 0 to 1. When the value is 0, it means that the point has never shown the target class, and the probability of the target class appearing at the point is extremely low. If the value is 1, it means that all the positions in the training set have shown the target class, and the target class has a great possibility to appear at the point. If the value is 0.5, it means that there is not enough confidence to determine whether the point belongs to the target class, and these points are not easy to learn in the training process.

[0059] As Figure 4 The probability weight matrix visualization diagram in the medical image segmentation model fused with the structural prior knowledge is shown. In the actual model training process, greater weight should be assigned to the pixel points that are not easy to learn, and vice versa. Figure 4 In the probability weight matrix, (a), (b), and (c) represent the probability weight matrix of the prostate slice, rectum slice, and seminal vesicle slice respectively. It can be seen from the visualization result that the edge of the target organ is more obvious than the background and the internal organ, which indicates that the weight of the background and the internal organ should be higher than that of the organ edge.

[0060] The following will take the prostate image segmentation as an example for illustration.

[0061] Step 1, obtaining the medical image to be segmented, labeling the target region to form a data set;

[0062] Taking prostate organ segmentation as an example, the data set used in the experiment is the MR image obtained by abdominal scanning imaging through different frequency and pulse combinations, and there are a total of 82 sample data.

[0063] Step 2, preprocessing the original data. In the data preprocessing process, in order to ensure that the images have the same size and resolution, the spatial resolution of the original slice image and the labeled image is uniformly sampled to 224x224;

[0064] Step 3, dividing the image data set into training set and test set according to the preset proportion, the training set is 66 examples, and the test set is 16 examples;

[0065] Step 4, generating a medical image segmentation model, inputting the training set of the data set into the medical image segmentation model to train the medical image segmentation model;

[0066] Step 5, inputting the test set of the data set into the trained image segmentation model to obtain the segmentation image.

[0067] The segmentation result is evaluated by a Dice Similarity Coefficient (DSC) and an average Hausdorff Distance (HD).

[0068] As Figure 5 The figure is a visualization of the prostate segmentation result using different functions. The red, white and yellow areas in the figure represent three different organs in the segmentation task. It can be seen from the figure that the overall segmentation result using the MAP loss function is better than the FocalLoss, DiceLoss and cross-entropy loss functions, and it can also be seen that the MAP is obviously superior to other loss functions in learning complex samples, and can make the network more balanced to learn all the classes, so that the overall segmentation effect is better.

[0069] After multiple rounds of experiments, as shown in Table 1, on the prostate organ segmentation dataset, the DSC reached 81.97%, and the HD reached 3.15.

[0070] Table 1 Experimental results

[0071]

[0072] The present application proposes a MAP loss function for multi-organ medical image segmentation based on a probability graph, which is mainly used to balance the weights between difficult segmentation samples, guide the network to perform more balanced feature learning, and can be combined with different network structure models to improve the segmentation effect of different models in the multi-organ medical image segmentation task.

[0073] Embodiment 2

[0074] The embodiment provides an electronic device, including one or more processors and a memory, the memory has one or more programs stored therein, and the one or more programs include instructions for executing the medical image segmentation method of fusing structure priori knowledge as described in embodiment 1.

[0075] Embodiment 3

[0076] The embodiment provides a computer-readable storage medium, including one or more programs for an electronic device to execute, and the one or more programs include instructions for executing the medical image segmentation method of fusing structure priori knowledge as described in embodiment 1.

[0077] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A medical image segmentation method incorporating prior structural knowledge, characterized in that, The acquired medical images are segmented using a pre-trained Swin-Unet-based image segmentation model. The training process of the image segmentation model includes the following steps: Obtain the labeled medical image dataset; Based on the aforementioned medical image dataset, a Swin-Unet-based image segmentation model is trained using a loss function to obtain a pre-trained image segmentation model. The loss function includes a probabilistic prior loss function. The calculation of the probabilistic prior loss function value includes the following steps: The organ category probability map output by the image segmentation model is converted into a MAP weight matrix using a Gaussian function. Based on the MAP weight matrix and the cross-entropy function, calculate the probabilistic prior loss function value for the current training round. The MAP weight matrix is ​​as follows: ， in, , These represent two hyperparameters, Indicates the first Probability atlases for each organ category Indicates the first The probability weight matrix of each organ. The range of the median value is [0,1]. A value of 0 indicates that the corresponding location has never been represented as the target organ category; when A value of 1 indicates that the corresponding position represents the target organ category; when... A value of 0.5 indicates that the corresponding location does not have sufficient confidence to determine whether it belongs to the target organ category. The probabilistic prior loss function value is calculated using the following formula: ， ， in, Indicates the first The probability prior loss function for each organ category, This represents the final probabilistic prior loss function value. Indicates a separator label. Indicates the prediction result. Indicates the number of organ categories. Indicates the first The probability weight matrix of each organ. , These represent the height and width of the image, respectively. It is a parameter. Indicates the number of samples.

2. The medical image segmentation method based on fused structural prior knowledge according to claim 1, characterized in that, The image segmentation model based on Swin-Unet includes: An encoder, comprising at least one encoding block; Intermediate layer; A decoder, comprising at least one encoded block.

3. The medical image segmentation method based on fused structural prior knowledge according to claim 2, characterized in that, Each of the coded blocks includes at least one Swing Transformer Block.

4. The medical image segmentation method based on fused structural prior knowledge according to claim 2, characterized in that, The intermediate layers include convolutional layers and ReLU activation functions.

5. The medical image segmentation method based on fused structural prior knowledge according to claim 1, characterized in that, After acquiring the medical image dataset, the following is also included: The medical image dataset is divided into a training set and a test set according to a preset ratio.

6. An electronic device, characterized in that, include: One or more processors and a memory, wherein the memory stores one or more programs, the one or more programs including instructions for performing a medical image segmentation method based on the fusion structure prior knowledge as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, Includes one or more programs executable by one or more processors of an electronic device, the one or more programs including instructions for performing a medical image segmentation method based on prior knowledge of the fused structure as described in any one of claims 1-5.

Images