Medical image segmentation model training method and system and medical image segmentation method

Abstract

The invention provides a medical image segmentation model training method and system and a medical image segmentation method, and the training method comprises the steps: pre-training a large model with high segmentation precision as a first teacher network, building a semantic segmentation student network with RGCN, building a confidence coefficient discrimination network, and carrying out the segmentation of the first teacher network and the second teacher network; and inputting a probability graph which is output by the first teacher network and represents each pixel point into the established confidence discrimination network, obtaining a confidence graph, and finally fusing the confidence graph into the training of the student network. Knowledge distillation with confidence is adopted, correct knowledge of a large model which is good in segmentation effect but low in segmentation speed is transmitted to a small model which is high in segmentation speed and has RGCN feature fusion, and the small model has segmentation efficiency and precision. According to the invention, the negative influence of the teacher network on the student network in the traditional knowledge distillation process can be avoided, so that a better knowledge distillation effect is obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a medical image segmentation model training method, system and medical image segmentation method. Background Art

[0002] Medical images of surgical scenes refer to real-time images or videos obtained through different medical imaging technologies during the surgical process. In medical image processing, medical image segmentation technology plays a vital role, especially in the field of robot autonomy. With the rapid development of medical robot technology, the robot's processing capabilities for medical images are increasingly required. These requirements require not only high-precision segmentation results, but also real-time performance to support the robot's autonomous decision-making and operation in a dynamic environment.

[0003] Traditional medical image segmentation methods mostly rely on rule-based algorithms, such as thresholding and region growing methods, but these methods usually face the problems of low accuracy and slow processing speed. With the significant progress made in deep learning, especially convolutional neural networks (CNN) in medical image segmentation, more accurate segmentation results can be provided. However, deep learning methods often require a lot of computing resources, which poses a challenge to real-time processing and robotic applications.

[0004] The Transformer is a model that uses the attention mechanism to improve long-distance semantic connections. The attention mechanism is a mechanism that improves the recognition accuracy of the model by adjusting the attention to the input weight distribution. The Transformer model encodes input and calculates output based on the attention mechanism, allowing the model to be trained in parallel and have global information, which helps to improve the recognition accuracy of the model.

[0005] In recent years, the application of image processing technology in natural images has made great progress, but there are still many difficulties in the processing of medical images. For example, the semantic information contained in medical images is relatively simple, so it is difficult to obtain effective information in medical images. How to perform semantic segmentation in a more accurate and effective way is still a problem that needs to be solved by industry insiders. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a medical image segmentation model training method, system and medical image segmentation method.

[0007] In a first aspect, the present invention provides a medical image segmentation model training method, comprising:

[0008] Pre-train the teacher network according to the existing medical images, and obtain the pre-trained teacher network as the first teacher network;

[0009] Using the first teacher network to infer the medical image sequence of the first target tissue, and obtaining a probability map predicted by the first teacher network;

[0010] Convert existing medical image labels into one-hot encodings;

[0011] The existing medical image and one-hot encoded label are used as the first set of data, the medical image of the first target tissue and the probability map predicted by the first teacher network are used as the second set of data, and are simultaneously input into the pixel-level confidence map generation network to obtain the first confidence map and the second confidence map respectively;

[0012] Determining a total loss of a pixel-level confidence map generation network to train the pixel-level confidence map generation network based on the first confidence map, the second confidence map, and the total loss of the pixel-level confidence map generation network;

[0013] The image to be segmented in the medical image sequence of the first target tissue is input into the student network of the dual-stream RGCN feature fusion to obtain the probability map predicted by the student network and the prediction result of the student network;

[0014] Inputting the image to be segmented in the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain a third confidence map;

[0015] determining a knowledge distillation loss based on the third confidence map, the probability map predicted by the student network, and the probability map predicted by the first teacher network;

[0016] Determine the Ohem loss based on the student network prediction results and the labels of the medical images;

[0017] The student network is trained according to the knowledge distillation loss and Ohem loss, and the trained student network is used as a medical image segmentation model.

[0018] Optionally, determining the total loss of the pixel-level confidence map generation network to train the pixel-level confidence map generation network according to the first confidence map, the second confidence map and the total loss of the pixel-level confidence map generation network includes:

[0019] The total loss L of the pixel-level confidence map generation network is calculated according to the following formula G :

[0020]

[0021] Where a' is the pixel-by-pixel loss L α The coefficient of b' is the similarity loss L β The coefficient of ; w is the width of the medical image; h is the height of the medical image; represents all elements of the jth column of the first confidence map used to train the confidence generation network; represents an all-1 matrix; n is the nth small segment in the medical image; t is the tth frame of the nth small segment in the medical image; represents all elements of the jth column of the all-one matrix; T represents the transpose of the matrix; represents the confidence value at row i and column j of the second confidence map; represents the probability map of the first teacher network and the JS divergence value calculated by the label; c is the total number of channels; The probability of the pixel coordinate (i, j) on channel k representing the label in one-hot encoding form; Represents the probability at pixel coordinate (i, j) on channel k of the probability map predicted by the first teacher network.

[0022] Optionally, the step of inputting the image to be segmented in the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into a trained pixel-level confidence map generation network to obtain a third confidence map comprises:

[0023] Constructing the third confidence map The expression is:

[0024]

[0025] Wherein, n is the nth small segment in the medical image; t is the tth frame of the nth small segment in the medical image; Indicates that the superimposed images are processed through the confidence generation network; An image to be segmented in a sequence of medical images organized as a first target; represents the superposition operation along the channel dimension; Probability plot predicted by the first teacher network.

[0026] Optionally, determining the knowledge distillation loss according to the third confidence map, the probability map predicted by the student network, and the probability map predicted by the first teacher network includes:

[0027] The knowledge distillation loss L is calculated according to the following formula soft :

[0028]

[0029] Among them, T d represents the distillation temperature; w is the width of the medical image; h is the height of the medical image; represents the value of the i-th row and j-th column of the third confidence map used for student network training; D KL (·) represents KL divergence; The probability vector of the probability map predicted by the first teacher network at the pixel (i, j); is the probability vector of the probability map predicted by the student network at the pixel (i, j); C' is the total number of probability value types; represents the probability value of the c'th class at (i, j) of the probability graph predicted by the first teacher network; Represents the probability value of the c'th class at (i,j) of the probability graph predicted by the student network.

[0030] Optionally, determining the Ohem loss according to the student network prediction result and the label of the medical image includes:

[0031] The Ohem loss L is calculated according to the following formula hard :

[0032]

[0033] Among them, N S Indicates the total number of elements considered in the loss calculation process; i is the i-th element involved in the calculation; I S (i) represents the cross entropy of the i-th pixel involved in the calculation; L CE The matrix obtained by calculating the cross entropy between the label and the student network segmentation result; Represents the one-hot encoded label of the existing medical image; F seg is the prediction result of the student network; I CE For L CE The expanded vector form; τ dis is the preset threshold; I CE (n min ) means greater than τ dis The minimum cross entropy loss value; I CE [:n min ] represents the vector composed of the maximum cross entropy value to the minimum cross entropy value after sorting.

[0034] Optionally, the step of training a student network according to the knowledge distillation loss and the Ohem loss, and using the trained student network as a medical image segmentation model, includes:

[0035] The total loss L of the student network is calculated according to the following formula total :

[0036] L total =αL soft +βL hard ;

[0037] Among them, α is the knowledge distillation loss L soft The coefficient of; β is the Ohm loss L hard The coefficient of

[0038] Take the total loss L of the student network total Train the student network and use the trained student network as a medical image segmentation model.

[0039] In a second aspect, the present invention provides a medical image segmentation model training system, comprising:

[0040] A pre-training module is used to pre-train the teacher network according to the existing medical images, and obtain the pre-trained teacher network as the first teacher network;

[0041] an inference module, configured to use the first teacher network to infer the medical image sequence of the first target tissue to obtain a probability map predicted by the first teacher network;

[0042] The conversion module is used to convert the existing medical image labels into one-hot encoding;

[0043] A first generation module is used to take the existing medical image and the one-hot encoded label as a first set of data, the medical image of the first target tissue and the probability map predicted by the first teacher network as a second set of data, and simultaneously input them into a pixel-level confidence map generation network to obtain a first confidence map and a second confidence map, respectively;

[0044] A first determination module is used to determine a total loss of a pixel-level confidence map generation network, so as to train the pixel-level confidence map generation network according to the first confidence map, the second confidence map and the total loss of the pixel-level confidence map generation network;

[0045] The second generation module is used to input the image to be segmented in the medical image sequence of the first target tissue into the student network of the dual-stream RGCN feature fusion to obtain the probability map predicted by the student network and the student network prediction result;

[0046] A third generation module is used to input the image to be segmented in the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain a third confidence map;

[0047] a second determination module, for determining a knowledge distillation loss according to a third confidence map, a probability map predicted by the student network, and a probability map predicted by the first teacher network;

[0048] The third determination module is used to determine the Ohem loss based on the student network prediction results and the labels of the medical images;

[0049] The training module is used to train the student network based on the knowledge distillation loss and Ohem loss, and use the trained student network as a medical image segmentation model.

[0050] In a third aspect, the present invention provides a computer device comprising a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the steps of the medical image segmentation model training method described in the first aspect are implemented.

[0051] In a fourth aspect, the present invention provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, the steps of the medical image segmentation model training method described in the first aspect are implemented.

[0052] In a fifth aspect, the present invention provides a medical image segmentation method, comprising:

[0053] acquiring a medical image of a second target tissue;

[0054] The medical image of the second target tissue is input into the medical image segmentation model to obtain a medical image segmentation result of the second target tissue output by the medical image segmentation model; wherein the medical image segmentation model is trained according to the medical image segmentation model training method described in the first aspect.

[0055] The present invention provides a medical image segmentation model training method, system and medical image segmentation method. The medical image segmentation model training method adopts knowledge distillation with confidence, and transmits the correct knowledge of a large model with good segmentation effect but slow segmentation speed to a small model with RGCN feature fusion with fast segmentation speed, thereby improving the segmentation performance of the small model and maintaining good segmentation efficiency. The present invention can avoid the negative impact of the teacher network on the student network in the traditional knowledge distillation process, thereby obtaining a better knowledge distillation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A schematic diagram of a flow chart of a medical image segmentation model training method provided by an embodiment of the present invention;

[0058] Figure 2 A schematic diagram of the structure of the multi-scale feature R-GCN fusion provided in an embodiment of the present invention;

[0059] Figure 3 A schematic diagram of the structure of the spatial context feature R-GCN fusion provided in an embodiment of the present invention;

[0060] Figure 4Results of surgical scene segmentation using different methods provided by embodiments of the present invention on public and internal data sets;

[0061] Figure 5 A qualitative comparison result diagram of different methods provided in embodiments of the present invention on Endovis18 data;

[0062] Figure 6 A qualitative comparison result diagram of different methods provided in embodiments of the present invention on the CholecSeg8K data;

[0063] Figure 7 Recognition result diagrams of various categories provided by the embodiments of the present invention;

[0064] Figure 8 A schematic diagram of the structure of a medical image segmentation model training system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] Example 1

[0067] like Figure 1 As shown, this embodiment provides a medical image segmentation model training method, including:

[0068] Step 101, pre-training a teacher network according to existing medical images, obtaining a pre-trained teacher network as a first teacher network.

[0069] In this embodiment, the medical image can be obtained through a laparoscope.

[0070] Step 102, using the first teacher network to infer the medical image sequence of the first target tissue to obtain a probability map predicted by the first teacher network.

[0071] In this embodiment, it is necessary to obtain the probability map predicted by the first teacher network before the softmax function in the first classroom network segmentation head.

[0072] Step 103: convert the existing medical image labels into one-hot codes.

[0073] In step 104, the existing medical image and one-hot encoded labels are used as the first set of data, the medical image of the first target tissue and the probability map predicted by the first teacher network are used as the second set of data, and are simultaneously input into the pixel-level confidence map generation network to obtain the first confidence map and the second confidence map, respectively.

[0074] This embodiment uses a pixel-level confidence map generation network (PCG) to more accurately evaluate the credibility of the knowledge of the first teacher network. Multi-scale features are extracted through multiple convolutional downsampling layers, and global context information is obtained in combination with the self-attention mechanism. In the upsampling stage, the feature map is restored to the original resolution using transposed convolution, while shallow features are connected to retain local details. The final output confidence matrix can achieve finer-grained distinctions compared to simple probability maps. In order to generate reliable confidence maps, this embodiment uses two sets of data as PCG training inputs. The first group includes existing medical images and one-hot encoded labels, and outputs all-one confidence maps during training. In order to enhance stability, this embodiment designs a pixel-by-pixel loss and uses the mean square error (MSE) to force PCG to generate results that are closer to all 1.

[0075] Step 105, determining the total loss of the pixel-level confidence map generation network, so as to train the pixel-level confidence map generation network according to the first confidence map, the second confidence map and the total loss of the pixel-level confidence map generation network.

[0076] In order to improve the robustness of pixel-level confidence map generation (PCG), we hope to accurately identify pixels under ideal conditions, effectively distinguish the wrong parts in complex non-ideal environments, and avoid the model from converging too quickly and overfitting. Through JS divergence, the similarity between the output probability of the first teacher network and the one-hot label is calculated to obtain a similarity matrix. By comparing the similarity matrix with the output results of the PCG network, the PCG network is updated to enable it to face more complex scenes.

[0077] Exemplarily, the total loss L of the pixel-level confidence map generation network is calculated according to the following formula G :

[0078]

[0079] Where a' is the pixel-by-pixel loss L α The coefficient of b' is the similarity loss L β The coefficient of ; w is the width of the medical image; h is the height of the medical image; represents all elements of the jth column of the first confidence map used to train the confidence generation network; represents an all-1 matrix; n is the nth small segment in the medical image; t is the tth frame of the nth small segment in the medical image; represents all elements of the jth column of the all-one matrix; T represents the transpose of the matrix; represents the confidence value at row i and column j of the second confidence map; represents the probability map of the first teacher network and the JS divergence value calculated by the label; c is the total number of channels; The probability of the pixel coordinate (i, j) on channel k representing the label in one-hot encoding form; Represents the probability at pixel coordinate (i, j) on channel k of the probability map predicted by the first teacher network.

[0080] Step 106: input the image to be segmented in the medical image sequence of the first target tissue into the student network of the dual-stream RGCN feature fusion to obtain the probability map predicted by the student network and the prediction result of the student network.

[0081] This embodiment designs two feature fusion methods:

[0082] 1) Multi-scale feature R-GCN fusion

[0083] In multi-scale feature fusion, the feature maps processed by the three dilated convolutions with different dilation rates are subjected to 3×3 convolutions to project them to lower dimensions and recorded as graph nodes. Convolutions with smaller dilation rates correspond to lower receptive fields. It is expected that the feature maps with low and high receptive fields can jointly provide effective information for the feature maps with medium receptive fields to enhance detail representation and improve global perception. Therefore, edges are added between nodes to fuse information and form complementary features. The feature fusion framework is shown in Figure 2 As shown, the formula is as follows:

[0084]

[0085] in, represents the updated vector of node b; σ(·) is the activation function; c a is the normalization factor of node a; W a is the weight coefficient of node a; represents the node a before the update; c c is the normalization factor of node c; W c is the weight coefficient of node c; represents the node c before the update; W b is the weight coefficient of node b; Represents node b before the update.

[0086] 2) Spatial context feature R-GCN fusion

[0087] In the second feature extraction module, a sliding window and a self-attention module are used to capture the spatial relationship of pixels within the window. Although a large window can provide more pixel information, it will significantly increase the computational burden, thereby affecting efficiency. To this end, this embodiment designs four different sliding windows (1×1, 4×4, 8×8, and 16×16), performs self-attention calculations in each window, and extracts the associations between pixels. However, this method fails to achieve cross-window information transfer. Although a large window may contain information from a small window, there is a lack of interaction between 16×16 windows. Figure 3 As shown, this embodiment introduces a second RGCN (G2) to fuse spatial features.

[0088] The fusion graph G2 of the second stream is expressed as:

[0089]

[0090] in, Represents the weight of the node in row i and column j; represents the normalization factor; represents the (i, j) node before the update; W i,j is the weight of node (i, j); For node (i n ,j n )’s weight.

[0091] The feature maps with multi-scale and spatial context are superimposed together and passed to the segmentation head for semantic segmentation. In the ablation experiment, the calculation time is reduced and even computing resources are saved.

[0092] Step 107, input the image to be segmented in the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain a third confidence map.

[0093] In this step, illustratively, a third confidence map is constructed The expression is:

[0094]

[0095] Wherein, n is the nth small segment in the medical image; t is the tth frame of the nth small segment in the medical image; Indicates that the superimposed images are processed through the confidence generation network; An image to be segmented in a sequence of medical images organized as a first target; represents the superposition operation along the channel dimension; Probability plot predicted by the first teacher network.

[0096] Step 108, determining the knowledge distillation loss based on the third confidence map, the probability map predicted by the student network, and the probability map predicted by the first teacher network.

[0097] In this step, the knowledge distillation loss L is calculated illustratively according to the following formula: soft :

[0098]

[0099] Among them, T d represents the distillation temperature; w is the width of the medical image; h is the height of the medical image; represents the value of the i-th row and j-th column of the third confidence map used for student network training; D KL (·) represents KL divergence; The probability vector of the probability map predicted by the first teacher network at the pixel (i, j); is the probability vector of the probability map predicted by the student network at the pixel (i, j); C' is the total number of probability value types; represents the probability value of the c'th class at (i, j) of the probability graph predicted by the first teacher network; Represents the probability value of the c'th class at (i,j) of the probability graph predicted by the student network.

[0100] Step 109, determining the Ohem loss according to the student network prediction result and the label of the medical image.

[0101] This embodiment calculates the minimum number of pixels n that need to be included in the loss by using the image size and the number of downsampled pixels. min In order to improve the performance of supervised learning, the Ohem loss is used to guide the model to pay more attention to pixels with significant differences and ignore those pixels whose results are closer to the true value. To achieve this idea, a change threshold τ is preset. dis To measure the descending order of I CE The degree of difference,

[0102] In this step, the Ohm loss L is calculated illustratively according to the following formula: hard :

[0103]

[0104] Among them, N S Indicates the total number of elements considered in the loss calculation process; i is the i-th element involved in the calculation; I S (i) represents the cross entropy of the i-th pixel involved in the calculation; L CE The matrix obtained by calculating the cross entropy between the label and the student network segmentation result; Represents the one-hot encoded label of the existing medical image; Fseg is the prediction result of the student network; I CE For L CE The expanded vector form; τ dis is the preset threshold; I CE (n min ) means greater than τ dis The minimum cross entropy loss value; I CE [:n min ] represents the vector composed of the maximum cross entropy value to the minimum cross entropy value after sorting.

[0105] Step 1010, training the student network according to the knowledge distillation loss and the Ohem loss, and using the trained student network as a medical image segmentation model.

[0106] In this step, the total loss L of the student network is calculated illustratively according to the following formula: total :

[0107] L total =αL soft +βL hard .

[0108] Among them, α is the knowledge distillation loss L soft The coefficient; β is the Ohm loss L hard The coefficient of .

[0109] Take the total loss L of the student network total Train the student network and use the trained student network as a medical image segmentation model.

[0110] This embodiment uses the similarity coefficient (Dice) and the intersection over union (IoU) as evaluation indicators for the accuracy of the proposed method. Formally speaking, predictions can be divided into true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). Then the indicators Dice and IoU can be described as:

[0111]

[0112] Pixel-level segmentation accuracy comparison experiment: The method provided in this embodiment is compared with SOTA methods based on different architectures, including cumbersome teacher models STswinCL, NoisyLSTM, UNet3+, TransUNet, transformer-based Swin, and baseline DeeplabV3+. Both UNet3+ and DeepLabV3+ adopt various multi-scale feature extraction strategies. TransUnet and STswin utilize Transformer-based architectures to manage long-range dependencies and global context. However, they may lose details in the feature fusion stage. The model provided in this embodiment integrates RGCN, further fuses multi-scale and cross-space information, and achieves excellent coordination between global and local knowledge parsing and laparoscopic understanding.

[0113] like Figure 4 As shown in the figure, the PLDKD-Net proposed in this embodiment has achieved the best average Dice and IoU in two public datasets (EndoVis2018 & CholecSeg8K) and an internal dataset, and the standard deviation of the performance between different video clips is the smallest, which means stronger generalization and the most stable performance. Specifically, the average Dice of the 12-category segmentation task in the four test clips of EndoVis 2018 is 68.07%, 69.84%, 88.59% and 48.25%, and the average Dice of the 13-category semantic segmentation in the four test clips of the CholecSeg8K dataset is 74.83%, 96.62%, 97.46% and 72.72%. In the dataset of this embodiment, although the amount of data is relatively small, the method proposed in this embodiment still achieves the best effect.

[0114] In addition, in order to intuitively display the segmentation results, the qualitative comparison results of this embodiment on the EndoVis18 and Kaggle CholecSeg8K datasets are presented. This embodiment uses different colors to represent various tissues and instruments in the laparoscopic scene, and is compared with the baseline DeepLabV3+, STswin and the popular segmentation models UNet3+ and TransUnet. The model of this embodiment uses the discriminant module and is good at correcting the mistakes made by the teacher model, such as Figure 5 and Figure 6 As shown in FIG. 1 , the output result of the medical image segmentation model of this embodiment is also closer to the label.

[0115] Object-level segmentation performance comparison test: Considering the complex laparoscopic scene with various instruments and tissues, the segmentation performance of the model for each object category was tested accordingly. The results are shown in Figure 2. Figure 7As shown. Compared with methods based on different structures, the method of the present embodiment intuitively outperforms DeeplabV3+ and UNet3+ in most object classes of these two methods. Although the teacher model provides superior performance based on its cumbersome and complex learning mechanism, the method of the present embodiment can surpass the teacher's segmentation score through the ConfiKD module, which can identify the wrong parts of the teacher to avoid negative optimization. As emphasized, the model of the present embodiment surpasses the teacher model when segmenting renal parenchyma ("Kidney") and covered kidneys ("Covered") in the Endovis 2018 dataset, and achieves higher "liver" and "blood" segmentation IoU scores in CholecSeg8K, as well as when identifying the instrument shaft ("IS") and suction instrument ("SI") in the internal data of the present embodiment. These extensive validations collectively demonstrate that the method of the present embodiment can output satisfactory segmentation results for a variety of objects in complex environments.

[0116] In summary, this embodiment provides a medical image segmentation model training method, which uses knowledge distillation with confidence to transfer the correct knowledge of a large model with good segmentation effect but slow segmentation speed to a small model with RGCN feature fusion with faster segmentation speed, thereby improving the segmentation performance of the small model while maintaining good segmentation efficiency. The present invention can avoid the negative impact of the teacher network on the student network in the traditional knowledge distillation process, thereby obtaining a better knowledge distillation effect.

[0117] Example 2

[0118] Based on the same inventive concept as Example 1, this embodiment also provides a medical image segmentation model training system. Since the principle of solving the problem by this system is similar to the aforementioned medical image segmentation model training method, the implementation of this system can refer to the implementation of the medical image segmentation model training method.

[0119] like Figure 8 As shown, the medical image segmentation model training system includes:

[0120] The pre-training module 10 is used to pre-train the teacher network according to the existing medical images, and obtain the pre-trained teacher network as the first teacher network.

[0121] The reasoning module 20 is used to use the first teacher network to reason about the medical image sequence of the first target tissue to obtain a probability map predicted by the first teacher network.

[0122] The conversion module 30 is used to convert the existing medical image labels into one-hot codes.

[0123] The first generation module 40 is used to take the existing medical image and one-hot encoded labels as the first set of data, the medical image of the first target tissue and the probability map predicted by the first teacher network as the second set of data, and simultaneously input them into the pixel-level confidence map generation network to obtain the first confidence map and the second confidence map, respectively.

[0124] The first determination module 50 is used to determine the total loss of the pixel-level confidence map generation network, so as to train the pixel-level confidence map generation network according to the first confidence map, the second confidence map and the total loss of the pixel-level confidence map generation network.

[0125] The second generation module 60 is used to input the image to be segmented in the medical image sequence of the first target tissue into the student network of the dual-stream RGCN feature fusion to obtain the probability map predicted by the student network and the student network prediction result.

[0126] The third generation module 70 is used to input the image to be segmented in the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain a third confidence map.

[0127] The second determination module 80 is used to determine the knowledge distillation loss according to the third confidence map, the probability map predicted by the student network and the probability map predicted by the first teacher network.

[0128] The third determination module 90 is used to determine the Ohem loss according to the student network prediction result and the label of the medical image.

[0129] The training module 100 is used to train the student network according to the knowledge distillation loss and the Ohem loss, and use the trained student network as a medical image segmentation model.

[0130] For more specific working processes of the above modules, please refer to the corresponding contents disclosed in Example 1, which will not be repeated here.

[0131] Example 3

[0132] This embodiment provides a medical image segmentation method, including:

[0133] A medical image of a second target tissue is acquired.

[0134] The medical image of the second target tissue is input into the medical image segmentation model to obtain a medical image segmentation result of the second target tissue output by the medical image segmentation model; wherein the medical image segmentation model is trained according to the medical image segmentation model training method described in Example 1.

[0135] Example 4

[0136] This embodiment provides a computer device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the steps of the medical image segmentation model training method described in Example 1 are implemented.

[0137] For more specific details of the above method, please refer to the corresponding contents disclosed in Example 1, which will not be repeated here.

[0138] Example 5

[0139] This embodiment provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, the steps of the medical image segmentation model training method described in Embodiment 1 are implemented.

[0140] For more specific details of the above method, please refer to the corresponding contents disclosed in Example 1, which will not be repeated here.

[0141] Example 6

[0142] This embodiment provides a computer program product, including computer executable instructions or a computer program. When the computer executable instructions or the computer program are executed by a processor, the steps of the medical image segmentation model training method described in Example 1 are implemented.

[0143] For more specific details of the above method, please refer to the corresponding contents disclosed in Example 1, which will not be repeated here.

[0144] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the systems, devices, storage media, and computer program products disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0145] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.

[0146] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0147] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0148] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0149] The present invention has been described in detail above in conjunction with specific implementations and exemplary examples, but these descriptions cannot be understood as limiting the present invention. Those skilled in the art understand that, without departing from the spirit and scope of the present invention, a variety of equivalent substitutions, modifications or improvements may be made to the technical solution of the present invention and its implementation methods, all of which fall within the scope of the present invention. The scope of protection of the present invention shall be subject to the attached claims.

Claims

1. A medical image segmentation model training method, characterized in that: include: Pre-train the teacher network according to the existing medical images, and obtain the pre-trained teacher network as the first teacher network; Using the first teacher network to infer the medical image sequence of the first target tissue, and obtaining a probability map predicted by the first teacher network; Convert existing medical image labels into one-hot encodings; The existing medical image and one-hot encoded label are used as the first set of data, the medical image of the first target tissue and the probability map predicted by the first teacher network are used as the second set of data, and are simultaneously input into the pixel-level confidence map generation network to obtain the first confidence map and the second confidence map respectively; Determining a total loss of a pixel-level confidence map generation network to train the pixel-level confidence map generation network based on the first confidence map, the second confidence map, and the total loss of the pixel-level confidence map generation network; The image to be segmented in the medical image sequence of the first target tissue is input into the student network of the dual-stream RGCN feature fusion to obtain the probability map predicted by the student network and the prediction result of the student network; Inputting the image to be segmented in the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain a third confidence map; determining a knowledge distillation loss based on the third confidence map, the probability map predicted by the student network, and the probability map predicted by the first teacher network; Determine the Ohem loss based on the student network prediction results and the labels of the medical images; The student network is trained according to the knowledge distillation loss and Ohem loss, and the trained student network is used as a medical image segmentation model.

2. The medical image segmentation model training method according to claim 1, characterized in that: The determining the total loss of the pixel-level confidence map generation network to train the pixel-level confidence map generation network according to the first confidence map, the second confidence map and the total loss of the pixel-level confidence map generation network includes: The total loss L of the pixel-level confidence map generation network is calculated according to the following formula G : Where a' is the pixel-by-pixel loss L α The coefficient of b' is the similarity loss L β The coefficient of ; w is the width of the medical image; h is the height of the medical image; represents all elements of the jth column of the first confidence map used to train the confidence generation network; represents an all-1 matrix; n is the nth small segment in the medical image; t is the tth frame of the nth small segment in the medical image; represents all elements of the jth column of the all-one matrix; T represents the transpose of the matrix; represents the confidence value at row i and column j of the second confidence map; represents the probability map of the first teacher network and the JS divergence value calculated by the label; c is the total number of channels; The probability of the pixel coordinate (i, j) on channel k representing the label in one-hot encoding form; Represents the probability at pixel coordinate (i, j) on channel k of the probability map predicted by the first teacher network.

3. The medical image segmentation model training method according to claim 1, characterized in that: The image to be segmented in the medical image sequence of the first target tissue and the probability map predicted by the first teacher network are input into the trained pixel-level confidence map generation network to obtain a third confidence map, including: Constructing the third confidence map The expression is: Wherein, n is the nth small segment in the medical image; t is the tth frame of the nth small segment in the medical image; Indicates that the superimposed images are processed through the confidence generation network; An image to be segmented in a sequence of medical images organized as a first target; represents the superposition operation along the channel dimension; Probability plot predicted by the first teacher network.

4. The medical image segmentation model training method according to claim 1, characterized in that: The determining of the knowledge distillation loss according to the third confidence map, the probability map predicted by the student network, and the probability map predicted by the first teacher network includes: The knowledge distillation loss L is calculated according to the following formula soft : Among them, T d represents the distillation temperature; w is the width of the medical image; h is the height of the medical image; represents the value of the i-th row and j-th column of the third confidence map used for student network training; D KL (·) represents KL divergence; The probability vector of the probability map predicted by the first teacher network at the pixel (i, j); is the probability vector of the probability map predicted by the student network at the pixel (i, j); C' is the total number of probability value types; represents the probability value of the c'th class at (i, j) of the probability graph predicted by the first teacher network; Represents the probability value of the c'th class at (i,j) of the probability graph predicted by the student network.

5. The medical image segmentation model training method according to claim 1, characterized in that: The Ohem loss is determined based on the student network prediction results and the labels of the medical images, including: The Ohem loss L is calculated according to the following formula hard : Among them, N S Represents the total number of elements considered in the loss calculation process; i is the i-th element involved in the calculation; I S (i) represents the cross entropy of the i-th pixel involved in the calculation; L CE The matrix obtained by calculating the cross entropy between the label and the student network segmentation result; Represents the one-hot encoded label of the existing medical image; F seg is the prediction result of the student network; I CE For L CE The expanded vector form; τ dis is the preset threshold; I CE (n min ) means greater than τ dis The minimum cross entropy loss value; I CE [:n min ] represents the vector composed of the maximum cross entropy value to the minimum cross entropy value after sorting.

6. The medical image segmentation model training method according to claim 1, characterized in that: The student network is trained according to the knowledge distillation loss and the Ohem loss, and the trained student network is used as a medical image segmentation model, including: The total loss L of the student network is calculated according to the following formula total : L total =αL soft +βL hard ; Among them, α is the knowledge distillation loss L soft The coefficient of; β is the Ohm loss L hard The coefficient of Take the total loss L of the student network total Train the student network and use the trained student network as a medical image segmentation model.

7. A medical image segmentation model training system, characterized in that: include: A pre-training module is used to pre-train the teacher network according to the existing medical images, and obtain the pre-trained teacher network as the first teacher network; an inference module, configured to use the first teacher network to infer the medical image sequence of the first target tissue to obtain a probability map predicted by the first teacher network; The conversion module is used to convert the existing medical image labels into one-hot encoding; A first generation module is used to take the existing medical image and the one-hot encoded label as a first set of data, the medical image of the first target tissue and the probability map predicted by the first teacher network as a second set of data, and simultaneously input them into a pixel-level confidence map generation network to obtain a first confidence map and a second confidence map, respectively; A first determination module is used to determine a total loss of a pixel-level confidence map generation network, so as to train the pixel-level confidence map generation network according to the first confidence map, the second confidence map and the total loss of the pixel-level confidence map generation network; The second generation module is used to input the image to be segmented in the medical image sequence of the first target tissue into the student network of the dual-stream RGCN feature fusion to obtain the probability map predicted by the student network and the student network prediction result; A third generation module is used to input the image to be segmented in the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain a third confidence map; a second determination module, for determining a knowledge distillation loss according to a third confidence map, a probability map predicted by the student network, and a probability map predicted by the first teacher network; The third determination module is used to determine the Ohem loss based on the student network prediction results and the labels of the medical images; The training module is used to train the student network based on the knowledge distillation loss and Ohem loss, and use the trained student network as a medical image segmentation model.

8. A computer device, characterized in that: It comprises a processor and a memory; wherein, when the processor executes the computer program stored in the memory, the steps of the medical image segmentation model training method described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: Used to store computer programs; when the computer programs are executed by a processor, the steps of the medical image segmentation model training method described in any one of claims 1 to 6 are implemented.

10. A medical image segmentation method, characterized in that: include: acquiring a medical image of a second target tissue; The medical image of the second target tissue is input into the medical image segmentation model to obtain a medical image segmentation result of the second target tissue output by the medical image segmentation model; wherein the medical image segmentation model is trained according to the medical image segmentation model training method according to any one of claims 1 to 6.

Images