Medical image segmentation model training methods, systems, and medical image segmentation methods

Abstract

This invention provides a training method, system, and approach for medical image segmentation. The training method includes pre-training a large model with high segmentation accuracy as a first teacher network, constructing a semantic segmentation student network with RGCN, then building a confidence discriminant network. The probability map representing each pixel, output by the first teacher network, is input into the constructed confidence discriminant network to obtain a confidence map. Finally, the confidence map is fused into the training of the student network. This invention employs knowledge distillation with confidence, transferring the correct knowledge from a large model with good segmentation performance but slow segmentation speed to a smaller model with faster segmentation speed and RGCN feature fusion, thus enabling the smaller model to achieve both segmentation efficiency and accuracy. This invention avoids the negative impact of the teacher network on the student network during traditional knowledge distillation, thereby achieving better knowledge distillation results.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and particularly relates to a medical image segmentation model training method, system, and medical image segmentation method. Background Technology

[0002] Surgical scene medical images refer to real-time images or videos obtained during surgery using various medical imaging technologies. Medical image segmentation technology plays a crucial role in medical image processing, especially in the field of robotic autonomy. With the rapid development of medical robotics technology, the demands on robots' medical image processing capabilities are increasing. These requirements not only necessitate high-precision segmentation results but also real-time performance to support autonomous decision-making and operation of robots in dynamic environments.

[0003] Traditional medical image segmentation methods largely rely on rule-based algorithms, such as thresholding and region growing, but these methods typically suffer from low accuracy and slow processing speed. While deep learning, especially convolutional neural networks (CNNs), has made significant progress in medical image segmentation, offering more accurate results, deep learning methods often require substantial computational resources, posing challenges for real-time processing and robotic applications.

[0004] Self-attention networks (Transformers) are models that utilize attention mechanisms to improve long-distance semantic connections. The attention mechanism improves model recognition accuracy by adjusting the allocation of weights to the inputs. Transformer models encode inputs and compute outputs based on attention mechanisms, allowing for parallel training and access to global information, thus contributing to improved recognition accuracy.

[0005] In recent years, image processing technology has made significant progress in the application of natural images. However, many challenges remain in the processing of medical images. For example, medical images often contain relatively limited semantic information, making it difficult to extract meaningful information. Therefore, finding a more accurate and effective method for semantic segmentation remains a pressing issue for the industry. Summary of the Invention

[0006] This invention addresses the shortcomings of existing technologies by providing a medical image segmentation model training method, system, and medical image segmentation method.

[0007] In a first aspect, the present invention provides a method for training a medical image segmentation model, comprising:

[0008] The teacher network is pre-trained based on existing medical images to obtain a pre-trained teacher network, which is then used as the first teacher network.

[0009] The first teacher network is used to infer the medical image sequence of the first target tissue, and the probability map predicted by the first teacher network is obtained.

[0010] Convert the labels of existing medical images to one-hot encoding;

[0011] Existing medical images and one-hot encoded labels are used as the first set of data, and medical images of the first target tissue and probability maps predicted by the first teacher network are used as the second set of data. These are simultaneously input into a pixel-level confidence map generation network to obtain the first confidence map and the second confidence map, respectively.

[0012] Determine the total loss of the pixel-level confidence map generation network, and train the pixel-level confidence map generation network based on the first confidence map, the second confidence map, and the total loss of the pixel-level confidence map generation network;

[0013] The images that need to be segmented in the medical image sequence of the first target tissue are input into the student network of dual-stream RGCN feature fusion to obtain the probability map and prediction results of the student network.

[0014] The images to be segmented from the medical image sequence of the first target tissue and the probability map predicted by the first teacher network are input into the trained pixel-level confidence map generation network to obtain the third confidence map;

[0015] The knowledge distillation loss is determined based on the third confidence plot, the probability plot of the student network prediction, and the probability plot of the first teacher network prediction.

[0016] The Ohem loss was determined based on student network prediction results and the labels of medical images.

[0017] The student network is trained using knowledge distillation loss and Ohem loss, and the trained student network is used as a medical image segmentation model.

[0018] Optionally, determining the total loss of the pixel-level confidence map generation network to train the pixel-level confidence map generation network based on the first confidence map, the second confidence map, and the total loss of the pixel-level confidence map generation network includes:

[0019] The total loss L of the pixel-level confidence map generation network is calculated using the following formula. G :

[0020]

[0021] Where a' is the pixel-wise loss L α The coefficient of ; b' is the similarity loss L β The coefficient; w is the width of the medical image; h is the height of the medical image; This represents all elements in the j-th column of the first confidence graph used to train the confidence generation network; This represents a matrix of all ones; n is the nth segment in the medical image; t is the tth frame of the nth segment in the medical image; represents all elements in the j-th column of a matrix of all ones; T represents the transpose of the matrix; This represents the confidence value at row i and column j of the second confidence plot; denoted by JS divergence value calculated from the probability graph and labels of the first teacher network; c represents the total number of channels. This represents the probability at pixel coordinate (i,j) on channel k of a label in one-hot encoding. Let represent the probability at pixel coordinate (i,j) on channel k of the probability graph predicted by the first teacher network.

[0022] Optionally, the step of inputting the images to be segmented from the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain the third confidence map includes:

[0023] Constructing a third confidence graph The expression:

[0024]

[0025] Where n is the nth segment in the medical image; t is the tth frame of the nth segment in the medical image; This indicates that a confidence-based network is used to process the superimposed images; Images that need to be segmented within a medical image sequence for the primary target tissue; This indicates an overlay operation along the channel dimension; This is a probability graph predicted by the first teacher network.

[0026] Optionally, determining the knowledge distillation loss based on the third confidence graph, the probability graph predicted by the student network, and the probability graph predicted by the first teacher network includes:

[0027] The knowledge distillation loss L is calculated using the following formula. soft :

[0028]

[0029] Among them, T d To represent the distillation temperature; w is the width of the medical image; h is the height of the medical image; D represents the value in the i-th row and j-th column of the third confidence plot used for student network training; KL (·) represents the KL divergence; This is the probability vector at pixel (i,j) of the probability map predicted by the first teacher network. C' represents the probability vector at pixel (i,j) of the probability graph predicted for the student network; C' represents the total number of probability value types. Let c' represent the class c' probability value at (i,j) in the probability graph predicted by the first teacher network. Let represent the c'-th class probability value at (i,j) in the probability graph of the student's network prediction.

[0030] Optionally, determining the Ohem loss based on the student network prediction results and the labels of the medical images includes:

[0031] Calculate the Ohem loss L using the following formula. hard :

[0032]

[0033] Where, N S Indicates the total number of elements considered in the loss calculation process; i is the i-th element involved in the calculation; I S (i) represents the cross-entropy of the i-th pixel involved in the calculation; L CE This is the matrix obtained by calculating the cross-entropy using the labels and student network segmentation results; Labels representing the one-hot encoded form of existing medical images; F seg For the predicted results of the student network; I CE For L CE Expanded vector form; τ dis For the preset threshold; I CE (n min ) indicates greater than τ dis The minimum cross-entropy loss value; I CE [:n min [] represents the vector formed by sorting from the maximum cross-entropy value to the minimum cross-entropy value.

[0034] Optionally, training the student network based on knowledge distillation loss and Ohem loss, and using the trained student network as a medical image segmentation model, includes:

[0035] Calculate the total loss L of the student network using the following formula. total :

[0036] L total =αL soft +βL hard ;

[0037] Where α is the knowledge distillation loss L soft The coefficient; β is the Ohem loss L hard The coefficient;

[0038] The total loss of student networks L total Train the student network and use the trained student network as a medical image segmentation model.

[0039] Secondly, the present invention provides a medical image segmentation model training system, comprising:

[0040] The pre-training module is used to pre-train the teacher network based on existing medical images, resulting in a pre-trained teacher network that serves as the first teacher network.

[0041] The inference module is used to infer the medical image sequence of the first target tissue using the first teacher network to obtain the probability map predicted by the first teacher network.

[0042] A conversion module is used to convert the labels of existing medical images into one-hot encoding;

[0043] The first generation module is used to take existing medical images and one-hot encoded labels as the first set of data, and the medical image of the first target tissue and the probability map predicted by the first teacher network as the second set of data, and simultaneously input them into the pixel-level confidence map generation network to obtain the first confidence map and the second confidence map respectively.

[0044] The first determining module is used to determine the total loss of the pixel-level confidence map generation network, so as to train the pixel-level confidence map generation network based on the first confidence map, the second confidence map, and the total loss of the pixel-level confidence map generation network.

[0045] The second generation module is used to input the images that need to be segmented from the medical image sequence of the first target tissue into the student network of dual-stream RGCN feature fusion to obtain the probability map and prediction results of the student network.

[0046] The third generation module is used to input the images that need to be segmented in the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain the third confidence map.

[0047] The second determination module is used to determine the knowledge distillation loss based on the third confidence graph, the probability graph predicted by the student network, and the probability graph predicted by the first teacher network.

[0048] The third determination module is used to determine the Ohem loss based on the student network prediction results and the labels of the medical images;

[0049] The training module is used to train the student network based on knowledge distillation loss and Ohem loss, and the trained student network is used as a medical image segmentation model.

[0050] Thirdly, the present invention provides a computer device including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the steps of the medical image segmentation model training method described in the first aspect.

[0051] Fourthly, the present invention provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, it implements the steps of the medical image segmentation model training method described in the first aspect.

[0052] Fifthly, the present invention provides a medical image segmentation method, comprising:

[0053] Acquire medical images of the second target tissue;

[0054] The medical image of the second target tissue is input into the medical image segmentation model to obtain the medical image segmentation result of the second target tissue output by the medical image segmentation model; wherein, the medical image segmentation model is trained according to the medical image segmentation model training method described in the first aspect.

[0055] This invention provides a training method, system, and approach for medical image segmentation models. The training method employs knowledge distillation with confidence, transferring the correct knowledge from a large model with good segmentation performance but slow segmentation speed to a smaller model with faster segmentation speed and RGCN feature fusion. This improves the segmentation performance of the smaller model while maintaining good segmentation efficiency. This invention avoids the negative impact of the teacher network on the student network during traditional knowledge distillation, thus achieving better knowledge distillation results. Attached Figure Description

[0056] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating a medical image segmentation model training method provided in an embodiment of the present invention;

[0058] Figure 2 A schematic diagram of the structure of multi-scale feature R-GCN fusion provided in an embodiment of the present invention;

[0059] Figure 3 A schematic diagram of the structure of spatial context feature R-GCN fusion provided in an embodiment of the present invention;

[0060] Figure 4The images show the results of surgical scene segmentation on public and internal datasets using different methods provided in the embodiments of the present invention.

[0061] Figure 5 This is a qualitative comparison chart of different methods provided in the embodiments of the present invention on Endovis18 data;

[0062] Figure 6 This is a qualitative comparison of different methods provided in the embodiments of the present invention on CholecSeg8K data.

[0063] Figure 7 The recognition result diagrams for each category provided in the embodiments of the present invention;

[0064] Figure 8 This is a schematic diagram of the structure of a medical image segmentation model training system provided in an embodiment of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] Example 1

[0067] like Figure 1 As shown, this embodiment provides a method for training a medical image segmentation model, including:

[0068] Step 101: Pre-train the teacher network based on existing medical images to obtain a pre-trained teacher network, which will serve as the first teacher network.

[0069] In this embodiment, the medical images can be obtained via laparoscopy.

[0070] Step 102: Use the first teacher network to infer the medical image sequence of the first target tissue to obtain the probability map predicted by the first teacher network.

[0071] In this embodiment, the probability map predicted by the first teacher network needs to be obtained before the softmax function in the first classroom network segmentation head.

[0072] Step 103: Convert the labels of existing medical images to one-hot encoding.

[0073] Step 104: Take the existing medical images and one-hot encoded labels as the first set of data, and the medical images of the first target tissue and the probability map predicted by the first teacher network as the second set of data, and simultaneously input them into the pixel-level confidence map generation network to obtain the first confidence map and the second confidence map respectively.

[0074] This embodiment employs a pixel-level confidence map generation network (PCG) to more accurately evaluate the credibility of the first teacher network's knowledge. Multi-scale features are extracted through multiple convolutional downsampling layers, combined with a self-attention mechanism to obtain global contextual information. During the upsampling stage, transposed convolutions are used to restore the feature map to its original resolution, while shallow features are connected to preserve local details. The final output confidence matrix achieves finer-grained discrimination compared to a simple probability map. To generate reliable confidence maps, this embodiment uses two sets of data as PCG training inputs. The first set includes existing medical images and one-hot encoded labels, outputting a confidence map with all 1s during training. To enhance stability, this embodiment designs a pixel-wise loss using mean squared error (MSE) to force PCG to generate results closer to all 1s.

[0075] Step 105: Determine the total loss of the pixel-level confidence map generation network, and train the pixel-level confidence map generation network based on the first confidence map, the second confidence map, and the total loss of the pixel-level confidence map generation network.

[0076] To improve the robustness of Pixel-Level Confidence Map Generation (PCG), it is desirable to accurately identify pixels under ideal conditions and effectively distinguish erroneous parts in complex, non-ideal environments, while avoiding excessively rapid model convergence and overfitting. By using JS divergence, the similarity between the output probability of the first teacher network and the one-hot label is calculated, resulting in a similarity matrix. The PCG network is updated by comparing this similarity matrix with the output results of the PCG network, enabling it to handle more complex scenarios.

[0077] For example, the total loss L of the pixel-level confidence map generation network is calculated according to the following formula. G :

[0078]

[0079] Where a' is the pixel-wise loss L α The coefficient of ; b' is the similarity loss L β The coefficient; w is the width of the medical image; h is the height of the medical image; This represents all elements in the j-th column of the first confidence graph used to train the confidence generation network; This represents a matrix of all ones; n is the nth segment in the medical image; t is the tth frame of the nth segment in the medical image; represents all elements in the j-th column of a matrix of all ones; T represents the transpose of the matrix; This represents the confidence value at row i and column j of the second confidence plot; denoted by JS divergence value calculated from the probability graph and labels of the first teacher network; c represents the total number of channels. This represents the probability at pixel coordinate (i,j) on channel k of a label in one-hot encoding. Let represent the probability at pixel coordinate (i,j) on channel k of the probability graph predicted by the first teacher network.

[0080] Step 106: Input the images that need to be segmented from the medical image sequence of the first target tissue into the student network of dual-stream RGCN feature fusion to obtain the probability map and prediction results of the student network.

[0081] This embodiment designs two feature fusion methods:

[0082] 1) Multi-scale feature R-GCN fusion

[0083] In multi-scale feature fusion, feature maps processed by dilated convolutions with three different dilation rates are subjected to a 3×3 convolution to project them to a lower dimension, denoted as graph nodes. Convolutions with smaller dilation rates correspond to lower receptive fields. The goal is for feature maps with low and high receptive fields to collectively provide effective information for feature maps with medium receptive fields, thereby enhancing detail representation and improving global perception. Therefore, edges are added between nodes to fuse information and form complementary features. This feature fusion framework is as follows: Figure 2 As shown, the formula is expressed as follows:

[0084]

[0085] in, The vector representing the updated node b; σ(·) is the activation function; c a W is the normalization factor for node a; a Here is the weight coefficient for node a; Represents nodes a and c before the update. c W is the normalization factor for node c; c The weight coefficient for node c; Represents node c before the update; W b Here is the weight coefficient for node b; This represents node b before the update.

[0086] 2) Spatial context feature R-GCN fusion

[0087] In the second feature extraction module, a sliding window and a self-attention module are used to capture the spatial relationships between pixels within the window. While large windows provide more pixel information, they significantly increase computational burden, thus affecting efficiency. Therefore, this embodiment designs four different sliding windows (1×1, 4×4, 8×8, and 16×16), performing self-attention calculations within each window to extract the relationships between pixels. However, this method fails to achieve cross-window information transfer. Although a large window may contain information from smaller windows, there is a lack of interaction between 16×16 windows; therefore, as... Figure 3 As shown, this embodiment introduces a second RGCN (G2) to fuse spatial features.

[0088] The fusion graph G2 of the second stream is represented as:

[0089]

[0090] in, This represents the weight of the node in the i-th row and j-th column. Indicates the normalization factor; W represents the node (i,j) before the update; i,j Let (i,j) be the weight of node (i,j). For node (i n ,j n The weight of ).

[0091] By stacking feature maps with multiple scales and spatial context together and passing them to the segmentation head for semantic segmentation, computation time was reduced and computational resources were even saved in ablation experiments.

[0092] Step 107: Input the images to be segmented from the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain the third confidence map.

[0093] In this step, for example, a third confidence graph is constructed. The expression:

[0094]

[0095] Where n is the nth segment in the medical image; t is the tth frame of the nth segment in the medical image; This indicates that a confidence-based network is used to process the superimposed images; Images that need to be segmented within a medical image sequence for the primary target tissue; This indicates an overlay operation along the channel dimension; This is a probability graph predicted by the first teacher network.

[0096] Step 108: Determine the knowledge distillation loss based on the third confidence graph, the probability graph predicted by the student network, and the probability graph predicted by the first teacher network.

[0097] In this step, for example, the knowledge distillation loss L is calculated according to the following formula. soft :

[0098]

[0099] Among them, T d To represent the distillation temperature; w is the width of the medical image; h is the height of the medical image; D represents the value in the i-th row and j-th column of the third confidence plot used for student network training; KL (·) represents the KL divergence; This is the probability vector at pixel (i,j) of the probability map predicted by the first teacher network. C' represents the probability vector at pixel (i,j) of the probability graph predicted for the student network; C' represents the total number of probability value types. Let c' represent the class c' probability value at (i,j) in the probability graph predicted by the first teacher network. Let represent the c'-th class probability value at (i,j) in the probability graph of the student's network prediction.

[0100] Step 109: Determine the Ohem loss based on the student network prediction results and the labels of the medical images.

[0101] This embodiment calculates the minimum number of pixels n that need to be included in the loss based on the image size and the number of downsampled pixels. min To improve supervised learning performance, the Ohem loss is used to guide the model to focus more on pixels with significant differences, while ignoring pixels whose results are closer to the true value. To achieve this, a variable threshold τ is preset. dis To measure the descending order of I CE The degree of difference

[0102] In this step, for example, the Ohem loss L is calculated according to the following formula. hard :

[0103]

[0104] Where, N S Indicates the total number of elements considered in the loss calculation process; i is the i-th element involved in the calculation; I S (i) represents the cross-entropy of the i-th pixel involved in the calculation; L CE This is the matrix obtained by calculating the cross-entropy using the labels and student network segmentation results; Labels representing the one-hot encoded form of existing medical images; Fseg For the predicted results of the student network; I CE For L CE Expanded vector form; τ dis For the preset threshold; I CE (n min ) indicates greater than τ dis The minimum cross-entropy loss value; I CE [:n min [] represents the vector formed by sorting from the maximum cross-entropy value to the minimum cross-entropy value.

[0105] Step 1010: Train the student network based on knowledge distillation loss and Ohem loss, and use the trained student network as a medical image segmentation model.

[0106] In this step, for example, the total loss L of the student network is calculated according to the following formula. total :

[0107] L total =αL soft +βL hard .

[0108] Where α is the knowledge distillation loss L soft The coefficient; β is the Ohem loss L hard The coefficient.

[0109] The total loss of student networks L total Train the student network and use the trained student network as a medical image segmentation model.

[0110] This embodiment uses the similarity coefficient (Dice) and the intersection of unions (IoU) as evaluation metrics for the accuracy of the proposed method. Formally, predictions can be categorized as true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). Therefore, the metrics Dice and IoU can be described as follows:

[0111]

[0112] Pixel-level segmentation accuracy comparison experiment: The method provided in this embodiment is compared with state-of-the-art (SOTA) methods based on different architectures, including the cumbersome teacher model STswinCL, NoisyLSTM, UNet3+, TransUNet, the transformer-based Swin, and the baseline DeeplabV3+. UNet3+ and DeepLabV3+ both employ various multi-scale feature extraction strategies. TransUNet and STswin utilize a Transformer-based architecture to manage long-range dependencies and global context. However, they may lose details during the feature fusion stage. The model provided in this embodiment integrates RGCN, further fusing multi-scale and cross-spatial information, achieving excellent coordination between global and local knowledge parsing and laparoscopic understanding.

[0113] like Figure 4 As shown, the PLDKD-Net proposed in this embodiment achieves the best average Dice and IoU on two public datasets (EndoVis2018 & CholecSeg8K) and one internal dataset, and has the smallest standard deviation of performance across different video segments, which means it has stronger generalization and the most stable performance. Specifically, in the four test segments of the EndoVis 2018 dataset, the average Dice for the 12-class segmentation task is 68.07%, 69.84%, 88.59%, and 48.25%, respectively. In the four test segments of the CholecSeg8K dataset, performing 13-class semantic segmentation, the average Dice is 74.83%, 96.62%, 97.46%, and 72.72%, respectively. In the dataset of this embodiment, despite the relatively small amount of data, the proposed method still achieves the best results.

[0114] Furthermore, to visually demonstrate the segmentation results, this embodiment presents qualitative comparison results on the EndoVis18 and Kaggle CholecSeg8K datasets. This embodiment uses different colors to represent various tissues and instruments in laparoscopic scenarios and compares them with baseline DeepLabV3+, STswin, and popular segmentation models UNet3+ and TransUnet. The model in this embodiment utilizes a discriminant module, which excels at correcting errors made by the teacher model, such as... Figure 5 and Figure 6 As shown. Overall, the output of the medical image segmentation model in this embodiment is also closer to the label.

[0115] Object-level segmentation performance comparison experiment: Considering the complex laparoscopic scenario with various instruments and tissues, the segmentation performance of the model for each object category was tested accordingly. The results are as follows: Figure 7As shown. Compared to methods based on different architectures, the method of this embodiment intuitively outperforms DeeplabV3+ and UNet3+ for most object classes. Although the teacher model provides superior performance based on its cumbersome and complex learning mechanism, the method of this embodiment can surpass the teacher's segmentation score through the ConfiKD module, which can identify the teacher's erroneous parts, thereby avoiding negative optimization. As highlighted, the model of this embodiment outperforms the teacher model when segmenting kidney parenchyma (“Kidney”) and covered kidney (“Covered”) on the Endovis 2018 dataset, and achieves higher IoU scores for “liver” and “blood” segmentation in CholecSeg8K, as well as when identifying instrument axis (“IS”) and suction instrument (“SI”) on the internal data of this embodiment. These extensive validations collectively demonstrate that the method of this embodiment can output satisfactory segmentation results for a wide variety of objects in complex environments.

[0116] In summary, this embodiment provides a method for training a medical image segmentation model. It employs knowledge distillation with confidence, transferring the correct knowledge from a large model with good segmentation performance but slow segmentation speed to a smaller model with faster segmentation speed and RGCN feature fusion. This improves the segmentation performance of the smaller model while maintaining good segmentation efficiency. This invention avoids the negative impact of the teacher network on the student network during traditional knowledge distillation, thus achieving better knowledge distillation results.

[0117] Example 2

[0118] Based on the same inventive concept as Embodiment 1, this embodiment also provides a medical image segmentation model training system. Since the principle of this system in solving the problem is similar to that of the aforementioned medical image segmentation model training method, the implementation of this system can refer to the implementation of the medical image segmentation model training method.

[0119] like Figure 8 As shown, the medical image segmentation model training system includes:

[0120] The pre-training module 10 is used to pre-train the teacher network based on existing medical images to obtain a pre-trained teacher network, which serves as the first teacher network.

[0121] The reasoning module 20 is used to reason about the medical image sequence of the first target tissue using the first teacher network to obtain the probability map predicted by the first teacher network.

[0122] The conversion module 30 is used to convert the labels of existing medical images into one-hot encoding.

[0123] The first generation module 40 is used to take existing medical images and one-hot encoded labels as the first set of data, and the medical image of the first target tissue and the probability map predicted by the first teacher network as the second set of data, and simultaneously input them into the pixel-level confidence map generation network to obtain the first confidence map and the second confidence map respectively.

[0124] The first determining module 50 is used to determine the total loss of the pixel-level confidence map generation network, so as to train the pixel-level confidence map generation network based on the first confidence map, the second confidence map, and the total loss of the pixel-level confidence map generation network.

[0125] The second generation module 60 is used to input the images that need to be segmented from the medical image sequence of the first target tissue into the student network of dual-stream RGCN feature fusion to obtain the probability map and prediction results of the student network.

[0126] The third generation module 70 is used to input the images to be segmented in the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain the third confidence map.

[0127] The second determination module 80 is used to determine the knowledge distillation loss based on the third confidence graph, the probability graph predicted by the student network, and the probability graph predicted by the first teacher network.

[0128] The third determination module 90 is used to determine the Ohem loss based on the student network prediction results and the labels of the medical images.

[0129] Training module 100 is used to train the student network based on knowledge distillation loss and Ohem loss, and the trained student network is used as a medical image segmentation model.

[0130] For more detailed information on the working process of each of the above modules, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0131] Example 3

[0132] This embodiment provides a medical image segmentation method, including:

[0133] Acquire medical images of the second target tissue.

[0134] The medical image of the second target tissue is input into the medical image segmentation model to obtain the medical image segmentation result of the second target tissue output by the medical image segmentation model; wherein, the medical image segmentation model is trained according to the medical image segmentation model training method described in Example 1.

[0135] Example 4

[0136] This embodiment provides a computer device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the steps of the medical image segmentation model training method described in Embodiment 1.

[0137] For a more detailed explanation of the above method, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0138] Example 5

[0139] This embodiment provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, it implements the steps of the medical image segmentation model training method described in Embodiment 1.

[0140] For a more detailed explanation of the above method, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0141] Example 6

[0142] This embodiment provides a computer program product, including computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, they implement the steps of the medical image segmentation model training method described in Embodiment 1.

[0143] For a more detailed explanation of the above method, please refer to the relevant content disclosed in Example 1, which will not be repeated here.

[0144] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems, devices, storage media, and computer program products disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0145] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0146] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0147] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0148] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0149] The present invention has been described in detail above with reference to specific embodiments and exemplary examples; however, these descriptions should not be construed as limiting the present invention. Those skilled in the art will understand that various equivalent substitutions, modifications, or improvements can be made to the technical solutions and embodiments of the present invention without departing from the spirit and scope of the invention, and all such modifications and improvements fall within the scope of the present invention. The scope of protection of the present invention is defined by the appended claims.

Claims

1. A method for training a medical image segmentation model, characterized in that, include: The teacher network is pre-trained based on existing medical images to obtain a pre-trained teacher network, which is then used as the first teacher network. The first teacher network is used to infer the medical image sequence of the first target tissue, and the probability map predicted by the first teacher network is obtained. Convert the labels of existing medical images to one-hot encoding; Existing medical images and one-hot encoded labels are used as the first set of data, and medical images of the first target tissue and probability maps predicted by the first teacher network are used as the second set of data. These are simultaneously input into a pixel-level confidence map generation network to obtain the first confidence map and the second confidence map, respectively. Determine the total loss of the pixel-level confidence map generation network, and train the pixel-level confidence map generation network based on the first confidence map, the second confidence map, and the total loss of the pixel-level confidence map generation network; The images that need to be segmented in the medical image sequence of the first target tissue are input into the student network of dual-stream RGCN feature fusion to obtain the probability map and prediction results of the student network. The images to be segmented from the medical image sequence of the first target tissue and the probability map predicted by the first teacher network are input into the trained pixel-level confidence map generation network to obtain the third confidence map; The knowledge distillation loss is determined based on the third confidence plot, the probability plot of the student network prediction, and the probability plot of the first teacher network prediction. The knowledge distillation loss L is calculated using the following formula. soft : ； Among them, T d To represent the distillation temperature; w is the width of the medical image; h is the height of the medical image; D represents the value in the i-th row and j-th column of the third confidence plot used for student network training; KL (·) represents the KL divergence; This is the probability vector at pixel (i,j) of the probability map predicted by the first teacher network. C' represents the probability vector at pixel (i,j) of the probability graph predicted for the student network; C' represents the total number of probability value types. Let c' represent the class c' probability value at (i,j) in the probability graph predicted by the first teacher network. Let c' be the probability value of the class c at (i,j) in the probability graph of the student network prediction. The Ohem loss was determined based on student network prediction results and the labels of medical images. The student network is trained using knowledge distillation loss and Ohem loss, and the trained student network is used as a medical image segmentation model.

2. The medical image segmentation model training method according to claim 1, characterized in that, The process of determining the total loss of the pixel-level confidence map generation network, and training the pixel-level confidence map generation network based on the first confidence map, the second confidence map, and the total loss of the pixel-level confidence map generation network, includes: The total loss L of the pixel-level confidence map generation network is calculated using the following formula. G : ； Where a' is the pixel-wise loss L α The coefficient of ; b' is the similarity loss L β The coefficients; w is the width of the medical image; h is the height of the medical image; represents all elements in the j-th column of the first confidence graph used to train the confidence generation network; This represents a matrix of all ones; n is the nth segment in the medical image; t is the tth frame of the nth segment in the medical image; represents all elements in the j-th column of a matrix of all ones; T represents the transpose of the matrix; This represents the confidence value at row i and column j of the second confidence plot; denoted by JS divergence value calculated from the probability graph and labels of the first teacher network; c represents the total number of channels. This represents the probability at pixel coordinate (i,j) on channel k of a label in one-hot encoding. Let represent the probability at pixel coordinate (i,j) on channel k of the probability graph predicted by the first teacher network.

3. The medical image segmentation model training method according to claim 1, characterized in that, The step of inputting the images to be segmented from the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain the third confidence map includes: Constructing a third confidence graph The expression: ； Where n is the nth segment in the medical image; t is the tth frame of the nth segment in the medical image; This indicates that a confidence-based network is used to process the superimposed images; Images that need to be segmented in a medical image sequence for the first target tissue; ⊕ indicates a stacking operation along the channel dimension; This is a probability graph predicted by the first teacher network.

4. The medical image segmentation model training method according to claim 1, characterized in that, The determination of Ohem loss based on student network prediction results and medical image labels includes: Calculate Ohem loss using the following formula. : ； Where, N S Indicates the total number of elements considered in the loss calculation process; i is the i-th element involved in the calculation; I S (i) represents the cross-entropy of the i-th pixel involved in the calculation; L CE This is the matrix obtained by calculating the cross-entropy using the labels and student network segmentation results; Labels representing the one-hot encoded form of existing medical images; F seg For the predicted results of the student network; I CE For L CE Expanded vector form; τ dis The preset threshold; Indicates greater than τ dis The minimum cross-entropy loss value; I CE [:n min [] represents the vector formed by sorting from the maximum cross-entropy value to the minimum cross-entropy value.

5. The medical image segmentation model training method according to claim 1, characterized in that, The process of training a student network based on knowledge distillation loss and Ohem loss, and then using the trained student network as a medical image segmentation model, includes: Calculate the total loss L of the student network using the following formula. total : ； Where α is the knowledge distillation loss L soft The coefficient; β is the Ohem loss. The coefficient; The total loss of student networks L total Train the student network and use the trained student network as a medical image segmentation model.

6. A medical image segmentation model training system, characterized in that, include: The pre-training module is used to pre-train the teacher network based on existing medical images, resulting in a pre-trained teacher network that serves as the first teacher network. The inference module is used to infer the medical image sequence of the first target tissue using the first teacher network to obtain the probability map predicted by the first teacher network. A conversion module is used to convert the labels of existing medical images into one-hot encoding; The first generation module is used to take existing medical images and one-hot encoded labels as the first set of data, and the medical image of the first target tissue and the probability map predicted by the first teacher network as the second set of data, and simultaneously input them into the pixel-level confidence map generation network to obtain the first confidence map and the second confidence map respectively. The first determining module is used to determine the total loss of the pixel-level confidence map generation network, so as to train the pixel-level confidence map generation network based on the first confidence map, the second confidence map, and the total loss of the pixel-level confidence map generation network. The second generation module is used to input the images that need to be segmented from the medical image sequence of the first target tissue into the student network of dual-stream RGCN feature fusion to obtain the probability map and prediction results of the student network. The third generation module is used to input the images that need to be segmented in the medical image sequence of the first target tissue and the probability map predicted by the first teacher network into the trained pixel-level confidence map generation network to obtain the third confidence map. The second determination module is used to determine the knowledge distillation loss based on the third confidence graph, the probability graph of the student network prediction, and the probability graph of the first teacher network prediction, and calculates the knowledge distillation loss L according to the following formula. soft : ； Among them, T d To represent the distillation temperature; w is the width of the medical image; h is the height of the medical image; D represents the value in the i-th row and j-th column of the third confidence plot used for student network training; KL (·) represents the KL divergence; This is the probability vector at pixel (i,j) of the probability map predicted by the first teacher network. C' represents the probability vector at pixel (i,j) of the probability graph predicted for the student network; C' represents the total number of probability value types. Let c' represent the class c' probability value at (i,j) in the probability graph predicted by the first teacher network. Let c' be the probability value of the class c at (i,j) in the probability graph of the student network prediction. The third determination module is used to determine the Ohem loss based on the student network prediction results and the labels of the medical images; The training module is used to train the student network based on knowledge distillation loss and Ohem loss, and the trained student network is used as a medical image segmentation model.

7. A computer device, characterized in that, It includes a processor and a memory; wherein, when the processor executes the computer program stored in the memory, it implements the steps of the medical image segmentation model training method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, Used to store computer programs; when executed by a processor, the computer programs implement the steps of the medical image segmentation model training method according to any one of claims 1-5.

9. A medical image segmentation method, characterized in that, include: Acquire medical images of the second target tissue; The medical image of the second target tissue is input into the medical image segmentation model to obtain the medical image segmentation result of the second target tissue output by the medical image segmentation model; wherein, the medical image segmentation model is trained according to the medical image segmentation model training method according to any one of claims 1-5.

Images