Preoperative image segmentation method and system based on hybrid supervision enhancement

Abstract

The invention relates to the technical field of medical image processing, in particular to a preoperative image segmentation method and system based on hybrid supervision enhancement, and the method comprises the following steps: constructing an organ tissue segmentation model by using a Unet network, and constructing a vascular tissue segmentation model by using the Unet network; setting a supervision enhancement module, and combining the organ tissue segmentation model and the vascular tissue segmentation model into a teacher model; a student model is constructed by using an FCN network, and a preoperative image segmentation model is obtained through mixed supervised learning of a teacher model and the student model. For preoperative three-dimensional medical image segmentation, a mixed supervision segmentation model is provided, a supervision enhancement mode is adopted when a teacher model is constructed, it can be guaranteed that the teacher model accurately segments each organization, and a lightweight model structure is obtained based on a teacher-student multi-fine-grained feature learning strategy; and a student model capable of precisely segmenting each organization can be realized, so that the segmentation precision and the segmentation efficiency coexist.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a preoperative image segmentation method and system based on hybrid supervised enhancement. Background Art

[0002] During laparoscopic surgery, doctors operating surgical robots use images of organs transmitted to monitors to understand the real-time surgical scene and guide the use of surgical instruments. Although the images returned by the monitor can accurately reflect the condition of the organs being operated on, the limited field of view of the endoscope and obstruction caused by overlapping tissues make it difficult to understand the internal conditions of the tissues in the obscured space.

[0003] At present, in order to enable doctors to accurately perceive the internal morphology of organs and tissues during laparoscopic surgery, the three-dimensional organ model constructed before the operation is usually holographically projected onto the three-dimensional organ and blood vessel model established in real time during the operation through alignment. This allows doctors to intuitively grasp the tissue situation in the organ-occluded space during the operation, achieve a high sense of presence during the operation, and enhance the doctor's perception of the organ and blood vessel tissue during the operation.

[0004] Among them, the existing technology constructs a preoperative organ and vascular three-dimensional model based on preoperative images. It is necessary to segment the organ tissue, vascular tissue, fat tissue and other part information in the preoperative image, and then reconstruct it into a three-dimensional model based on the segmented tissue information. The current preoperative image segmentation of vascular tissue and organ tissue mostly uses a segmentation model to uniformly segment each tissue. In order to ensure accurate segmentation of each tissue, the segmentation model will inevitably cause the segmentation model structure to be complex, thereby reducing the segmentation efficiency. If the segmentation model has to sacrifice the segmentation accuracy of some tissues in order to ensure segmentation efficiency, the segmentation of each tissue can only achieve overall optimization, and it is difficult to ensure that each tissue segmentation achieves the most accurate segmentation. Therefore, the segmentation accuracy and segmentation efficiency cannot coexist. Summary of the Invention

[0005] The purpose of the present invention is to provide a preoperative image segmentation method and system based on hybrid supervised enhancement to solve the technical problem that the segmentation accuracy and segmentation efficiency of the preoperative image segmentation model constructed in the prior art cannot coexist.

[0006] In order to solve the above technical problems, the present invention specifically provides the following technical solutions:

[0007] A preoperative image segmentation method based on hybrid supervised enhancement includes the following steps:

[0008] Using the Unet network to construct an organ tissue segmentation model for segmenting organ tissue in preoperative images, and using the Unet network to construct a vascular tissue segmentation model for segmenting vascular tissue in preoperative images;

[0009] A supervision enhancement module is set between the organ tissue segmentation model and the vascular tissue segmentation model, and the organ tissue segmentation model and the vascular tissue segmentation model are combined into a teacher model;

[0010] The FCN network is used to construct a student model, and through mixed supervised learning of the teacher model and the student model, a preoperative image segmentation model for segmenting vascular tissue and organ tissue in preoperative images is obtained.

[0011] As a preferred embodiment of the present invention, the method for constructing the organ tissue segmentation model includes:

[0012] Acquire multiple preoperative images and mark the organ tissue masks in each preoperative image;

[0013] Combining each preoperative image and the organ tissue mask in each preoperative image into a dataset, and dividing the dataset into a training set and a test set;

[0014] Based on the training set, the preoperative image is used as the input of the Unet network, the organ tissue mask in the preoperative image is used as the output of the Unet network, and the Unet network is trained to obtain the organ tissue segmentation model;

[0015] Based on the test set, performing a performance evaluation on the organ tissue segmentation effect of the organ tissue segmentation model;

[0016] The organ tissue segmentation model is:

[0017] mask organ =Unet(G);

[0018] Where, Mask organ is the organ tissue mask output by the organ tissue segmentation model, G is the preoperative image, and Unet is the Unet network;

[0019] The loss function of the organ tissue segmentation model is:

[0020]

[0021] Where, L organ is the loss function value of the organ tissue segmentation model, mask organ,i is the organ tissue mask output by the organ tissue segmentation model for the i-th preoperative image in the dataset, mask organ,i (real) is the labeled value of the organ tissue mask in the i-th preoperative image in the dataset, is the L2 norm, and n is the total number of preoperative images in the dataset.

[0022] As a preferred embodiment of the present invention, the method for constructing the vascular tissue segmentation model includes:

[0023] Acquire multiple preoperative images and mark the vascular tissue mask in each preoperative image;

[0024] Combining each preoperative image and the vascular tissue mask in each preoperative image into a dataset, and dividing the dataset into a training set and a test set;

[0025] Based on the training set, the preoperative image is used as the input of the Unet network, and the vascular tissue mask in the preoperative image is used as the output of the Unet network to train the Unet network to obtain the vascular tissue segmentation model;

[0026] Based on the test set, a performance evaluation is performed on the vascular tissue segmentation effect of the vascular tissue segmentation model;

[0027] The vascular tissue segmentation model is:

[0028] mask vessel =Unet(G);

[0029] Where, mask vessel is the vascular tissue mask output by the vascular tissue segmentation model, G is the preoperative image, and Unet is the Unet network;

[0030] The loss function of the vascular tissue segmentation model is:

[0031]

[0032] Where, L vessel is the loss function value of the vascular tissue segmentation model, mask vessel,i is the vascular tissue mask output by the vascular tissue segmentation model for the i-th preoperative image in the dataset, mask vessel,i (real) is the labeled value of the vascular tissue mask in the i-th preoperative image in the dataset, is the L2 norm, and n is the total number of preoperative images in the dataset.

[0033] As a preferred solution of the present invention, the supervision enhancement module includes: In the formula, F1 is the first objective of supervision enhancement, min is the minimum operator, and L vessel (t) is the loss function value obtained at the tth training iteration of the vascular tissue segmentation model, L organ (t) is the loss function value obtained at the tth training iteration of the organ tissue segmentation model, and T is the total number of iterations of the training model;

[0034]

[0035] In the formula, F2 is the second target of supervision enhancement, min is the minimum operator, and mask vessel,i (T) is the vascular tissue mask output by the vascular tissue segmentation model for the i-th preoperative image in the segmentation dataset at the T-th training iteration, mask organ,i (t) is the organ tissue mask output by the vascular tissue segmentation model for the i-th preoperative image in the segmentation dataset at the T-th training iteration, mask mix,i (real) is the combined mask annotation value of the i-th preoperative image in the dataset, and n is the total number of preoperative images in the dataset.

[0036] As a preferred embodiment of the present invention, the method for constructing the teacher model includes:

[0037] The output ends of the vascular tissue segmentation model and the organ tissue segmentation model after T training iterations are connected through a superposition module to obtain the teacher model;

[0038] The teacher model is: mask g,mix =mask vessel (T)+mask organ (T);

[0039] Where, mask g,mix The combined mask of vascular tissue and organ tissue output by the teacher model, mask vessel (T) is the vascular tissue mask output by the vascular tissue segmentation model after T training iterations, mask organ (T) is the organ tissue mask output by the organ tissue segmentation model after T training iterations.

[0040] As a preferred embodiment of the present invention, the method for constructing the student model includes:

[0041] Acquire a plurality of preoperative images with combined masks labeled with vascular tissue and organ tissue, and acquire a preoperative image without the combined masks labeled, and output the combined masks of the preoperative images without the combined masks labeled through a teacher model;

[0042] The preoperative images and the combined masks in the preoperative images are combined into a dataset, and the dataset is divided into a training set and a test set;

[0043] Based on the training set, the preoperative image is used as the input of the FCN network, the combined mask in the preoperative image is used as the output of the Unet network, and the Unet network is trained to obtain the student model;

[0044] Based on the test set, the performance of the student model for vascular tissue and organ tissue segmentation is evaluated;

[0045] The student model is:

[0046] mask s,mix =Unet(G);

[0047] Where, mask s,mix is the combined mask output by the student model, G is the preoperative image, and Unet is the Unet network;

[0048] The loss function of the student model includes the distillation loss L KL and segmentation loss L CE ,in:

[0049]

[0050] L total =αL KL +(1―α)L CE ;

[0051] Where, L total is the total loss, L KL is the distillation loss, L CE is the segmentation loss, mask s,mix,j is the combined mask output by the student model for the jth preoperative image in the dataset that is not labeled with the combined mask, mask g,mix,j is the combined mask output by the teacher model for the jth preoperative image in the dataset that is not labeled with the combined mask, m1 is the total number of preoperative images in the dataset that are not labeled with the combined mask, and mask s,mix,k is the combined mask output by the student model for the kth preoperative image in the dataset that has been labeled with the combined mask, mask mix,k (real) is the combined mask annotation value of the kth preoperative image in the dataset that has been annotated with the combined mask, m2 is the total number of preoperative images in the dataset that have been annotated with the combined mask, α is a hyperparameter, and Both are L2 norm forms.

[0052] As a preferred embodiment of the present invention, the method for determining the preoperative image segmentation model includes:

[0053] The model parameters are iteratively transferred and updated between the student model and the teacher model through the exponential moving average EMA method;

[0054] The student model after the model parameters are iteratively transferred and updated is used as the preoperative image segmentation model.

[0055] As a preferred solution of the present invention, all preoperative images constituting the data set are normalized.

[0056] As a preferred solution of the present invention, the annotated value of the vascular tissue mask corresponds to the true value of the vascular tissue mask, the annotated value of the organ tissue mask corresponds to the true value of the organ tissue mask, and the annotated value of the combination mask corresponds to the true value of the combination mask.

[0057] As a preferred embodiment of the present invention, the present invention provides a preoperative image segmentation system based on hybrid supervised enhancement, which is applied to a preoperative image segmentation method based on hybrid supervised enhancement. The system includes:

[0058] a segmentation model construction unit, configured to construct an organ tissue segmentation model for segmenting organ tissues in preoperative images using a Unet network, and to construct a vascular tissue segmentation model for segmenting vascular tissues in preoperative images using a Unet network;

[0059] a teacher model construction unit, configured to set a supervision enhancement module between the organ tissue segmentation model and the vascular tissue segmentation model, and combine the organ tissue segmentation model and the vascular tissue segmentation model into a teacher model;

[0060] The student model construction unit is used to construct a student model using the FCN network, and obtain a preoperative image segmentation model for segmenting vascular tissue and organ tissue in preoperative images through mixed supervised learning of the teacher model and the student model.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] The present invention proposes a hybrid supervised segmentation model for preoperative three-dimensional medical image segmentation. When constructing the teacher model, a supervised enhancement method is adopted to ensure that the teacher model can accurately segment each tissue. Based on the "teacher-student" multi-fine-grained feature learning strategy, a lightweight model structure is obtained, and a student model that can accurately segment each tissue is realized, so that segmentation accuracy and efficiency coexist. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.

[0064] Figure 1 A flow chart of a preoperative image segmentation method based on hybrid supervised enhancement provided by an embodiment of the present invention;

[0065] Figure 2 A block diagram of a preoperative image segmentation system based on hybrid supervised enhancement provided by an embodiment of the present invention;

[0066] Figure 3 A schematic diagram of the structure of the teacher-student model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0068] like Figure 1 and Figure 3 As shown, the present invention provides a preoperative image segmentation method based on hybrid supervised enhancement, comprising the following steps:

[0069] Using the Unet network to construct an organ tissue segmentation model for segmenting organ tissue in preoperative images, and using the Unet network to construct a vascular tissue segmentation model for segmenting vascular tissue in preoperative images;

[0070] A supervision enhancement module is set between the organ tissue segmentation model and the vascular tissue segmentation model, and the organ tissue segmentation model and the vascular tissue segmentation model are combined into a teacher model;

[0071] The FCN network is used to construct a student model, and through mixed supervised learning of the teacher model and the student model, a preoperative image segmentation model for segmenting vascular tissue and organ tissue in preoperative images is obtained.

[0072] The present invention utilizes the algorithm framework of knowledge distillation to construct a multi-fine-grained learning strategy of the teacher-student model to obtain a lightweight model for accurately segmenting vascular tissue and organ tissue. The lightweight model can run quickly under limited hardware resources by reducing model parameters and computational complexity, significantly improving computational efficiency, thereby ensuring both segmentation efficiency and segmentation accuracy. Moreover, the lightweight model has a small memory footprint and can be easily deployed on mobile devices or embedded systems. It is suitable for a variety of devices and application scenarios. Therefore, the publishing and deployment cost of the lightweight model is low, and it can be more easily deployed on various platforms, making it more practical.

[0073] The teacher-student model is a hybrid supervised learning approach, a machine learning method that combines supervised and unsupervised learning. In this approach, the teacher model is trained on labeled data, while the student model, a relatively simple model structure to be trained, mimics the teacher model's behavior to learn, thereby imitating the teacher model's ability to segment tissue based on preoperative images.

[0074] In order to ensure that the teacher model has a good guiding ability for the student model, the present invention first ensures that the segmentation performance of the teacher model is optimized when constructing the training teacher model. That is, the teacher model constructed by the present invention has the performance of accurately segmenting various tissues (vascular tissue, organ tissue, and other tissues such as fat tissue. The present invention takes vascular tissue and organ tissue as an example) in preoperative images. Therefore, when constructing the teacher model, the present invention first establishes a single segmentation model for each tissue, such as a vascular tissue segmentation model and an organ tissue segmentation model, which can each achieve separate and accurate segmentation of vascular tissue and organ tissue. Then, the single segmentation models of each tissue are combined to form a teacher model, so that the teacher model can integrate the performance of each single segmentation model. Due to the independence of the segmentation process of each tissue, there will be no mutual interference between the segmentations of each tissue in the teacher model, and accurate segmentation of each tissue can be achieved in the preoperative image, thereby ensuring that the teacher model can obtain the segmentation results of each tissue combination in the preoperative image (that is, the combined mask of vascular tissue and organ tissue), and the segmentation accuracy of each individual tissue in the combined mask can be maximized (both vascular tissue and organ tissue are most accurately segmented).

[0075] Specifically, the present invention is implemented by a supervision enhancement module when combining the single segmentation models of various tissues. The supervision enhancement module can supervise the training process of each single segmentation model with two supervision objectives, wherein the first supervision objective is: That is, during each iterative training process of each single segmentation model, the minimum segmentation loss can be maintained for training. Therefore, using this supervision objective to supervise the training of each single segmentation model can ensure that each single segmentation model has the most accurate performance in the segmentation of its own tissue after training, that is, it can be ensured that the combined mask has the most accurate segmentation of vascular tissue and organ tissue. This supervision objective can enhance the segmentation accuracy of the teacher model on each tissue.

[0076] The second monitoring objective is to: That is, the minimum error is maintained between the combined mask formed by the single tissue masks (vascular tissue masks and organ tissue masks) output after the iterative training of each single segmentation model is completed and the true value of the combined mask. Therefore, using this supervision target to supervise the training of each single segmentation model can ensure that there is no segmentation deviation between the outputs of each single segmentation model after the training is completed, and it has segmentation synchronization and can be reconstructed into a combined mask closest to the true value. It can ensure that the teacher model finally outputs the combined mask by mask fusion of the most accurate segmentation results of each single segmentation model. In other words, the teacher model outputs the high-precision vascular tissue mask obtained by the vascular tissue segmentation model and the high-precision organ tissue obtained by the organ tissue segmentation model, and will not output the high-precision vascular tissue mask obtained by the vascular tissue segmentation model and the low-precision organ tissue obtained by the organ tissue segmentation model to form a combined mask, nor will it output the low-precision vascular tissue mask obtained by the vascular tissue segmentation model and the high-precision organ tissue obtained by the organ tissue segmentation model to form a combined mask.

[0077] In summary, the supervision enhancement module adopted in the present invention can ensure that each single-tissue segmentation model has high-precision performance after training through the first supervision goal, while the second supervision goal is to ensure the synchronous combination of each single-tissue segmentation model, and to ensure the high-precision performance combination between each single-tissue segmentation model, and finally ensure that the teacher model can output a combined mask that integrates the precise segmentation results of each tissue. The teacher model integrates the optimization of the local segmentation performance of a single tissue to form the optimization of the overall segmentation performance of all tissues, and establishes the possibility of guiding the student model.

[0078] After establishing a teacher model with the best overall segmentation performance for all tissues, the teacher model is used to guide the training of the student model, enabling the student model to obtain accurate segmentation results for all tissues in preoperative images with a simple and lightweight structure.

[0079] In order to ensure that the teacher model has good guidance capabilities for the student model, the present invention first ensures that the segmentation performance of the teacher model is optimized when constructing and training the teacher model. That is, the teacher model constructed by the present invention has the performance of accurately segmenting various tissues (vascular tissue, organ tissue, and other tissues such as adipose tissue. The present invention takes vascular tissue and organ tissue as examples) in preoperative images. The specific performance is as follows:

[0080] The methods for constructing organ tissue segmentation models include:

[0081] Acquire multiple preoperative images and mark the organ tissue masks in each preoperative image;

[0082] Combining each preoperative image and the organ tissue mask in each preoperative image into a dataset, and dividing the dataset into a training set and a test set;

[0083] Based on the training set, the preoperative image is used as the input of the Unet network, and the organ tissue mask in the preoperative image is used as the output of the Unet network to train the Unet network and obtain the organ tissue segmentation model;

[0084] Based on the test set, the organ and tissue segmentation effect of the organ and tissue segmentation model is evaluated;

[0085] The organ tissue segmentation model is:

[0086] mask organ =Unet(G);

[0087] Where, mask organ is the organ tissue mask output by the organ tissue segmentation model, G is the preoperative image, and Unet is the Unet network;

[0088] The loss function of the organ tissue segmentation model is:

[0089]

[0090] Where, L organ is the loss function value of the organ tissue segmentation model, mask organ,i is the organ tissue mask output by the organ tissue segmentation model for the i-th preoperative image in the dataset, mask organ,i (real) is the labeled value of the organ tissue mask in the i-th preoperative image in the dataset, is the L2 norm, and n is the total number of preoperative images in the dataset.

[0091] The method for constructing the vascular tissue segmentation model includes:

[0092] Acquire multiple preoperative images and mark the vascular tissue mask in each preoperative image;

[0093] Combining each preoperative image and the vascular tissue mask in each preoperative image into a dataset, and dividing the dataset into a training set and a test set;

[0094] Based on the training set, the preoperative image is used as the input of the Unet network, and the vascular tissue mask in the preoperative image is used as the output of the Unet network to train the Unet network and obtain the vascular tissue segmentation model;

[0095] Based on the test set, the performance of the vascular tissue segmentation model is evaluated;

[0096] The vascular tissue segmentation model is:

[0097] mask vessel =Unet(G);

[0098] Where, mask vessel is the vascular tissue mask output by the vascular tissue segmentation model, G is the preoperative image, and Unet is the Unet network;

[0099] The loss function of the vascular tissue segmentation model is:

[0100]

[0101] Where, L vessel is the loss function value of the vascular tissue segmentation model, mask vessel,i is the vascular tissue mask output by the vascular tissue segmentation model for the i-th preoperative image in the dataset, mask vessel,i (real) is the labeled value of the vascular tissue mask in the i-th preoperative image in the dataset, is the L2 norm, and n is the total number of preoperative images in the dataset.

[0102] The supervision enhancement module includes: In the formula, F1 is the first objective of supervision enhancement, min is the minimum operator, and L vesse (t) is the loss function value obtained at the tth training iteration of the vascular tissue segmentation model, L organ (t) is the loss function value obtained at the tth training iteration of the organ tissue segmentation model, and T is the total number of iterations of the training model.

[0103] The first monitoring goal of the present invention is: That is, during each iterative training process of each single segmentation model, the minimum segmentation loss can be maintained for training. Therefore, using this supervision objective to supervise the training of each single segmentation model can ensure that each single segmentation model has the most accurate performance in the segmentation of its own tissue after training, that is, it can be ensured that the combined mask has the most accurate segmentation of vascular tissue and organ tissue. This supervision objective can enhance the segmentation accuracy of the teacher model on each tissue.

[0104]

[0105] In the formula, F2 is the second target of supervision enhancement, min is the minimum operator, and mask vessel,i (T) is the vascular tissue mask output by the vascular tissue segmentation model for the i-th preoperative image in the segmentation dataset at the T-th training iteration, mask organ,i (T) is the organ tissue mask output by the vascular tissue segmentation model for the i-th preoperative image in the segmentation dataset at the T-th training iteration, mask mix,i(real) is the combined mask annotation value of the i-th preoperative image in the dataset, and n is the total number of preoperative images in the dataset.

[0106] The second monitoring objective is to: That is, the minimum error is maintained between the combined mask formed by the single tissue masks (vascular tissue masks and organ tissue masks) output after the iterative training of each single segmentation model is completed and the true value of the combined mask. Therefore, using this supervision target to supervise the training of each single segmentation model can ensure that there is no segmentation deviation between the outputs of each single segmentation model after the training is completed, and it has segmentation synchronization and can be reconstructed into a combined mask closest to the true value. It can ensure that the teacher model finally outputs the combined mask by mask fusion of the most accurate segmentation results of each single segmentation model. In other words, the teacher model outputs the high-precision vascular tissue mask obtained by the vascular tissue segmentation model and the high-precision organ tissue obtained by the organ tissue segmentation model, and will not output the high-precision vascular tissue mask obtained by the vascular tissue segmentation model and the low-precision organ tissue obtained by the organ tissue segmentation model to form a combined mask, nor will it output the low-precision vascular tissue mask obtained by the vascular tissue segmentation model and the high-precision organ tissue obtained by the organ tissue segmentation model to form a combined mask.

[0107] In summary, the supervision enhancement module adopted in the present invention can ensure that each single-tissue segmentation model has high-precision performance after training through the first supervision goal, while the second supervision goal is to ensure the synchronous combination of each single-tissue segmentation model, and to ensure the high-precision performance combination between each single-tissue segmentation model, and finally ensure that the teacher model can output a combined mask that integrates the precise segmentation results of each tissue. The teacher model integrates the optimization of the local segmentation performance of a single tissue to form the optimization of the overall segmentation performance of all tissues, and establishes the possibility of guiding the student model.

[0108] The construction methods of the teacher model include:

[0109] The output ends of the vascular tissue segmentation model and the organ tissue segmentation model after T training iterations are connected through a superposition module to obtain a teacher model;

[0110] The teacher model is: mask g,mix =mask vessel (T)+mask organ (T);

[0111] Where, mask g,mix The combined mask of vascular tissue and organ tissue output by the teacher model, mask vessel (T) is the vascular tissue mask output by the vascular tissue segmentation model after T training iterations, mask organ(T) is the organ tissue mask output by the organ tissue segmentation model after T training iterations.

[0112] After establishing a teacher model that optimizes the overall segmentation performance of all tissues, the teacher model is used to guide the training of the student model, enabling the student model to obtain accurate segmentation results for all tissues in preoperative images with a simple and lightweight structure. The details are as follows:

[0113] The methods for constructing the student model include:

[0114] Acquire multiple preoperative images with combined masks of labeled vascular tissues and organ tissues, and acquire preoperative images without labeled combined masks, and output the combined masks of the preoperative images without labeled combined masks through a teacher model;

[0115] The preoperative images and the combined masks in the preoperative images are combined into a dataset, and the dataset is divided into a training set and a test set;

[0116] Based on the training set, the preoperative image is used as the input of the FCN network, and the combined mask in the preoperative image is used as the output of the Unet network to train the Unet network and obtain the student model;

[0117] Based on the test set, the performance of the student model's vascular tissue and organ tissue segmentation effects is evaluated;

[0118] The student model is:

[0119] mask s,mix =Unet(G);

[0120] Where, mask s,mix is the combined mask output by the student model, G is the preoperative image, and Unet is the Unet network.

[0121] When the teacher model is used to guide the student model, the present invention adopts a loss function common to the teacher-student model, the distillation loss L KL and segmentation loss L CE , as follows:

[0122] The loss function of the student model includes the distillation loss L KL and segmentation loss L CE ,in:

[0123]

[0124] L total =αL KL +(1―α)L CE ;

[0125] Where, L total is the total loss, LKL is the distillation loss, L CE is the segmentation loss, mask s,mix,j is the combined mask output by the student model for the jth preoperative image in the dataset that has not been labeled with a combined mask, mask g,mix,j is the combined mask output by the teacher model for the jth preoperative image in the dataset that is not labeled with the combined mask, m1 is the total number of preoperative images in the dataset that are not labeled with the combined mask, mask s,mix,k is the combined mask output by the student model for the kth preoperative image in the dataset that has been labeled with the combined mask, mask mix,k (real) is the combined mask annotation value of the kth preoperative image in the dataset that has been annotated with the combined mask, m2 is the total number of preoperative images in the dataset that have been annotated with the combined mask, α is a hyperparameter, and Both are L2 norm forms.

[0126] Methods for determining the preoperative image segmentation model include:

[0127] The model parameters are iteratively transferred and updated between the student model and the teacher model through the exponential moving average EMA method;

[0128] The student model after iterative transfer and update of model parameters is used as the preoperative image segmentation model.

[0129] All preoperative images constituting the dataset were normalized.

[0130] The labeled value of the vascular tissue mask corresponds to the true value of the vascular tissue mask, the labeled value of the organ tissue mask corresponds to the true value of the organ tissue mask, and the labeled value of the combined mask corresponds to the true value of the combined mask.

[0131] like Figure 2 As shown, the present invention provides a preoperative image segmentation system based on hybrid supervised enhancement, which is applied to a preoperative image segmentation method based on hybrid supervised enhancement. The system includes:

[0132] a segmentation model construction unit, configured to construct an organ tissue segmentation model for segmenting organ tissues in preoperative images using a Unet network, and to construct a vascular tissue segmentation model for segmenting vascular tissues in preoperative images using a Unet network;

[0133] a teacher model construction unit, configured to set a supervision enhancement module between the organ tissue segmentation model and the vascular tissue segmentation model, and combine the organ tissue segmentation model and the vascular tissue segmentation model into a teacher model;

[0134] The student model construction unit is used to construct a student model using the FCN network, and obtain a preoperative image segmentation model for segmenting vascular tissue and organ tissue in preoperative images through mixed supervised learning of the teacher model and the student model.

[0135] The present invention proposes a hybrid supervised segmentation model for preoperative three-dimensional medical image segmentation. When constructing the teacher model, a supervised enhancement method is adopted to ensure that the teacher model can accurately segment each tissue. Based on the "teacher-student" multi-fine-grained feature learning strategy, a lightweight model structure is obtained, and a student model that can accurately segment each tissue is realized, so that segmentation accuracy and efficiency coexist.

[0136] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.

Claims

1. A preoperative image segmentation method based on hybrid supervised enhancement, characterized in that: The following steps are involved: Using the Unet network to construct an organ tissue segmentation model for segmenting organ tissue in preoperative images, and using the Unet network to construct a vascular tissue segmentation model for segmenting vascular tissue in preoperative images; A supervision enhancement module is set between the organ tissue segmentation model and the vascular tissue segmentation model, and the organ tissue segmentation model and the vascular tissue segmentation model are combined into a teacher model; The FCN network is used to construct a student model, and through mixed supervised learning of the teacher model and the student model, a preoperative image segmentation model for segmenting vascular tissue and organ tissue in preoperative images is obtained.

2. The preoperative image segmentation method based on hybrid supervised enhancement according to claim 1, characterized in that: The method for constructing the organ tissue segmentation model includes: Acquire multiple preoperative images and mark the organ tissue masks in each preoperative image; Combining each preoperative image and the organ tissue mask in each preoperative image into a dataset, and dividing the dataset into a training set and a test set; Based on the training set, the preoperative image is used as the input of the Unet network, the organ tissue mask in the preoperative image is used as the output of the Unet network, and the Unet network is trained to obtain the organ tissue segmentation model; Based on the test set, performing a performance evaluation on the organ tissue segmentation effect of the organ tissue segmentation model; The organ tissue segmentation model is: mask organ =Dreams(G); Where, mask organ is the organ tissue mask output by the organ tissue segmentation model, G is the preoperative image, and Unet is the Unet network; The loss function of the organ tissue segmentation model is: Where, L organ is the loss function value of the organ tissue segmentation model, mask organ,i is the organ tissue mask output by the organ tissue segmentation model for the i-th preoperative image in the dataset, mask organ,i (real) is the labeled value of the organ tissue mask in the i-th preoperative image in the dataset, is the L2 norm, and n is the total number of preoperative images in the dataset.

3. The preoperative image segmentation method based on hybrid supervised enhancement according to claim 2, characterized in that: The method for constructing the vascular tissue segmentation model includes: Acquire multiple preoperative images and mark the vascular tissue mask in each preoperative image; Combining each preoperative image and the vascular tissue mask in each preoperative image into a dataset, and dividing the dataset into a training set and a test set; Based on the training set, the preoperative image is used as the input of the Unet network, and the vascular tissue mask in the preoperative image is used as the output of the Unet network to train the Unet network to obtain the vascular tissue segmentation model; Based on the test set, a performance evaluation is performed on the vascular tissue segmentation effect of the vascular tissue segmentation model; The vascular tissue segmentation model is: mask vessel =Dreams(G); Where, mask vessel is the vascular tissue mask output by the vascular tissue segmentation model, G is the preoperative image, and Unet is the Unet network; The loss function of the vascular tissue segmentation model is: Where, L vessel is the loss function value of the vascular tissue segmentation model, mask vessel,i is the vascular tissue mask output by the vascular tissue segmentation model for the i-th preoperative image in the dataset, mask vessel,i (real) is the labeled value of the vascular tissue mask in the i-th preoperative image in the dataset, is the L2 norm, and n is the total number of preoperative images in the dataset.

4. The preoperative image segmentation method based on hybrid supervised enhancement according to claim 3, characterized in that: The supervision enhancement module includes: In the formula, F1 is the first objective of supervision enhancement, min is the minimum operator, and L vessel (t) is the loss function value obtained at the tth training iteration of the vascular tissue segmentation model, L organ (t) is the loss function value obtained at the tth training iteration of the organ tissue segmentation model, and T is the total number of iterations of the training model; In the formula, F2 is the second target of supervision enhancement, min is the minimum operator, and mask vessel,i (T) is the vascular tissue mask output by the vascular tissue segmentation model for the i-th preoperative image in the segmentation dataset at the T-th training iteration, mask organ,i (T) is the organ tissue mask output by the vascular tissue segmentation model for the i-th preoperative image in the segmentation dataset at the T-th training iteration, mask mix,i (real) is the combined mask annotation value of the i-th preoperative image in the dataset, and n is the total number of preoperative images in the dataset.

5. The preoperative image segmentation method based on hybrid supervised enhancement according to claim 4, characterized in that: The method for constructing the teacher model includes: The output ends of the vascular tissue segmentation model and the organ tissue segmentation model after T training iterations are connected through a superposition module to obtain the teacher model; The teacher model is: mask g,mix =mask vessel (T)+mask organ (T); Where, mask g,mix The combined mask of vascular tissue and organ tissue output by the teacher model, mask vessel (T) is the vascular tissue mask output by the vascular tissue segmentation model after T training iterations, mask organ (T) is the organ tissue mask output by the organ tissue segmentation model after T training iterations.

6. The preoperative image segmentation method based on hybrid supervised enhancement according to claim 5, characterized in that: The method for constructing the student model includes: Acquire a plurality of preoperative images with combined masks labeled with vascular tissue and organ tissue, and acquire a preoperative image without the combined masks labeled, and output the combined masks of the preoperative images without the combined masks labeled through a teacher model; The preoperative images and the combined masks in the preoperative images are combined into a dataset, and the dataset is divided into a training set and a test set; Based on the training set, the preoperative image is used as the input of the FCN network, the combined mask in the preoperative image is used as the output of the Unet network, and the Unet network is trained to obtain the student model; Based on the test set, the performance of the student model for vascular tissue and organ tissue segmentation is evaluated; The student model is: mask s,mix =Dreams(G); Where, mask s,mix is the combined mask output by the student model, G is the preoperative image, and Unet is the Unet network; The loss function of the student model includes the distillation loss L KL and segmentation loss L CE ,in: L total =αL KL +(1―α)L CE ; Where, L total is the total loss, L KL is the distillation loss, L CE is the segmentation loss, mask s,mix,j is the combined mask output by the student model for the jth preoperative image in the dataset that is not labeled with the combined mask, mask g,mix,j is the combined mask output by the teacher model for the jth preoperative image in the dataset that is not labeled with the combined mask, m1 is the total number of preoperative images in the dataset that are not labeled with the combined mask, and mask s,mix,k is the combined mask output by the student model for the kth preoperative image in the dataset that has been labeled with the combined mask, mask mix,k (real) is the combined mask annotation value of the kth preoperative image in the dataset that has been annotated with the combined mask, m2 is the total number of preoperative images in the dataset that have been annotated with the combined mask, α is a hyperparameter, and Both are L2 norm forms.

7. The preoperative image segmentation method based on hybrid supervised enhancement according to claim 6, characterized in that: The method for determining the preoperative image segmentation model includes: The model parameters are iteratively transferred and updated between the student model and the teacher model through the exponential moving average EMA method; The student model after the model parameters are iteratively transferred and updated is used as the preoperative image segmentation model.

8. The preoperative image segmentation method based on hybrid supervised enhancement according to claim 1, characterized in that: All preoperative images constituting the dataset were normalized.

9. The preoperative image segmentation method based on hybrid supervised enhancement according to claim 6, characterized in that: The labeled value of the vascular tissue mask corresponds to the true value of the vascular tissue mask, the labeled value of the organ tissue mask corresponds to the true value of the organ tissue mask, and the labeled value of the combination mask corresponds to the true value of the combination mask.

10. A preoperative image segmentation system based on hybrid supervised enhancement, characterized in that: A preoperative image segmentation method based on hybrid supervised enhancement as described in any one of claims 1 to 9, the system comprising: a segmentation model construction unit, configured to construct an organ tissue segmentation model for segmenting organ tissues in preoperative images using a Unet network, and to construct a vascular tissue segmentation model for segmenting vascular tissues in preoperative images using a Unet network; a teacher model construction unit, configured to set a supervision enhancement module between the organ tissue segmentation model and the vascular tissue segmentation model, and combine the organ tissue segmentation model and the vascular tissue segmentation model into a teacher model; The student model construction unit is used to construct a student model using the FCN network, and obtain a preoperative image segmentation model for segmenting vascular tissue and organ tissue in preoperative images through mixed supervised learning of the teacher model and the student model.

Images