A method, apparatus, electronic device, and storage medium for region of interest segmentation.

Abstract

This invention discloses a method, apparatus, electronic device, and storage medium for region-of-interest (ROI) segmentation. The method includes: acquiring a target medical image for ROI segmentation, and a trained target segmentation model for ROI segmentation; inputting the target medical image into the target segmentation model, and obtaining the segmentation result of the target medical image for the ROI based on the output of the target segmentation model; wherein the target segmentation model is trained based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature library. The technical solution of this invention can reduce the possibility of segmenting non-ROI regions as ROI regions, thereby improving segmentation performance.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and particularly relate to a method and apparatus for segmenting a region of interest, an electronic device, and a storage medium. BACKGROUND

[0002] In medical image analysis, accurately segmenting a region of interest from a medical image is a basic and important step for subsequent analysis and quantification.

[0003] The current region of interest segmentation scheme has low segmentation performance of the model in the case of training the model using a small number of labeled training samples, and needs to be solved. SUMMARY

[0004] Embodiments of the present application provide a method and apparatus for segmenting a region of interest, an electronic device, and a storage medium, which can reduce the possibility of segmenting a non-region of interest as a region of interest and improve segmentation performance.

[0005] According to an aspect of the present application, a method for segmenting a region of interest is provided, which can include:

[0006] obtaining a target medical image to be segmented for a region of interest, and a target segmentation model for segmenting a region of interest that has been trained;

[0007] inputting the target medical image into the target segmentation model, and obtaining a segmentation result of the target medical image for the region of interest according to an output result of the target segmentation model;

[0008] wherein the target segmentation model is trained based on a plurality of first training samples, a plurality of second training samples, and a medical image feature library;

[0009] each of the plurality of first training samples includes a first medical image containing a region of interest and a target segmentation label of the first medical image for the region of interest;

[0010] each of the plurality of second training samples includes a second medical image containing a region of interest;

[0011] The medical image feature library is determined based on a plurality of third medical images, and the plurality of third medical images do not contain a region of interest.

[0012] According to another aspect of the present application, a device for segmenting a region of interest is provided, which can include:

[0013] a target segmentation model obtaining module configured to obtain a target medical image to be segmented for a region of interest, and a target segmentation model for segmenting a region of interest that has been trained;

[0014] a segmentation result obtaining module configured to input the target medical image into the target segmentation model, and obtain a segmentation result of the target medical image for the region of interest according to an output result of the target segmentation model;

[0015] The target segmentation model is trained based on a plurality of first training samples, a plurality of second training samples and a medical image feature library.

[0016] Each of the plurality of first training samples comprises a first medical image containing the region of interest and a target segmentation label of the first medical image for the region of interest.

[0017] Each of the plurality of second training samples comprises a second medical image containing the region of interest.

[0018] The medical image feature library is determined based on a plurality of third medical images, and the plurality of third medical images do not contain the region of interest.

[0019] According to another aspect of the present application, an electronic device can include:

[0020] at least one processor; and

[0021] a memory connected to the at least one processor in communication; wherein

[0022] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to cause the at least one processor to implement the region of interest segmentation method provided by any of the embodiments of the present application.

[0023] According to another aspect of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores computer instructions for causing a processor to implement the region of interest segmentation method provided by any of the embodiments of the present application.

[0024] The technical scheme of the embodiment of the present application comprises the following steps: obtaining a target medical image to be segmented in a region of interest, and a target segmentation model for segmenting the region of interest which has been trained; inputting the target medical image into the target segmentation model, and obtaining a segmentation result of the target medical image for the region of interest according to an output result of the target segmentation model; wherein the target segmentation model is trained based on a plurality of first training samples, a plurality of second training samples and a medical image feature library; each of the plurality of first training samples comprises a first medical image containing the region of interest and a target segmentation label of the first medical image for the region of interest; each of the plurality of second training samples comprises a second medical image containing the region of interest; and the medical image feature library is determined based on a plurality of third medical images, and the plurality of third medical images do not contain the region of interest. The above technical scheme is based on the third medical images not containing the region of interest for model training, realizes sufficient learning of the third medical images, expands and perfects the features of the non-region of interest, so that the possibility of segmenting the non-region of interest as the region of interest can be reduced, and the segmentation performance is improved.

[0025] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0027] Figure 1 is a flowchart of a region of interest segmentation method according to an embodiment of the present application;

[0028] Figure 2 is a flowchart of another region of interest segmentation method according to an embodiment of the present application;

[0029] Figure 3 is a flowchart of another region of interest segmentation method according to an embodiment of the present application;

[0030] Figure 4 is a flowchart of a training feature library construction model in another region of interest segmentation method according to an embodiment of the present application;

[0031] Figure 5 is a flowchart of another region of interest segmentation method according to an embodiment of the present application;

[0032] Figure 6 This is a flowchart of an optional example of another region of interest segmentation method provided according to an embodiment of the present invention;

[0033] Figure 7 This is a flowchart of another region of interest segmentation method provided by an embodiment of the present invention;

[0034] Figure 8 This is a flowchart of another region of interest segmentation method provided by an embodiment of the present invention;

[0035] Figure 9 This is a flowchart of another region of interest segmentation method provided by an embodiment of the present invention;

[0036] Figure 10 This is a flowchart of an optional example of another region of interest segmentation method provided according to an embodiment of the present invention;

[0037] Figure 11 This is a structural block diagram of a region of interest segmentation device according to an embodiment of the present invention;

[0038] Figure 12 This is a structural block diagram of an electronic device that implements the region of interest segmentation method of the present invention. Detailed Implementation

[0039] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. The same applies to "target," "original," etc., and will not be repeated here. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0041] Figure 1 This is a flowchart of a region of interest (ROI) segmentation method provided in an embodiment of the present invention. This embodiment is applicable to ROI segmentation. The method can be executed by the ROI segmentation apparatus provided in this embodiment, which can be implemented in software and / or hardware and can be integrated into an electronic device, such as various user terminals or servers.

[0042] See Figure 1 The method of this invention specifically includes the following steps:

[0043] S101. Obtain the target medical image to be segmented into a region of interest, and a trained target segmentation model for segmenting the region of interest. The target segmentation model is trained based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature library. Each set of first training samples includes a first medical image containing a region of interest and a target segmentation label for the region of interest in the first medical image. Each set of second training samples includes a second medical image containing a region of interest. The medical image feature library is determined based on multiple third medical images, none of which contain a region of interest.

[0044] The target medical image is the medical image to be segmented into a region of interest (ROI). The target medical image can be a two-dimensional, three-dimensional, or four-dimensional medical image, for example, a three-dimensional medical image with time-series information. The target medical image can be a medical image obtained through computed tomography (CT) or magnetic resonance imaging (MR). In this embodiment of the invention, the source and type of the target medical image are not specifically limited. The target medical image can be a medical image containing a ROI or a medical image not containing a ROI. The ROI can be understood as the region to be segmented from the target medical image. For example, the ROI can be a tumor region or a nodule region, which can serve as a reference factor for medical personnel to locate lesions. The ROI can also be a physiological structural region, such as the heart region, bone region, or liver region. In this embodiment of the invention, the type of ROI is not specifically limited.

[0045] In this embodiment of the invention, a target segmentation model capable of segmenting regions of interest (ROIs) of a target medical image can be pre-trained based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature library. The first training samples are samples used to train the target segmentation model, including target segmentation labels; a set of first training samples may include a first medical image and its corresponding target segmentation label for the ROI. The second training samples are samples used to train the target segmentation model without target segmentation labels; a set of second training samples may include a second medical image. The medical image feature library is a feature library containing features of a third medical image. The first medical image is a sample image for training the target segmentation model that has a pre-determined ROI, and the first medical image corresponds to a target segmentation label for its ROI. The second medical image is a sample image for training the target segmentation model that has a pre-determined ROI, although the second medical image has a determined ROI, it does not correspond to a target segmentation label for its ROI. The third medical image is a sample image for determining the medical image feature library that does not contain a ROI. The target segmentation label is a label that reflects the label obtained after segmenting the ROI of the first medical image.

[0046] It is important to note that medical images without regions of interest are easier to obtain than those with regions of interest. Furthermore, the more medical images without regions of interest there are, the more complete the feature space for these images becomes. A complete feature space for these images ensures that a model with good segmentation performance can be trained using fewer or even no medical images with regions of interest. This reduces the need for medical images with regions of interest and lowers the labeling cost of such images.

[0047] Exemplarily, a pre-built original segmentation model, N medical images containing regions of interest, and M third medical images not containing regions of interest can be obtained; randomly select P (P < N) medical images from the N medical images containing regions of interest as the first medical images, and perform region-of-interest annotation on the P first medical images to obtain the target segmentation labels corresponding to the P first medical images respectively; obtain P groups of first training samples according to the P first medical images and the target segmentation labels corresponding to the P first medical images respectively; use the remaining medical images in the N medical images containing regions of interest as N - P (N - P > 0) second medical images; obtain N - P groups of second training samples according to the N - P second medical images; determine a medical image feature library based on the M third medical images; train the original segmentation model based on the P groups of first training samples, the N - P second training samples, and the medical image feature library to obtain a target segmentation model. Among them, the original segmentation model can be understood as a model to be trained that can be used to segment regions of interest.

[0048] S102. Input the target medical image into the target segmentation model, and obtain the segmentation result of the target medical image for the region of interest according to the output result of the target segmentation model.

[0049] In the embodiment of the present invention, input the target medical image into the target segmentation model, and obtain the segmentation result of the target medical image for the region of interest according to the output result of the target segmentation model that can reflect the segmentation result.

[0050] It should be noted that when the target medical image is a medical image that does not contain a region of interest, a segmentation result of the target medical image for the region of interest can be obtained. This segmentation result may not include the segmented region of interest. For example, in the result map corresponding to the segmentation result, the region of interest can be represented by the identifier "1" in the result map, and the non-region of interest can be represented by the identifier "0" in the result map. When the target medical image is a medical image that does not contain a region of interest, the identifiers on the result map corresponding to the segmentation result of the target medical image for the region of interest can all be "0".

[0051] The technical solution of this invention involves acquiring a target medical image for region of interest (ROI) segmentation and a trained target segmentation model for ROI segmentation. The target medical image is input into the target segmentation model, and the segmentation result of the target medical image for the ROI is obtained based on the output of the target segmentation model. The target segmentation model is trained based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature library. Each set of first training samples includes a first medical image containing the ROI and a target segmentation label for the ROI. Each set of second training samples includes a second medical image containing the ROI. The medical image feature library is determined based on multiple third medical images, none of which contain the ROI. This technical solution, by training the model on third medical images that do not contain the ROI, achieves sufficient learning of the third medical images, expands and improves the features of non-ROI regions, thereby reducing the possibility of segmenting non-ROI regions as ROI regions and improving segmentation performance.

[0052] Figure 2 This is a flowchart of another region-of-interest segmentation method provided in this embodiment of the invention. This embodiment is based on and optimized from the above-described technical solutions. In this embodiment, optionally, the target segmentation model is pre-trained through the following steps: obtaining a pre-built original segmentation model, as well as multiple sets of first training samples, multiple sets of second training samples, and multiple third medical images; extracting features from the multiple third medical images to obtain a medical image feature library; and training the original segmentation model based on the multiple sets of first training samples, multiple sets of second training samples, and the medical image feature library to obtain the target segmentation model. The explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0053] See Figure 2 The method in this embodiment may specifically include the following steps:

[0054] S201. Obtain the pre-built original segmentation model, as well as multiple sets of first training samples, multiple sets of second training samples, and multiple third medical images. Each set of first training samples includes a first medical image containing a region of interest and a target segmentation label for the region of interest in the first medical image. Each set of second training samples includes a second medical image containing a region of interest. None of the multiple third medical images contain a region of interest.

[0055] In this embodiment of the invention, a pre-built original segmentation model can be obtained, which includes a first medical image and a target segmentation label, and the first medical image and the target segmentation label can be used as a first training sample; a second medical image can be obtained, which can be used as a second training sample, and multiple third medical images can be obtained.

[0056] S202. Extract features from multiple third-party medical images to obtain a medical image feature library.

[0057] In this embodiment of the invention, for example, features can be extracted from multiple third medical images, and the extracted features can be subjected to feature clustering and mean feature calculation. The medical image feature library can be determined based on the obtained mean feature.

[0058] S203. Based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature library, the original segmentation model is trained to obtain a target segmentation model for segmenting regions of interest.

[0059] In this embodiment of the invention, loss can be calculated using multiple sets of first training samples and a segmentation loss function, and also using multiple sets of second training samples, a medical image feature library, and a segmentation loss function to adjust the parameters in the original segmentation model and obtain the target segmentation model. In this embodiment of the invention, the method of training the original segmentation model based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature library to obtain the target segmentation model, and the loss function used, are not specifically limited.

[0060] S204. Obtain the target medical image for region of interest segmentation.

[0061] S205. Input the target medical image into the target segmentation model, and obtain the segmentation result of the target medical image for the region of interest based on the output result of the target segmentation model.

[0062] The technical solution of this invention can obtain a medical image feature library based on a third medical image that does not contain a region of interest, thereby achieving full learning of the third medical image that does not contain a region of interest, expanding and improving the features of the non-region of interest, and training the original segmentation model based on the medical image feature library that includes complete features of the non-region of interest, thereby reducing the possibility of segmenting the non-region of interest as a region of interest and improving the segmentation performance of the trained target segmentation model.

[0063] Figure 3This is a flowchart of another region-of-interest segmentation method provided in this embodiment of the invention. This embodiment is based on the above-mentioned technical solutions and optimized. In this embodiment, optionally, feature extraction is performed on multiple third medical images to obtain a medical image feature library, including: obtaining a feature library construction model to be trained, wherein the feature library construction model includes at least a first encoder; for each of the multiple third medical images, the third medical image is input into the first encoder to obtain a first feature map; the first features in the first feature map are clustered to obtain at least one feature cluster; for each feature cluster in the at least one feature cluster, the mean feature of the first feature in the feature cluster is determined, and the mean feature is stored in an intermediate image feature library; the feature library construction model is trained based on the mean feature, and the feature library construction model is updated according to the obtained training results; the steps for each of the multiple third medical images are repeated; when the feature library construction model training is completed, the currently obtained intermediate image feature library is used as the medical image feature library. The explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0064] See Figure 3 The method in this embodiment may specifically include the following steps:

[0065] S301. Obtain the pre-built original segmentation model, as well as multiple sets of first training samples, multiple sets of second training samples, and multiple third medical images. Each set of first training samples includes a first medical image containing a region of interest and a target segmentation label for the region of interest in the first medical image. Each set of second training samples includes a second medical image containing a region of interest. None of the multiple third medical images contain a region of interest.

[0066] S302. Obtain the feature library construction model to be trained, wherein the feature library construction model includes at least a first encoder.

[0067] The feature library construction model is a model that can be used to construct a medical image feature library. The feature library construction model can also be used to train a first encoder, which is then used to train a target segmentation model. The first encoder is an encoder used to encode the third medical image input therein. In this embodiment of the invention, the structure and type of the first encoder are not specifically limited.

[0068] S303. For each of the multiple third medical images, input the third medical image into the first encoder to obtain the first feature map.

[0069] The first feature map is the feature map that the first encoder can obtain after inputting the third medical image; the first feature map can be a continuous feature map or a discrete feature map.

[0070] It should be noted that when the first feature map is a continuous feature map, it may be a discrete feature map with a certain degree of continuity. For example, the first feature map Fea-c=[[1],[2],[3],[6],[7],[8]], where “[[1],[2],[3]]” and “[[6],[7],[8]]” are discrete features, but have a certain degree of continuity.

[0071] S304. Perform clustering on the first feature in the first feature map to obtain at least one feature cluster.

[0072] Here, the first feature is the feature in the first feature map. The feature cluster is a cluster of features that are similar to each other.

[0073] In this embodiment of the invention, the first feature map may include at least one first feature. The first features in the first feature map can be clustered according to the distance between them to obtain at least one feature cluster. For example, when the first feature map is a three-dimensional feature map, and the size of the first feature map Fea-c is C×H×W, where C is the number of channels in the first feature map, H is the height of the first feature map, and W is the width of the first feature map, then Fea-c contains a total of H×W first features {f 11 ,f 12 ,...,f HW}, each f ij The size of (i=1,2,...,H,j=1,2,...,W) is C×1×1. These H×W features can be clustered according to the distance between the features to obtain K feature clusters. For example, when the first feature map is a two-dimensional feature map, the first feature map Fea-c=[[1],[2],[3],[6],[7],[8]], then there are 6 first features in Fea-c, namely [[1]], [[2]], [[3]], [[6]], [[7]], [[8]], which can be clustered according to the Euclidean distance between the features to obtain 2 feature clusters. One feature cluster is {[[1]], [[2]], [[3]]}, and the other feature cluster is {[[6]], [[7]], [[8]]}.

[0074] S305. For each feature cluster in at least one feature cluster, determine the mean feature of the first feature in the feature cluster, and store the mean feature in the intermediate image feature library.

[0075] The intermediate image feature library can be understood as an incomplete medical image feature library.

[0076] In this embodiment of the invention, for each feature cluster in at least one feature cluster, the mean feature of the first feature in the feature cluster can be determined. The mean feature is the feature obtained by averaging the first feature in the feature cluster. The mean feature is stored in the intermediate image feature library. In conjunction with the example above where the first feature map is a three-dimensional feature map, for example, for each feature cluster in K feature clusters, the mean feature of the first feature in the feature cluster can be determined, resulting in K mean features. The K mean features are then stored in the intermediate image feature library. In conjunction with the example above where the first feature map is a two-dimensional feature map, for example, for each feature cluster in two feature clusters {[[1]], [[2]], [[3]]} and {[[6]], [[7]], [[8]]}, the mean feature of the first feature in the feature cluster can be determined, resulting in the mean feature of the first feature cluster being [[2]] and the mean feature of the second feature cluster being [[7]]. These two mean features are then stored in the intermediate image feature library.

[0077] S306. Based on the mean feature, train the feature library construction model and update the feature library construction model according to the training results.

[0078] In this embodiment of the invention, the feature library construction model can be trained based on mean features and third-party medical images, and the parameters in the feature library construction model can be updated according to the training results.

[0079] S307. Return to execute S303, and if the feature library construction model training is completed, use the currently obtained intermediate image feature library as the medical image feature library.

[0080] It should be noted that because the training level of the first encoder varies when repeatedly executing the steps for each of the multiple third medical images, the resulting first feature maps may differ even for the same third medical image. In this embodiment of the invention, the mean features obtained after each repeated execution of the steps for each of the multiple third medical images can be stored in the intermediate image feature library; alternatively, the mean features can be stored in the intermediate image feature library only after the first encoder has been trained a certain number of times and its accuracy is sufficiently high. The mean features obtained before the training number exceeds a certain number of times are only used to train the feature library and build the model, and are not stored in the intermediate image feature library.

[0081] In this embodiment of the invention, once the feature library construction model training is completed, it means that the medical image feature library has also been constructed. At this time, the currently obtained intermediate image feature library can be used as the medical image feature library.

[0082] S308. Based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature library, the original segmentation model is trained to obtain a target segmentation model for segmenting regions of interest.

[0083] S309. Obtain the target medical image for region of interest segmentation.

[0084] S310. Input the target medical image into the target segmentation model, and obtain the segmentation result of the target medical image for the region of interest based on the output of the target segmentation model.

[0085] The technical solution of this invention can perform clustering processing on the first feature in the first feature map, determine the mean feature of the first feature of the feature cluster obtained by the clustering processing, store the mean feature in the intermediate image feature library, and, when the feature library construction model training is completed, use the currently obtained intermediate image feature library as the medical image feature library. This can make the features of the non-interest regions in the obtained medical image feature library more complete, thereby further reducing the possibility of segmenting the non-interest regions as interest regions during the model training process and improving the segmentation performance of the trained target segmentation model.

[0086] An optional technical solution involves storing the mean feature in an intermediate image feature library, including: obtaining the number of training iterations of the feature library construction model, and storing the mean feature in the intermediate image feature library when the number of training iterations exceeds a preset number.

[0087] The preset number of training iterations is the number of times the mean feature can be stored in the intermediate image feature library after exceeding the preset number of training iterations. The preset number of training iterations can be determined, for example, by the number of training iterations that enable the first encoder to achieve a preset accuracy, or by the accuracy required for the mean feature in the medical image feature library, or by the size of the required medical image feature library, and so on.

[0088] The technical solution of this invention can store the mean features in the intermediate image feature library after the number of training times exceeds a preset number and the accuracy of the mean features is high enough, thereby ensuring the accuracy of the mean features stored in the medical image feature library.

[0089] Another optional technical solution includes a feature library construction model that further includes a first reconstruction decoder and a second reconstruction decoder. The feature library construction model is trained based on the mean feature, including: inputting a first feature map into the first reconstruction decoder to obtain its output; performing a replacement operation on each first feature in the first feature map corresponding to the mean feature, based on the mean feature, to obtain a second feature map; inputting the second feature map into the second reconstruction decoder to obtain its output; and training the feature library construction model based on the third medical image, the output of the first reconstruction decoder, and the output of the second reconstruction decoder.

[0090] The second reconstruction decoder is a decoder capable of decoding the second feature map. The first reconstruction decoder is a decoder capable of decoding the first feature map. In this embodiment of the invention, the specific types and structures of the second and first reconstruction decoders are not specifically limited. The second feature map is a feature map obtained by replacing each first feature corresponding to the mean feature in the first feature map according to the mean feature.

[0091] For example, see Figure 4The feature library construction model includes a first encoder Enc, a first reconstruction decoder Dec-c, and a second reconstruction decoder Dec-d. For each third medical image O-Data in multiple third medical images, O-Data is input into the first encoder Enc to obtain a first feature map Fea-c, which is a continuous feature map. The first feature map Fea-c is input into the first reconstruction decoder Dec-c to obtain the output result O-Data'' of the first reconstruction decoder Dec-c. The first features in the first feature map Fea-c are clustered to obtain at least one feature cluster, where the first features in each feature cluster are similar to each other. For each feature cluster, the mean of the first features in the feature cluster is calculated to obtain the mean feature, and the mean feature is stored in the intermediate image feature library Lib-f. Based on the mean feature, each third medical image corresponding to the mean feature in the first feature map is clustered. A feature replacement operation is performed to obtain a second feature map Fea-d; the second feature map Fea-d is input into the second reconstruction decoder Dec-d to obtain the output result O-Data' of the second reconstruction decoder Dec-d; loss can be calculated using O-Data', O-Data'', and the consistency loss function, and then using O-Data'', O-Data, and the reconstruction loss function to adjust the parameters in the feature library construction model and train the feature library construction model; the steps for O-Data in each of the multiple third medical images are repeated until the feature library construction model converges, that is, the feature library construction model training is completed; when the feature library construction model training is completed, the currently obtained intermediate image feature library Lib-f is used as the medical image feature library. It is important to note that the gradient of the second reconstruction decoder Dec-d can be directly transmitted to the first encoder Enc for gradient update of the network. Furthermore, since the inputs of the two branches corresponding to the first and second reconstruction decoders are different, an error will occur between O-Data' and O-Data''. Therefore, a consistency loss function can be used to constrain the image reconstruction, and the consistency loss function has advantages in the reconstruction of local image details. In addition, by adopting the above technical solution, the features included in the obtained medical image feature library can be made more universal.

[0092] It should be noted that, in this embodiment of the invention, no specific limitations are made on the method of training the feature library construction model based on the third medical image, the output of the first reconstruction decoder, and the output of the second reconstruction decoder, or on the loss function used. In this embodiment of the invention, no specific limitations are made on the type and structure of the first and second reconstruction decoders; the type and structure of the first and second reconstruction decoders can be the same or different.

[0093] In this embodiment of the invention, the first feature map can be input into the first reconstruction decoder to obtain the output result of the first reconstruction decoder; according to the mean feature, each first feature in the first feature map corresponding to the mean feature is replaced to obtain the second feature map. Combining the above examples of the case where the first feature map is a two-dimensional feature map, for example, the first feature map Fea-c=[[1],[2],[3],[6],[7],[8]], and the mean feature of the feature cluster {[[1]], [[2]], [[3]]} is [[2]]. The mean feature of the feature cluster {[[6]], [[7]], [[8]]} is [[7]]. The first feature corresponding to the mean feature in the first feature map is replaced to obtain the second feature map Fea-d=[[2],[2],[2],[7],[7],[7]]. The second feature map is input into the second reconstruction decoder to obtain the output result of the second reconstruction decoder. The feature library construction model is trained according to the third medical image, the output result of the first reconstruction decoder and the output result of the second reconstruction decoder. The above technical solution trains the feature library construction model through the two branches corresponding to the first reconstruction decoder and the second reconstruction decoder, which can make the trained feature library construction model more accurate, so as to facilitate the application of the feature library construction model to the trained target segmentation model, so that the medical image feature library is suitable for the target segmentation model.

[0094] Figure 5This is a flowchart of another region of interest segmentation method provided in this embodiment of the invention. This embodiment is based on the above-mentioned technical solutions and optimized. In this embodiment, optionally, the original segmentation model includes a second encoder and a first segmentation decoder, the second encoder having the same structure as the first encoder; after the feature library construction model training is completed, it further includes: updating the currently obtained first encoder to the first encoder; training the original segmentation model based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature library to obtain a target segmentation model, including: updating the parameters of the second encoder according to the parameters of the first encoder, and freezing the parameters of the second encoder; for each set of first training samples in the multiple sets of first training samples, inputting the first medical image in the first training sample into the second encoder to obtain a third feature map, and then using the third... The feature map is input to the first segmentation decoder to obtain the first output result of the first segmentation decoder; for each of the multiple sets of second training samples, the second medical image in the second training sample is input to the second encoder to obtain the fourth feature map, and the fourth feature map is input to the first segmentation decoder to obtain the second output result of the first segmentation decoder; based on the fourth feature map and the medical image feature library, a pseudo-segmentation label is determined; based on the first output result and target segmentation label of the first segmentation decoder corresponding to the multiple sets of first training samples, and the second output result and pseudo-segmentation label of the first segmentation decoder corresponding to the multiple sets of second training samples, the original segmentation model is trained to obtain the target segmentation model. The explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0095] See Figure 5 The method in this embodiment may specifically include the following steps:

[0096] S401. Obtain a pre-built original segmentation model, as well as multiple sets of first training samples, multiple sets of second training samples, and multiple third medical images. Each set of first training samples includes a first medical image containing a region of interest and a target segmentation label for the region of interest in the first medical image. Each set of second training samples includes a second medical image containing a region of interest. None of the multiple third medical images contain a region of interest. The original segmentation model includes a second encoder and a first segmentation decoder.

[0097] The second encoder is used to encode the first medical image and / or the second medical image input therein. The first segmentation decoder is used to decode the third feature map and / or the fourth feature map input therein. In this embodiment of the invention, the structure and type of the first segmentation decoder are not specifically limited.

[0098] S402. Obtain the feature library construction model to be trained, wherein the feature library construction model includes at least a first encoder and the second encoder has the same structure as the first encoder.

[0099] In this embodiment of the invention, the second encoder has the same structure as the first encoder, so as to facilitate the subsequent updating of the parameters of the second encoder based on the parameters of the first encoder after training.

[0100] S403. For each of the multiple third medical images, input the third medical image into the first encoder to obtain the first feature map.

[0101] S404. Perform clustering on the first feature in the first feature map to obtain at least one feature cluster.

[0102] S405. For each feature cluster in at least one feature cluster, determine the mean feature of the first feature in the feature cluster, and store the mean feature in the intermediate image feature library.

[0103] S406. Based on the mean feature, train the feature library construction model and update the feature library construction model according to the training results.

[0104] S407. Return to execute S403, and if the feature library construction model training is completed, use the currently obtained intermediate image feature library as the medical image feature library.

[0105] S408. Update the currently obtained first encoder to the first encoder.

[0106] In this embodiment of the invention, once the feature library construction model has been trained, the first encoder in the currently trained feature library construction model can be updated to the first encoder so that the trained first encoder can be used subsequently.

[0107] S409. Update the parameters of the second encoder according to the parameters of the first encoder, and freeze the parameters of the second encoder.

[0108] In this embodiment of the invention, the parameters of the second encoder can be updated according to the parameters of the first encoder. For example, the parameters of the first encoder can be used as the parameters of the second encoder. Alternatively, the first encoder can be directly used as the second encoder, and the parameters of the second encoder can be frozen, that is, the parameters of the second encoder are not adjusted in the subsequent process of the target segmentation model.

[0109] In this embodiment of the invention, the second encoder does not participate in parameter updates, and only trains the first segmentation decoder.

[0110] S410. For each group of first training samples in the multiple groups of first training samples, input the first medical image in the first training sample into the second encoder to obtain the third feature map, and input the third feature map into the first segmentation decoder to obtain the first output result of the first segmentation decoder.

[0111] In this embodiment of the invention, for each of the multiple sets of first training samples, the first medical image in the first training sample can be input into the second encoder to obtain a third feature map. The third feature map is the feature map output by the second encoder when the first medical image is input into the second encoder. The third feature map is then input into the first segmentation decoder to obtain the first output result of the first segmentation decoder.

[0112] S411. For each group of second training samples in the multiple groups of second training samples, input the second medical image in the second training sample into the second encoder to obtain the fourth feature map, and input the fourth feature map into the first segmentation decoder to obtain the second output result of the first segmentation decoder.

[0113] In this embodiment of the invention, for each of the multiple sets of second training samples, the second medical image in the second training sample is input into the second encoder to obtain a fourth feature map. This fourth feature map is the feature map output by the second encoder when the second medical image is input into the second encoder. The fourth feature map is then input into the first segmentation decoder to obtain the second output result of the first segmentation decoder.

[0114] S412. Determine the pseudo-segmentation label based on the fourth feature map and the medical image feature library.

[0115] Among them, the pseudo segmentation label is a pseudo label obtained based on the fourth feature map and the medical image feature database.

[0116] It should be noted that since the second medical image in the second training sample does not have a corresponding label, the second medical image itself cannot contribute learning information to the target segmentation model. Furthermore, the medical image feature library does not include features related to the region of interest. Therefore, the features of the region of interest in the fourth feature map are significantly different from the features in the medical image feature library, while the features of the non-region of interest are similar to the features in the medical image feature library. In this embodiment of the invention, the above phenomenon can be used to determine the pseudo segmentation label based on the fourth feature map and the medical image feature library.

[0117] In this embodiment of the invention, the original segmentation model may further include a pseudo-segmentation label generation module. The fourth feature map can be input into the pseudo-segmentation label generation module, which processes the fourth feature map based on a medical image feature library to obtain pseudo-segmentation labels. Specifically, the pseudo-segmentation label generation module in this embodiment can be understood as a module capable of determining pseudo-segmentation labels based on the fourth feature map and the medical image feature library. It should be noted that in this embodiment, the pseudo-segmentation label generation module is only used for training the original segmentation model; it is not included in the target segmentation model obtained after training.

[0118] S413. Based on the first output results and target segmentation labels of the first segmentation decoder corresponding to the multiple sets of first training samples, and the second output results and pseudo segmentation labels of the first segmentation decoder corresponding to the multiple sets of second training samples, the original segmentation model is trained to obtain the target segmentation model.

[0119] For example, the loss can be calculated using the first output of the first segmentation decoder corresponding to multiple sets of first training samples, the target segmentation label, and the segmentation loss function, and the loss can be calculated using the second output of the first segmentation decoder corresponding to multiple sets of second training samples, the pseudo segmentation label, and the segmentation loss function, so as to adjust the parameters in the original segmentation model, train the original segmentation model, and obtain the target segmentation model.

[0120] It should be noted that, in this embodiment of the invention, the method of training the original segmentation model to obtain the target segmentation model and the loss function used are not specifically limited. This is based on the first output results and target segmentation labels of the first segmentation decoder corresponding to the multiple sets of first training samples, and the second output results and pseudo segmentation labels of the first segmentation decoder corresponding to the multiple sets of second training samples.

[0121] S414. Obtain the target medical image for region of interest segmentation.

[0122] S415. Input the target medical image into the target segmentation model, and obtain the segmentation result of the target medical image for the region of interest based on the output of the target segmentation model.

[0123] The technical solution of this invention, by making the second encoder have the same structure as the first encoder, and updating the currently obtained first encoder after the feature library construction model training is completed, allows the non-interest region features in the fourth feature map obtained by inputting the second medical image from the second training sample into the second encoder to better match the medical image feature library, thereby improving the accuracy of the determined pseudo-segmentation labels. Furthermore, by allowing the pseudo-segmentation labels to participate in model training, the possibility of the trained target segmentation model segmenting non-interest regions as interest regions can be reduced, improving segmentation performance. At the same time, the demand for medical images containing interest regions can be reduced, lowering the labeling cost of medical images containing interest regions. To better understand the technical solution of the above embodiments of this invention, an optional example is provided here. For example, see... Figure 6The process involves updating the parameters of the second encoder (Enc) based on the parameters of the first encoder, and then freezing the parameters of the second encoder (Enc). For each set of first training samples, the first medical image (L-Data) from the first training sample is input into the second encoder (Enc) to obtain a third feature map. This third feature map is then input into the first segmentation decoder (Dec-seg) to obtain the first output result (L-Pred) of the first segmentation decoder (Dec-seg). For each set of second training samples, the second medical image (U-Data) from the second training sample is input into the second encoder (Enc) to obtain a fourth feature map. This fourth feature map is then input into the first segmentation decoder (Dec-seg) to obtain the second output result (U-Pred) of the first segmentation decoder (Dec-seg). For each fourth feature (f) in the fourth feature map, the similarity between the fourth feature (f) and each mean feature in the medical image feature library (Lib-f) is determined by comparison. Based on the similarity between the fourth feature (f) and each mean feature in the medical image feature library (Lib-f), a similarity sequence is obtained. Statistical values ​​are calculated for the similarities included in the similarity sequence to determine the first... The similarity statistics corresponding to four features f are calculated. Based on the position of each fourth feature f in the fourth feature map, the statistics corresponding to each fourth feature f are arranged to obtain a similarity description map. The statistics in the similarity description map are clustered to obtain at least one statistical cluster. A similarity threshold is determined based on the at least one statistical cluster. A label map is obtained based on the similarity threshold and the similarity description map; this label map is a low-resolution image. The label map is upsampled to obtain a pseudo-segmentation label U-Pseudo. Loss can be calculated using the first output result L-Pred of the first segmentation decoder Dec-seg corresponding to multiple sets of first training samples, the target segmentation label L-Label, and segmentation loss functions such as cross-entropy loss function or DICE loss function. Loss can also be calculated using the second output result U-Pred of the first segmentation decoder Dec-seg corresponding to multiple sets of second training samples, the pseudo-segmentation label U-Pseudo, and consistency loss functions such as cross-entropy loss function, DICE loss function, L1 loss function, or L2 loss function. These methods are used to adjust the parameters in the original segmentation model, train the original segmentation model, and obtain the target segmentation model.

[0124] Figure 7This is a flowchart of another region-of-interest segmentation method provided in this embodiment of the invention. This embodiment is based on and optimized from the above-described technical solutions. In this embodiment, optionally, determining pseudo-segmentation labels based on the fourth feature map and the medical image feature library includes: for each fourth feature in the fourth feature map, determining the similarity between the fourth feature and each mean feature in the medical image feature library; determining the similarity statistics corresponding to the fourth feature, wherein the statistics include a statistical description of the similarity; arranging the statistics corresponding to each fourth feature according to the position of each fourth feature in the fourth feature map to obtain a similarity description map; performing clustering processing on the statistics in the similarity description map to obtain at least one statistical cluster; determining a similarity threshold based on the at least one statistical cluster; and determining pseudo-segmentation labels based on the similarity threshold and the similarity description map. The explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0125] See Figure 7 The method in this embodiment may specifically include the following steps:

[0126] S501. Obtain a pre-built original segmentation model, as well as multiple sets of first training samples, multiple sets of second training samples, and multiple third medical images. Each set of first training samples includes a first medical image containing a region of interest and a target segmentation label for the region of interest in the first medical image. Each set of second training samples includes a second medical image containing a region of interest. None of the multiple third medical images contain a region of interest. The original segmentation model includes a second encoder and a first segmentation decoder.

[0127] S502. Obtain the feature library construction model to be trained, wherein the feature library construction model includes at least a first encoder and the second encoder has the same structure as the first encoder.

[0128] S503. For each of the multiple third medical images, input the third medical image into the first encoder to obtain the first feature map.

[0129] S504. Perform clustering on the first feature in the first feature map to obtain at least one feature cluster.

[0130] S505. For each feature cluster in at least one feature cluster, determine the mean feature of the first feature in the feature cluster, and store the mean feature in the intermediate image feature library.

[0131] S506. Based on the mean feature, train the feature library construction model and update the feature library construction model according to the training results.

[0132] S507. Return to execute S503, and if the feature library construction model training is completed, use the currently obtained intermediate image feature library as the medical image feature library.

[0133] S508. Update the currently obtained first encoder to the first encoder.

[0134] S509. Update the parameters of the second encoder according to the parameters of the first encoder, and freeze the parameters of the second encoder.

[0135] S510. For each group of first training samples in the multiple groups of first training samples, input the first medical image in the first training sample into the second encoder to obtain the third feature map, and input the third feature map into the first segmentation decoder to obtain the first output result of the first segmentation decoder.

[0136] S511. For each group of second training samples in the multiple groups of second training samples, input the second medical image in the second training sample into the second encoder to obtain the fourth feature map, and input the fourth feature map into the first segmentation decoder to obtain the second output result of the first segmentation decoder.

[0137] S512. For each fourth feature in the fourth feature map, determine the similarity between the fourth feature and each mean feature in the medical image feature library.

[0138] Among them, similarity can characterize the degree of similarity between the fourth feature and the mean feature; similarity can be determined by any index that can evaluate the degree of similarity, such as similarity can be obtained by entropy or variance, etc., without specific limitations.

[0139] In this embodiment of the invention, for each fourth feature in the fourth feature map, the similarity between the fourth feature and each mean feature in the medical image feature library can be determined. That is, the fourth feature is compared with each mean feature in the medical image feature library, and the similarity is determined.

[0140] S513. Determine the similarity statistics corresponding to the fourth feature, wherein the statistics include a statistical description of the similarity.

[0141] Among them, the statistic is a general measure that can characterize the similarity between the fourth feature and each mean feature in the medical image feature library.

[0142] In this embodiment of the invention, a statistical measure of similarity corresponding to the fourth feature can be determined. For example, at least one of the statistical descriptions such as the mean, standard deviation, median, and kurtosis value of the similarity corresponding to the fourth feature can be determined as a statistical measure.

[0143] For example, a similarity sequence can be obtained based on the similarity between the fourth feature and each mean feature in the medical image feature library. That is, the similarity between the fourth feature and each mean feature in the medical image feature library is arranged into a sequence, and the similarity statistics included in the similarity sequence are calculated.

[0144] S514. Arrange the statistics corresponding to each fourth feature according to the position of each fourth feature in the fourth feature map to obtain the similarity description map.

[0145] Among them, the similarity description map is a description map that can reflect the similarity between each fourth feature in the four-feature map and the mean feature in the medical image feature library.

[0146] In this embodiment of the invention, the statistics corresponding to each fourth feature can be arranged according to the position of each fourth feature in the fourth feature map. That is, the statistics corresponding to each fourth feature can be arranged into a similarity description map according to the positional relationship of the fourth features in the fourth feature map. For example, when the fourth feature map is a three-dimensional feature map of size C×H×W, the size of each fourth feature in the fourth feature map is C×1×1. The statistics corresponding to each fourth feature are arranged according to the position of each fourth feature in the fourth feature map to obtain a similarity description map of size H×W.

[0147] S515. Cluster the statistics in the similarity description graph to obtain at least one statistical cluster.

[0148] Among them, a statistical cluster is a cluster whose statistics are similar.

[0149] In this embodiment of the invention, the statistics in the similarity description graph can be clustered according to the distance between the statistics to obtain at least one statistical cluster.

[0150] S516. Determine the similarity threshold based on at least one statistical cluster.

[0151] In this embodiment of the invention, a similarity threshold can be determined based on at least one statistical cluster. This similarity threshold is the threshold for processing the statistics in the similarity description graph to determine the pseudo-segmentation label.

[0152] S517. Determine the pseudo segmentation label based on the similarity threshold and the similarity description map.

[0153] In this embodiment of the invention, the statistics in the similarity description graph can be processed according to the similarity threshold to determine the pseudo segmentation label.

[0154] S518. Based on the first output results and target segmentation labels of the first segmentation decoder corresponding to the multiple sets of first training samples, and the second output results and pseudo segmentation labels of the first segmentation decoder corresponding to the multiple sets of second training samples, the original segmentation model is trained to obtain the target segmentation model.

[0155] S519. Obtain the target medical image for region of interest segmentation.

[0156] S520. Input the target medical image into the target segmentation model, and obtain the segmentation result of the target medical image for the region of interest based on the output of the target segmentation model.

[0157] The technical solution of this invention, through steps such as determining similarity, determining statistics, determining similarity description maps, clustering processing, and determining similarity thresholds, makes the accuracy of the determined pseudo-segmentation labels higher. Thus, based on the more accurate pseudo-segmentation labels, the possibility of the trained target segmentation model segmenting non-interest regions as interest regions is further reduced, thereby improving segmentation performance.

[0158] An optional technical solution involves determining pseudo-segmentation labels based on a similarity threshold and a similarity description map, including: for each statistic in the similarity description map, marking the statistic as 1 if it is greater than or equal to the similarity threshold, and marking it as 0 otherwise; obtaining a label map based on the labels corresponding to each statistic; and performing upsampling processing on the label map to obtain pseudo-segmentation labels.

[0159] In this embodiment of the invention, for each statistic in the similarity description map, if the statistic is greater than or equal to a similarity threshold, it indicates that the region corresponding to the statistic may be a region of interest, and the statistic is marked as 1; if the statistic is less than the similarity threshold, it indicates that the region corresponding to the statistic may be a region of non-interest, and the statistic is marked as 0. A labeled map is obtained based on the labels corresponding to each statistic. This labeled map is an image obtained based on the labels corresponding to each statistic. For example, according to the position corresponding to each statistic, corresponding labels are marked at the corresponding positions on an image to obtain a labeled map. The labeled map is then upsampled to obtain pseudo-segmentation labels. The above technical solution can achieve more accurate determination of pseudo-segmentation labels based on the similarity threshold and the similarity description map.

[0160] Another optional technical solution involves determining a similarity threshold based on at least one statistical cluster, including: for each statistical cluster in the at least one statistical cluster, taking the mean of the statistics in the statistical cluster as the cluster mean of the statistical cluster; determining the total mean and / or total median of the cluster mean corresponding to the at least one statistical cluster; and determining the similarity threshold based on the total mean and / or total median.

[0161] Wherein, the cluster mean is the mean of the statistics in the statistical cluster. The overall mean is the mean of the cluster means corresponding to each statistical cluster. The overall median is the median of the cluster means corresponding to each statistical cluster.

[0162] For example, two statistical clusters can be obtained. For each of the two statistical clusters, the mean of the statistics in the statistical cluster can be used as the cluster mean, resulting in two cluster mean values. The overall mean of the two cluster mean values ​​is then determined. Based on the overall mean value, a similarity threshold is determined.

[0163] For example, three statistical clusters can be obtained. For each of the three statistical clusters, the mean of the statistics in the statistical cluster can be used as the cluster mean, resulting in three cluster mean values. These three cluster mean values ​​are then sorted, and the cluster mean value in the middle is used as the similarity threshold.

[0164] The technical solution of this invention can determine a higher similarity threshold that matches the similarity description map based on at least one statistical cluster.

[0165] Figure 8This is a flowchart of another region of interest segmentation method provided in this embodiment of the invention. This embodiment is based on the above-mentioned technical solutions and optimized. In this embodiment, optionally, the original segmentation model includes at least a third encoder, a fourth encoder, a second segmentation decoder, and a pseudo-segmentation label generation module. The fourth encoder has the same structure as the first encoder. After the feature library construction model training is completed, it further includes: updating the currently obtained first encoder to the first encoder; training the original segmentation model based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature library to obtain a target segmentation model, including: updating the parameters of the third encoder and the fourth encoder according to the parameters of the first encoder, and freezing the parameters of the fourth encoder; for each set of first training samples in the multiple sets of first training samples, inputting the first medical image in the first training sample into the third encoder to obtain a fifth feature map, and inputting the first medical image in the first training sample into the fourth encoder to obtain a sixth feature map; inputting the sixth feature map into the pseudo-segmentation label generation module, so as to obtain a target segmentation model based on the medical image feature library through the pseudo-segmentation label generation module. The sixth feature map is processed to obtain the first segmentation label; the fifth feature map and the first segmentation label are input into the second segmentation decoder to obtain the first output result of the second segmentation decoder; for each of the multiple sets of second training samples, the second medical image in the second training sample is input into the third encoder to obtain the seventh feature map, and the second medical image in the second training sample is input into the fourth encoder to obtain the eighth feature map; the eighth feature map is input into the pseudo segmentation label generation module, so that the eighth feature map is processed based on the medical image feature library to obtain the second segmentation label; the seventh feature map and the second segmentation label are input into the second segmentation decoder to obtain the second output result of the second segmentation decoder; based on the first output result and target segmentation label of the second segmentation decoder corresponding to the multiple sets of first training samples, and the second output result of the second segmentation decoder corresponding to the multiple sets of second training samples, the original segmentation model is trained to obtain the target segmentation model. The explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here.

[0166] See Figure 8 The method in this embodiment may specifically include the following steps:

[0167] S601. Obtain a pre-built original segmentation model, as well as multiple sets of first training samples, multiple sets of second training samples, and multiple third medical images. Each set of first training samples includes a first medical image containing a region of interest and a target segmentation label for the region of interest in the first medical image. Each set of second training samples includes a second medical image containing a region of interest. None of the multiple third medical images contain a region of interest. The original segmentation model includes at least a third encoder, a fourth encoder, a second segmentation decoder, and a pseudo segmentation label generation module.

[0168] The third encoder is used to encode the first and / or second medical images input therein. The fourth encoder is used to encode the first and / or second medical images input therein. The second segmentation decoder is used to decode the feature map and segmentation labels input therein. In this embodiment of the invention, the structure and type of the second segmentation decoder are not specifically limited.

[0169] S602. Obtain the feature library construction model to be trained, wherein the feature library construction model includes at least a first encoder, and the fourth encoder has the same structure as the first encoder.

[0170] In this embodiment of the invention, the fourth encoder has the same structure as the first encoder, so as to facilitate the subsequent updating of the parameters of the fourth encoder based on the parameters of the first encoder after training.

[0171] It should be noted that, in this embodiment of the invention, the other layers in the third encoder, except for the normalization layer, have the same structure as the first encoder. The normalization layer in the third encoder may be the same as or different from the normalization layer in the first encoder. When the normalization layer in the third encoder is the same as the normalization layer in the first encoder, the parameters of the normalization layer in the third encoder are different from those in the first encoder and can be adjusted.

[0172] S603. For each of the multiple third medical images, input the third medical image into the first encoder to obtain the first feature map.

[0173] S604. Perform clustering on the first feature in the first feature map to obtain at least one feature cluster.

[0174] S605. For each feature cluster in at least one feature cluster, determine the mean feature of the first feature in the feature cluster, and store the mean feature in the intermediate image feature library.

[0175] S606. Based on the mean feature, train the feature library construction model and update the feature library construction model according to the training results.

[0176] S607, return to execution 603, and if the feature library construction model training is completed, use the currently obtained intermediate image feature library as the medical image feature library.

[0177] S608. Update the currently obtained first encoder to the first encoder.

[0178] S609. Update the parameters of the third encoder and the fourth encoder according to the parameters of the first encoder, and freeze the parameters of the fourth encoder.

[0179] In this embodiment of the invention, the parameters of the fourth encoder can be updated according to the parameters of the first encoder, and the parameters of the fourth encoder can be frozen. For example, the parameters of the first encoder can be used as the parameters of the fourth encoder, or the first encoder can be directly used as the fourth encoder and the parameters of the fourth encoder can be frozen. That is, the parameters of the fourth encoder are not adjusted in the subsequent process of the target segmentation model.

[0180] In this embodiment of the invention, the parameters of the third encoder can be updated according to the parameters of the first encoder. For example, the parameters of the other layers of the first encoder except the normalization layer can be used as the parameters of the other layers of the fourth encoder except the normalization layer. Alternatively, the other layers of the first encoder except the normalization layer can be directly used as the other layers of the fourth encoder except the normalization layer, and the parameters of the other layers of the fourth encoder except the normalization layer can be frozen. That is, in the subsequent process of the target segmentation model, the parameters of the other layers of the fourth encoder except the normalization layer are not adjusted.

[0181] S610. For each group of first training samples in multiple groups of first training samples, input the first medical image in the first training sample into the third encoder to obtain the fifth feature map, and input the first medical image in the first training sample into the fourth encoder to obtain the sixth feature map.

[0182] In this embodiment of the invention, for each of the multiple sets of first training samples, the first medical image in the first training sample is input into the third encoder to obtain a fifth feature map. This fifth feature map is the feature map output by the third encoder when the first medical image is input into the third encoder. The first medical image in the first training sample is input into the fourth encoder to obtain a sixth feature map. This sixth feature map is the feature map output by the fourth encoder when the first medical image is input into the fourth encoder.

[0183] S611. Input the sixth feature map into the pseudo segmentation label generation module so that the pseudo segmentation label generation module processes the sixth feature map based on the medical image feature library to obtain the first segmentation label.

[0184] The first segmentation label is a pseudo-label obtained based on the sixth feature map and the medical image feature database.

[0185] In this embodiment of the invention, the original segmentation model may further include a pseudo-segmentation label generation module. A sixth feature map can be input into the pseudo-segmentation label generation module, which processes the sixth feature map based on a medical image feature library to obtain a first segmentation label. Specifically, the pseudo-segmentation label generation module in this embodiment is a module capable of determining pseudo-labels based on the input feature map and the medical image feature library. The working principle of this pseudo-segmentation label generation module is the same as that in the above embodiments and will not be described in detail here. It should be noted that in this embodiment of the invention, the trained target segmentation model may include this pseudo-segmentation label generation module.

[0186] S612. Input the fifth feature map and the first segmentation label into the second segmentation decoder to obtain the first output result of the second segmentation decoder.

[0187] In this embodiment of the invention, the parameters of the normalization layer of the second segmentation decoder can also be adjusted according to the first segmentation label input to the second segmentation decoder.

[0188] Optionally, the normalization layer of the second segmentation decoder can be a normalization layer obtained by improving the conventional normalization layer, called a corrected normalization layer, and its specific operation formula is as follows:

[0189]

[0190]

[0191]

[0192] in, This is a normalized layer; This is the input for the regular normalization layer; and These are two learnable parameters used for correction; β and γ are parameters that can be learned by adjusting the segmentation labels. This is calculated after performing a convolution (conv) operation. It's important to note that the segmentation labels are processed before the convolution calculation. A size adjustment operation was performed; the required adjustment factor depends on... The size of the mask after adjustment and... same.

[0193] S613. For each group of second training samples in multiple groups of second training samples, input the second medical image in the second training sample into the third encoder to obtain the seventh feature map, and input the second medical image in the second training sample into the fourth encoder to obtain the eighth feature map.

[0194] In this embodiment of the invention, for each of the multiple sets of second training samples, the second medical image in the second training sample is input into the third encoder to obtain a seventh feature map. This seventh feature map is the feature map output by the third encoder when the second medical image is input into the third encoder. The second medical image in the second training sample is input into the fourth encoder to obtain an eighth feature map. This eighth feature map is the feature map output by the fourth encoder when the second medical image is input into the fourth encoder.

[0195] S614. Input the eighth feature map into the pseudo segmentation label generation module so that the pseudo segmentation label generation module processes the eighth feature map based on the medical image feature library to obtain the second segmentation label.

[0196] The second segmentation label is a pseudo-label obtained based on the eighth feature map and the medical image feature database.

[0197] In this embodiment of the invention, the eighth feature map can be input into the pseudo segmentation label generation module, so that the pseudo segmentation label generation module can process the eighth feature map based on the medical image feature library to obtain the second segmentation label.

[0198] S615. Input the seventh feature map and the second segmentation label into the second segmentation decoder to obtain the second output result of the second segmentation decoder.

[0199] In this embodiment of the invention, the parameters of the normalization layer of the second segmentation decoder can also be adjusted according to the second segmentation label input to the second segmentation decoder.

[0200] S616. Based on the first output results of the second segmentation decoder corresponding to the multiple sets of first training samples and the target segmentation label, and the second output results of the second segmentation decoder corresponding to the multiple sets of second training samples, the original segmentation model is trained to obtain the target segmentation model.

[0201] It should be noted that, in this embodiment of the invention, the method of training the original segmentation model to obtain the target segmentation model and the loss function used are not specifically limited. This is based on the first output results of the second segmentation decoder corresponding to the multiple sets of first training samples and the target segmentation label, and the second output results of the second segmentation decoder corresponding to the multiple sets of second training samples.

[0202] S617. Obtain the target medical image for region of interest segmentation.

[0203] S618. Input the target medical image into the target segmentation model, and obtain the segmentation result of the target medical image for the region of interest based on the output result of the target segmentation model. The target segmentation model is trained based on multiple sets of first training samples, multiple sets of second training samples and medical image feature library.

[0204] The technical solution of this invention, by making the fourth encoder have the same structure as the first encoder, and updating the currently obtained first encoder after the feature library construction model training is completed, can make the features of the non-interest regions of interest in the sixth feature map obtained by inputting the first medical image in the first training sample into the fourth encoder, and the features of the non-interest regions of interest in the eighth feature map obtained by inputting the second medical image in the second training sample into the fourth encoder, more closely match the medical image feature library, thereby improving the accuracy of determining the first segmentation label and the second segmentation label; furthermore, by allowing the first segmentation label and the second segmentation label to participate in the model training, while reducing the possibility that the trained target segmentation model will segment non-interest regions as regions of interest and improving segmentation performance, it can also reduce the demand for medical images with regions of interest and reduce the labeling cost of medical images containing regions of interest.

[0205] Figure 9This is a flowchart of another region of interest segmentation method provided in this embodiment of the invention. This embodiment is based on the above-mentioned technical solutions and optimized. In this embodiment, optionally, the original segmentation model further includes a third segmentation decoder; after inputting the first medical image in the first training sample into the third encoder to obtain the fifth feature map, it further includes: inputting the fifth feature map into the third segmentation decoder to obtain the first output result of the third segmentation decoder; after inputting the second medical image in the second training sample into the third encoder to obtain the seventh feature map, it further includes: inputting the seventh feature map into the third segmentation decoder to obtain the second output result of the third segmentation decoder; training the original segmentation model according to the first output results of the second segmentation decoder corresponding to multiple sets of first training samples and the target segmentation label, and the second output results of the second segmentation decoder corresponding to multiple sets of second training samples, to obtain the target segmentation model, including: training the original segmentation model according to the first output results of the second segmentation decoder corresponding to multiple sets of first training samples, the first output results of the third segmentation decoder and the target segmentation label, and the second output results of the second segmentation decoder corresponding to multiple sets of second training samples and the second output results of the third segmentation decoder, to obtain the target segmentation model. The explanations of terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0206] See Figure 9 The method in this embodiment may specifically include the following steps:

[0207] S701. Obtain a pre-built original segmentation model, as well as multiple sets of first training samples, multiple sets of second training samples, and multiple third medical images. Each set of first training samples includes a first medical image containing a region of interest and a target segmentation label for the region of interest in the first medical image. Each set of second training samples includes a second medical image containing a region of interest. None of the multiple third medical images contain a region of interest. The original segmentation model includes at least a third encoder, a fourth encoder, a second segmentation decoder, and a pseudo segmentation label generation module. The original segmentation model also includes a third segmentation decoder.

[0208] The third segmentation decoder is a segmentation decoder that decodes the feature map in the input. In this embodiment of the invention, the structure and type of the third segmentation decoder are not specifically limited.

[0209] S702. Obtain the feature library construction model to be trained, wherein the feature library construction model includes at least a first encoder, and the fourth encoder has the same structure as the first encoder.

[0210] S703. For each of the multiple third medical images, input the third medical image into the first encoder to obtain the first feature map.

[0211] S704. Perform clustering on the first feature in the first feature map to obtain at least one feature cluster.

[0212] S705. For each feature cluster in at least one feature cluster, determine the mean feature of the first feature in the feature cluster, and store the mean feature in the intermediate image feature library.

[0213] S706. Based on the mean feature, train the feature library construction model and update the feature library construction model according to the training results.

[0214] S707, Return to execute S703, and if the feature library construction model training is completed, use the currently obtained intermediate image feature library as the medical image feature library.

[0215] S708: Update the currently obtained first encoder to the first encoder.

[0216] S709. Update the parameters of the third and fourth encoders according to the parameters of the first encoder, and freeze the parameters of the fourth encoder.

[0217] S710. For each group of first training samples in multiple groups of first training samples, input the first medical image in the first training sample into the third encoder to obtain the fifth feature map, and input the first medical image in the first training sample into the fourth encoder to obtain the sixth feature map.

[0218] S711. Input the fifth feature map into the third segmentation decoder to obtain the first output result of the third segmentation decoder.

[0219] S712. Input the sixth feature map into the pseudo segmentation label generation module, so that the pseudo segmentation label generation module processes the sixth feature map based on the medical image feature library to obtain the first segmentation label.

[0220] S713. Input the fifth feature map and the first segmentation label into the second segmentation decoder to obtain the first output result of the second segmentation decoder.

[0221] S714. For each group of second training samples in multiple groups of second training samples, input the second medical image in the second training sample into the third encoder to obtain the seventh feature map, and input the second medical image in the second training sample into the fourth encoder to obtain the eighth feature map.

[0222] S715. Input the seventh feature map into the third segmentation decoder to obtain the second output result of the third segmentation decoder.

[0223] S716. Input the eighth feature map into the pseudo segmentation label generation module so that the pseudo segmentation label generation module processes the eighth feature map based on the medical image feature library to obtain the second segmentation label.

[0224] S717. Input the seventh feature map and the second segmentation label into the second segmentation decoder to obtain the second output result of the second segmentation decoder.

[0225] S718. Based on the first output results of the second segmentation decoder, the first output results of the third segmentation decoder, and the target segmentation label corresponding to the multiple sets of first training samples, and the second output results of the second segmentation decoder and the second output results of the third segmentation decoder corresponding to the multiple sets of second training samples, the original segmentation model is trained to obtain the target segmentation model.

[0226] In this embodiment of the invention, loss can be calculated using the first output results of the second segmentation decoder corresponding to multiple sets of first training samples, the first output results of the third segmentation decoder, the second output results of the second segmentation decoder corresponding to multiple sets of second training samples, the second output results of the third segmentation decoder, and a consistency loss function. Similarly, loss can be calculated using the first output results of the second segmentation decoder corresponding to multiple sets of first training samples, the first output results of the third segmentation decoder, the target segmentation label, and the segmentation loss function to adjust the parameters in the original segmentation model, thereby obtaining the target segmentation model. In this embodiment of the invention, the method of training the original segmentation model based on the first output results of the second segmentation decoder corresponding to multiple sets of first training samples, the first output results of the third segmentation decoder, and the target segmentation label, as well as the second output results of the second segmentation decoder corresponding to multiple sets of second training samples, to obtain the target segmentation model, and the loss function used are not specifically limited.

[0227] S719. Obtain the target medical image for region of interest segmentation.

[0228] S720. Input the target medical image into the target segmentation model, and obtain the segmentation result of the target medical image for the region of interest based on the output of the target segmentation model.

[0229] The technical solution of this invention can be improved by adding a third segmentation decoder. This allows the original segmentation model to be trained based on the first output results of the second segmentation decoder and the first output results of the third segmentation decoder corresponding to multiple sets of first training samples, as well as the target segmentation label and the second output results of the second segmentation decoder and the second output results of the third segmentation decoder corresponding to multiple sets of second training samples. This results in the target segmentation model being trained, thereby improving the segmentation accuracy of the target segmentation model by adding more training factors.

[0230] An optional technical solution includes a target segmentation model comprising a fifth encoder corresponding to a third encoder and a fourth segmentation decoder corresponding to a third segmentation decoder; inputting a target medical image into the target segmentation model and obtaining a segmentation result of the target medical image for the region of interest based on the output of the target segmentation model, comprising: inputting the target medical image into the fifth encoder and inputting the output of the fifth encoder into the fourth segmentation decoder; and obtaining a segmentation result of the target medical image for the region of interest based on the output of the fourth segmentation decoder.

[0231] In this embodiment of the invention, the target segmentation model may include a fifth encoder corresponding to the third encoder, that is, the fifth encoder is the trained third encoder; the target segmentation model may include a fourth segmentation decoder corresponding to the third segmentation decoder, that is, the fourth segmentation decoder is the trained third encoder.

[0232] The technical solution of this invention can perform region of interest segmentation based on a target segmentation model that has been trained and includes a fifth encoder and a fourth segmentation decoder that correspond to some structures in the original segmentation model.

[0233] Another optional technical solution involves a target segmentation model that includes a sixth encoder corresponding to the third encoder, a seventh encoder corresponding to the fourth encoder, a pseudo-segmentation label generation module, and a fifth segmentation decoder corresponding to the second segmentation decoder. The target medical image is input into the target segmentation model, and the segmentation result of the target medical image for the region of interest is obtained based on the output of the target segmentation model. This includes: inputting the target medical image into the sixth encoder and the seventh encoder respectively, and inputting the output of the seventh encoder into the pseudo-segmentation label generation module; inputting the output of the sixth encoder and the output of the pseudo-segmentation label generation module into the fifth segmentation decoder; and obtaining the segmentation result of the target medical image for the region of interest based on the output of the fifth segmentation decoder.

[0234] In this embodiment of the invention, the sixth encoder is the trained third encoder; the seventh encoder is the trained fourth encoder; the target segmentation model may include a pseudo segmentation label generation module; and the fifth segmentation decoder is the trained second segmentation decoder.

[0235] In this embodiment of the invention, the target segmentation model may further include a sixth segmentation decoder, a sixth encoder, a seventh encoder, a pseudo-segmentation label generation module, and a fifth segmentation decoder, corresponding to the third segmentation decoder. The target medical image can be input into the sixth encoder, and the output of the sixth encoder can be input into the sixth segmentation decoder to obtain the output of the sixth segmentation decoder. The target medical image can be input into both the sixth and seventh encoders, and the output of the seventh encoder can be input into the pseudo-segmentation label generation module. The outputs of the sixth encoder and the pseudo-segmentation label generation module can be input into the fifth segmentation decoder to obtain the output of the fifth segmentation decoder. The outputs of the sixth and fifth segmentation decoders can be weighted and averaged, and the resulting weighted average is used as the segmentation result of the target medical image for the region of interest.

[0236] The technical solution of this invention can perform region of interest segmentation based on a trained target segmentation model that includes a sixth encoder, a seventh encoder, a pseudo segmentation label generation module, and a fifth segmentation decoder that correspond to some structures in the original segmentation model.

[0237] To better understand the technical solutions of the above embodiments of the present invention, an optional example is provided herein. For example, see... Figure 10The original segmentation model includes at least a third encoder Enc-new, a fourth encoder Enc, a second segmentation decoder Dec-seg2, a pseudo-segmentation label generation module, a third segmentation decoder Dec-seg1, a fourth encoder Enc, and other layers of the first encoder in the third encoder Enc-new except for the normalization layer. The parameters of the other layers in the third encoder Enc-new except for the normalization layer and the fourth encoder Enc are updated according to the parameters of the first encoder, and the parameters of the fourth encoder Enc and the other layers in the third encoder Enc-new except for the normalization layer are frozen. For each set of first training samples in multiple sets of first training samples, the first medical image L-Data in the first training sample is input into the third encoder Enc-new to obtain the fifth feature map; the fifth feature map is input into the third segmentation decoder Dec-seg1 to obtain the first output result L-Pred1 of the third segmentation decoder; the first medical image L-Data in the first training sample is input into the fourth encoder Enc to obtain the sixth feature map; the sixth feature map is input into the pseudo-segmentation label generation module. In the label generation module, the sixth feature map is processed based on the medical image feature library by the pseudo segmentation label generation module to obtain the first segmentation label; the fifth feature map and the first segmentation label are input into the second segmentation decoder Dec-seg2 to obtain the first output result L-Pred2 of the second segmentation decoder; for each group of second training samples in multiple groups of second training samples, the second medical image U-Data in the second training samples is input into the third encoder Enc-new to obtain the seventh feature map; the seventh feature map is input into the third segmentation decoder Dec-seg1 to obtain the second output result U-Pred1 of the third segmentation decoder; the second medical image U-Data in the second training samples is input into the fourth encoder Enc to obtain the eighth feature map; the eighth feature map is input into the pseudo segmentation label generation module, which processes the eighth feature map based on the medical image feature library to obtain the second segmentation label; the seventh feature map and the second segmentation label are input into the second segmentation decoder Dec-seg2 to obtain the second output result U-Pred2 of the second segmentation decoder;Loss can be calculated using the first output L-Pred1 of the third segment decoder, the target segmentation label L-Label, and the cross-entropy loss function. Similarly, loss can be calculated using the first output L-Pred2 of the second segment decoder, the target segmentation label L-Label, and the cross-entropy loss function. Since there are structural differences between the second and third segment decoders, there should also be differences between the first output L-Pred1 of the third segment decoder and the first output L-Pred2 of the second segment decoder. Therefore, loss can be calculated using the first output L-Pred1 of the third segment decoder, the first output L-Pred2 of the second segment decoder, and consistency loss functions such as the L1 or L2 loss function. Finally, loss can be calculated using the second output U-Pred1 of the third segment decoder, the second output U-Pred2 of the second segment decoder, and consistency loss functions such as the L1 or L2 loss function. It is important to note that the structure of the third encoder, Enc-new, is the same as that of the fourth encoder, Enc. All normalization layers in the third encoder, Enc-new, need to be retrained, but the parameters of the other operational layers are the same as those in the fourth encoder, Enc, and are not updated during training.

[0238] Figure 11 This is a structural block diagram of a region of interest (ROI) segmentation apparatus provided in an embodiment of the present invention. This apparatus is used to execute the ROI segmentation method provided in any of the above embodiments. This apparatus and the ROI segmentation methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the ROI segmentation apparatus can be found in the embodiments of the ROI segmentation methods described above. See also... Figure 11 The device may specifically include: a target segmentation model acquisition module 810 and a segmentation result acquisition module 820.

[0239] Among them, the target segmentation model acquisition module 810 is used to acquire the target medical image to be segmented into the region of interest, and the trained target segmentation model for segmenting the region of interest.

[0240] The segmentation result acquisition module 820 is used to input the target medical image into the target segmentation model and obtain the segmentation result of the target medical image for the region of interest based on the output result of the target segmentation model;

[0241] The target segmentation model is trained based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature database.

[0242] Each of the multiple sets of first training samples includes a first medical image containing a region of interest and a target segmentation label for the region of interest in the first medical image.

[0243] Each of the multiple sets of second training samples includes a second medical image containing a region of interest.

[0244] The medical image feature library is determined based on multiple third-party medical images, none of which contain the region of interest.

[0245] Optionally, the region-of-interest segmentation device may also include the following modules to pre-train the target segmentation model:

[0246] The third medical image acquisition module is used to acquire the pre-built original segmentation model, as well as multiple sets of first training samples, multiple sets of second training samples, and multiple third medical images.

[0247] The medical image feature library acquisition module is used to extract features from multiple third-party medical images to obtain a medical image feature library;

[0248] The target segmentation model acquisition module is used to train the original segmentation model based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature library to obtain the target segmentation model.

[0249] Optionally, based on the above-described device, the medical image feature database acquisition module may include:

[0250] The feature library construction model acquisition unit is used to acquire the feature library construction model to be trained, wherein the feature library construction model includes at least a first encoder;

[0251] The first feature map obtaining unit is used to input the third medical image into the first encoder for each of the multiple third medical images to obtain the first feature map;

[0252] A feature clustering unit is used to perform clustering processing on the first feature in the first feature map to obtain at least one feature cluster.

[0253] The mean feature storage unit is used to determine the mean feature of the first feature in each feature cluster for at least one feature cluster, and store the mean feature in the intermediate image feature library.

[0254] The feature library construction model update unit is used to train the feature library construction model based on the mean feature and update the feature library construction model according to the training results.

[0255] The step repetition unit is used to repeatedly execute the steps for each of the multiple third medical images;

[0256] The medical image feature library serves as a unit, used to take the currently obtained intermediate image feature library as the medical image feature library after the feature library construction model training is completed.

[0257] Optionally, based on the above-described device, the mean feature storage unit may include:

[0258] The mean feature is stored in the sub-unit to obtain the number of training times for building the model in the feature library. If the number of training times exceeds the preset number, the mean feature is stored in the intermediate image feature library.

[0259] Optionally, based on the above-mentioned device, the feature library construction model further includes a first reconstruction decoder and a second reconstruction decoder;

[0260] The feature library builds a model update unit, which may include:

[0261] The first output result is used to obtain a sub-unit, which is used to input the first feature map into the first reconstruction decoder to obtain the output result of the first reconstruction decoder;

[0262] The second feature map obtains sub-units, which are used to replace each first feature in the first feature map corresponding to the mean feature according to the mean feature, so as to obtain the second feature map;

[0263] The second output result is used to obtain a sub-unit, which is used to input the second feature map into the second reconstruction decoder to obtain the output result of the second reconstruction decoder;

[0264] The feature library construction model training subunit is used to train the feature library construction model based on the output results of the third medical image, the first reconstruction decoder, and the second reconstruction decoder.

[0265] Optionally, based on the above-mentioned device, the original segmentation model includes a second encoder and a first segmentation decoder, wherein the second encoder has the same structure as the first encoder;

[0266] The region of interest segmentation device may further include:

[0267] The first encoder first update module is used to update the currently obtained first encoder after the feature library construction and model training are completed.

[0268] The target segmentation model yields modules, including:

[0269] The first parameter freezing unit is used to update the parameters of the second encoder according to the parameters of the first encoder, and freeze the parameters of the second encoder.

[0270] The first output result first obtaining unit is used to input the first medical image in the first training sample into the second encoder for each of the multiple first training samples to obtain the third feature map, and input the third feature map into the first segmentation decoder to obtain the first output result of the first segmentation decoder;

[0271] The second output result first obtaining unit is used to input the second medical image in the second training sample into the second encoder for each of the multiple sets of second training samples to obtain the fourth feature map, and input the fourth feature map into the first segmentation decoder to obtain the second output result of the first segmentation decoder;

[0272] The pseudo-segmentation label determination unit is used to determine pseudo-segmentation labels based on the fourth feature map and the medical image feature library.

[0273] The first target segmentation model obtaining unit is used to train the original segmentation model based on the first output results and target segmentation labels of the first segmentation decoder corresponding to multiple sets of first training samples, and the second output results and pseudo segmentation labels of the first segmentation decoder corresponding to multiple sets of second training samples, to obtain the target segmentation model.

[0274] Optionally, based on the above-described apparatus, the pseudo-segmentation tag determination unit may include:

[0275] The similarity determination subunit is used to determine the similarity between each fourth feature in the fourth feature map and each mean feature in the medical image feature library.

[0276] The statistics determination subunit is used to determine the statistics of similarity corresponding to the fourth feature, wherein the statistics include a statistical description of similarity;

[0277] The similarity description map yields sub-units, which are used to arrange the statistics corresponding to each fourth feature according to the position of each fourth feature in the fourth feature map, thus obtaining the similarity description map;

[0278] The statistical clusters are sub-units used to cluster the statistics in the similarity description graph to obtain at least one statistical cluster.

[0279] A similarity threshold determination subunit is used to determine a similarity threshold based on at least one statistical cluster;

[0280] The pseudo-segmentation label determination subunit is used to determine pseudo-segmentation labels based on similarity thresholds and similarity description maps.

[0281] Optionally, based on the above device, the pseudo-segmentation tag determines the sub-unit, specifically for:

[0282] For each statistic in the similarity description graph, if the statistic is greater than or equal to the similarity threshold, the statistic is marked as 1; otherwise, it is marked as 0.

[0283] Based on the labels corresponding to each statistic, a labeling plot is obtained;

[0284] The labeled image is upsampled to obtain pseudo segmentation labels.

[0285] Optionally, based on the above device, the similarity threshold determination subunit can be specifically used for:

[0286] For each statistical cluster in at least one statistical cluster, the mean of the statistics in the statistical cluster is taken as the cluster mean of the statistical cluster;

[0287] Determine the total mean and / or total median of the cluster means corresponding to at least one statistical cluster;

[0288] The similarity threshold is determined based on the total mean and / or total median.

[0289] Optionally, based on the above-mentioned device, the original segmentation model includes at least a third encoder, a fourth encoder, a second segmentation decoder, and a pseudo segmentation label generation module, wherein the fourth encoder has the same structure as the first encoder;

[0290] The region of interest segmentation device may further include:

[0291] The first encoder second update module is used to update the currently obtained first encoder after the feature library construction and model training are completed;

[0292] The target segmentation model can be used to obtain modules, which may include:

[0293] The second parameter freezing unit is used to update the parameters of the third encoder and the fourth encoder according to the parameters of the first encoder, and to freeze the parameters of the fourth encoder.

[0294] The sixth feature map obtaining unit is used to input the first medical image in the first training sample into the third encoder for each of the multiple first training samples to obtain the fifth feature map, and input the first medical image in the first training sample into the fourth encoder to obtain the sixth feature map.

[0295] The first segmentation label obtaining unit is used to input the sixth feature map into the pseudo segmentation label generation module, so that the pseudo segmentation label generation module processes the sixth feature map based on the medical image feature library to obtain the first segmentation label;

[0296] The first output result second obtaining unit is used to input the fifth feature map and the first segmentation label into the second segmentation decoder to obtain the first output result of the second segmentation decoder;

[0297] The eighth feature map obtaining unit is used to input the second medical image in the second training sample into the third encoder for each of the multiple sets of second training samples to obtain the seventh feature map, and input the second medical image in the second training sample into the fourth encoder to obtain the eighth feature map.

[0298] The second segmentation label obtaining unit is used to input the eighth feature map into the pseudo segmentation label generation module, so that the pseudo segmentation label generation module processes the eighth feature map based on the medical image feature library to obtain the second segmentation label;

[0299] The second output result second obtaining unit is used to input the seventh feature map and the second segmentation label into the second segmentation decoder to obtain the second output result of the second segmentation decoder;

[0300] The second target segmentation model obtaining unit is used to train the original segmentation model based on the first output results of the second segmentation decoder corresponding to multiple sets of first training samples and the target segmentation label, and the second output results of the second segmentation decoder corresponding to multiple sets of second training samples, to obtain the target segmentation model.

[0301] Optionally, based on the above-mentioned device, the original segmentation model also includes a third segmentation decoder;

[0302] The region of interest segmentation device may further include:

[0303] The first output result acquisition module is used to input the first medical image in the first training sample into the third encoder to obtain the fifth feature map, and then input the fifth feature map into the third segmentation decoder to obtain the first output result of the third segmentation decoder.

[0304] The region of interest segmentation device may further include:

[0305] The second output result module is used to input the second medical image in the second training sample into the third encoder to obtain the seventh feature map, and then input the seventh feature map into the third segmentation decoder to obtain the second output result of the third segmentation decoder.

[0306] The second target segmentation model yields units, which may include:

[0307] The target segmentation model obtains sub-units, which are used to train the original segmentation model based on the first output results of the second segmentation decoder, the first output results of the third segmentation decoder, and the target segmentation label corresponding to multiple sets of first training samples, and the second output results of the second segmentation decoder and the second output results of the third segmentation decoder corresponding to multiple sets of second training samples, to obtain the target segmentation model.

[0308] Optionally, based on the above-mentioned device, the target segmentation model includes a fifth encoder corresponding to the third encoder, and a fourth segmentation decoder corresponding to the third segmentation decoder;

[0309] The segmentation result yields module 820, which may include:

[0310] The first output result input unit is used to input the target medical image into the fifth encoder and input the output result of the fifth encoder into the fourth segmentation decoder;

[0311] The first segmentation result obtaining unit is used to obtain the segmentation result of the target medical image for the region of interest based on the output result of the fourth segmentation decoder.

[0312] Optionally, based on the above-mentioned device, the target segmentation model includes a sixth encoder corresponding to the third encoder, a seventh encoder corresponding to the fourth encoder, a pseudo segmentation label generation module, and a fifth segmentation decoder corresponding to the second segmentation decoder.

[0313] The segmentation result yields module 820, which may include:

[0314] The second output result input unit is used to input the target medical image into the sixth encoder and the seventh encoder respectively, and input the output result of the seventh encoder into the pseudo segmentation label generation module;

[0315] The third output result input unit is used to input the output result of the sixth encoder and the output result of the pseudo segmentation label generation module into the fifth segmentation decoder;

[0316] The second segmentation result obtaining unit is used to obtain the segmentation result of the target medical image for the region of interest based on the output result of the fifth segmentation decoder.

[0317] The technical solution of this invention involves acquiring a target medical image for region of interest (ROI) segmentation via a target segmentation model acquisition module, and a trained target segmentation model for ROI segmentation. A segmentation result acquisition module inputs the target medical image into the target segmentation model and obtains the segmentation result of the target medical image for the ROI based on the output of the target segmentation model. The target segmentation model is trained based on multiple sets of first training samples, multiple sets of second training samples, and a medical image feature library. Each set of first training samples includes a first medical image containing the ROI and a target segmentation label for the ROI. Each set of second training samples includes a second medical image containing the ROI. The medical image feature library is determined based on multiple third medical images, none of which contain the ROI. This device trains the model based on third medical images that do not contain the ROI, achieving sufficient learning of the third medical images and expanding and improving the features of non-ROI regions. This reduces the possibility of segmenting non-ROI regions as ROI regions and improves segmentation performance.

[0318] The region of interest segmentation apparatus provided in the embodiments of the present invention can execute the region of interest segmentation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0319] It is worth noting that in the embodiments of the above-mentioned region of interest segmentation device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0320] Figure 12 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0321] like Figure 12As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0322] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0323] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the region of interest segmentation method.

[0324] In some embodiments, the region of interest segmentation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the region of interest segmentation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the region of interest segmentation method by any other suitable means (e.g., by means of firmware).

[0325] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0326] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0327] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0328] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0329] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0330] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0331] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0332] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method of segmenting a region of interest, characterized by, The method comprises the following steps: obtaining a target medical image to be segmented, and a trained target segmentation model for segmenting the target medical image; inputting the target medical image into the target segmentation model, and obtaining a segmentation result of the target medical image for the region of interest according to an output result of the target segmentation model; wherein the target segmentation model is trained based on a plurality of first training samples, a plurality of second training samples, and a medical image feature library; each of the plurality of first training samples comprises a first medical image containing the region of interest and a target segmentation label of the first medical image for the region of interest; each of the plurality of second training samples comprises a second medical image containing the region of interest; the medical image feature library is determined based on a plurality of third medical images, and the medical image feature library is determined according to a mean feature, wherein the mean feature is obtained by performing feature extraction on the plurality of third medical images, and performing feature clustering processing and mean feature calculation on the extracted features, and the plurality of third medical images do not contain the region of interest.

2. The method of claim 1, wherein, The target segmentation model is pre-trained by the following steps: obtaining a pre-built original segmentation model, and the plurality of first training samples, the plurality of second training samples, and the plurality of third medical images; performing feature extraction on the plurality of third medical images to obtain the medical image feature library; training the original segmentation model based on the plurality of first training samples, the plurality of second training samples, and the medical image feature library to obtain the target segmentation model.

3. The method of claim 2, wherein, The feature extraction on the plurality of third medical images to obtain the medical image feature library comprises: obtaining a feature library construction model to be trained, wherein the feature library construction model comprises at least a first encoder; for each of the plurality of third medical images, inputting the third medical image into the first encoder to obtain a first feature map; performing clustering processing on the first features in the first feature map to obtain at least one feature cluster; for each of the at least one feature cluster, determining a mean feature of the first features in the feature cluster, and storing the mean feature in an intermediate image feature library; training the feature library construction model based on the mean feature, and updating the feature library construction model according to the obtained training result; repeating the step of inputting each of the plurality of third medical images into the first encoder; in the case that the feature library construction model is trained, the current obtained intermediate image feature library is taken as the medical image feature library.

4. The method of claim 3, wherein, The step of storing the mean feature in the intermediate image feature library comprises: obtaining a training number of the feature library construction model, and storing the mean feature in the intermediate image feature library in the case that the training number exceeds a preset number.

5. The method of claim 3, wherein, The feature library construction model further comprises a first reconstruction decoder and a second reconstruction decoder; The training of the feature library construction model based on the mean features comprises: inputting the first feature map into the first reconstruction decoder to obtain an output result of the first reconstruction decoder; replacing each first feature in the first feature map corresponding to the mean feature according to the mean feature to obtain a second feature map; inputting the second feature map into the second reconstruction decoder to obtain an output result of the second reconstruction decoder; training the feature library construction model according to the third medical image, the output result of the first reconstruction decoder and the output result of the second reconstruction decoder.

6. The method of claim 3, wherein, The original segmentation model comprises a second encoder and a first segmentation decoder, and the second encoder has the same structure as the first encoder; after the training of the feature library construction model is completed, the method further comprises: updating the current obtained first encoder to the first encoder; training the original segmentation model based on the plurality of groups of first training samples, the plurality of groups of second training samples and the medical image feature library to obtain the target segmentation model comprises: updating the parameters of the second encoder according to the parameters of the first encoder, and freezing the parameters of the second encoder; for each group of first training samples in the plurality of groups of first training samples, inputting a first medical image in the first training sample into the second encoder to obtain a third feature map, and inputting the third feature map into the first segmentation decoder to obtain a first output result of the first segmentation decoder; for each group of second training samples in the plurality of groups of second training samples, inputting a second medical image in the second training sample into the second encoder to obtain a fourth feature map, and inputting the fourth feature map into the first segmentation decoder to obtain a second output result of the first segmentation decoder; determining a pseudo segmentation label according to the fourth feature map and the medical image feature library; training the original segmentation model according to the first output result of the first segmentation decoder corresponding to each group of first training samples and the target segmentation label, and the second output result of the first segmentation decoder corresponding to each group of second training samples and the pseudo segmentation label to obtain the target segmentation model.

7. The method of claim 6, wherein, The determination of the pseudo segmentation label according to the fourth feature map and the medical image feature library comprises: for each fourth feature in the fourth feature map, determining the similarity of the fourth feature to each mean feature in the medical image feature library; determining a statistical quantity of the similarity corresponding to the fourth feature, wherein the statistical quantity comprises a statistical description of the similarity; arranging the statistical quantity corresponding to each fourth feature according to the position of each fourth feature in the fourth feature map to obtain a similarity description map; performing clustering processing on the statistical quantity in the similarity description map to obtain at least one statistical cluster; determining a similarity threshold according to the at least one statistical cluster; Determine a pseudo-segmentation label according to the similarity threshold and the similarity description graph.

8. The method of claim 7, wherein, The determining a pseudo-segmentation label according to the similarity threshold and the similarity description graph comprises: For each statistic in the similarity description graph, if the statistic is greater than or equal to the similarity threshold, mark the statistic as 1, otherwise mark the statistic as 0; Obtain a mark graph according to the mark corresponding to each statistic; Perform up-sampling processing on the mark graph to obtain a pseudo-segmentation label.

9. The method of claim 7, wherein, The determining a similarity threshold according to the at least one statistical cluster comprises: For each statistical cluster in the at least one statistical cluster, take the mean of the statistics in the statistical cluster as the cluster average of the statistical cluster; Determine a total average and / or a total median of the cluster averages corresponding to the at least one statistical cluster respectively; Determine a similarity threshold according to the total average and / or the total median.

10. The method of claim 3, wherein, The original segmentation model at least comprises a third encoder, a fourth encoder, a second segmentation decoder and a pseudo-segmentation label generation module, the fourth encoder has the same structure as the first encoder; After the feature library construction model training is completed, the method further comprises: Update the obtained first encoder as the first encoder; The training the original segmentation model based on the multiple groups of first training samples, the multiple groups of second training samples and the medical image feature library to obtain the target segmentation model comprises: Update the parameters of the third encoder and the fourth encoder according to the parameters of the first encoder, and freeze the parameters of the fourth encoder; For each group of first training samples in the multiple groups of first training samples, input the first medical image in the first training sample into the third encoder to obtain a fifth feature map, and input the first medical image in the first training sample into the fourth encoder to obtain a sixth feature map; Input the sixth feature map into the pseudo-segmentation label generation module, so as to process the sixth feature map based on the medical image feature library through the pseudo-segmentation label generation module to obtain a first segmentation label; Input the fifth feature map and the first segmentation label into the second segmentation decoder to obtain a first output result of the second segmentation decoder; For each group of second training samples in the multiple groups of second training samples, input the second medical image in the second training sample into the third encoder to obtain a seventh feature map, and input the second medical image in the second training sample into the fourth encoder to obtain an eighth feature map; Input the eighth feature map into the pseudo-segmentation label generation module, so as to process the eighth feature map based on the medical image feature library through the pseudo-segmentation label generation module to obtain a second segmentation label; Input the seventh feature map and the second segmentation label into the second segmentation decoder to obtain a second output result of the second segmentation decoder; According to the first output result of the second segmentation decoder corresponding to each of the plurality of groups of first training samples and the target segmentation label, and the second output result of the second segmentation decoder corresponding to each of the plurality of groups of second training samples, the original segmentation model is trained to obtain the target segmentation model.

11. The method of claim 10, wherein, The original segmentation model further comprises a third segmentation decoder; After the first medical image in the first training sample is input into the third encoder to obtain the fifth feature map, the method further comprises: inputting the fifth feature map into the third segmentation decoder to obtain the first output result of the third segmentation decoder; After the second medical image in the second training sample is input into the third encoder to obtain the seventh feature map, the method further comprises: inputting the seventh feature map into the third segmentation decoder to obtain the second output result of the third segmentation decoder; The method of training the original segmentation model to obtain the target segmentation model according to the first output result of the second segmentation decoder corresponding to each of the plurality of groups of first training samples and the target segmentation label, and the second output result of the second segmentation decoder corresponding to each of the plurality of groups of second training samples, comprises: According to the first output result of the second segmentation decoder corresponding to each of the plurality of groups of first training samples, the first output result of the third segmentation decoder, and the target segmentation label, and the second output result of the second segmentation decoder corresponding to each of the plurality of groups of second training samples and the second output result of the third segmentation decoder, the original segmentation model is trained to obtain the target segmentation model.

12. The method of claim 11, wherein, The target segmentation model comprises a fifth encoder corresponding to the third encoder, and a fourth segmentation decoder corresponding to the third segmentation decoder; The method of inputting the target medical image into the target segmentation model and obtaining the segmentation result of the target medical image for the region of interest according to the output result of the target segmentation model, comprises: inputting the target medical image into the fifth encoder and inputting the output result of the fifth encoder into the fourth segmentation decoder; obtaining the segmentation result of the target medical image for the region of interest according to the output result of the fourth segmentation decoder.

13. The method of claim 11, wherein, The target segmentation model comprises a sixth encoder corresponding to the third encoder, a seventh encoder corresponding to the fourth encoder, a pseudo segmentation label generation module, and a fifth segmentation decoder corresponding to the second segmentation decoder; The method of inputting the target medical image into the target segmentation model and obtaining the segmentation result of the target medical image for the region of interest according to the output result of the target segmentation model, comprises: inputting the target medical image into the sixth encoder and the seventh encoder respectively, and inputting the output result of the seventh encoder into the pseudo segmentation label generation module; inputting the output result of the sixth encoder and the output result of the pseudo segmentation label generation module into the fifth segmentation decoder; According to an output result of the fifth segmentation decoder, a segmentation result of the target medical image for the region of interest is obtained.

14. A region of interest segmentation apparatus characterized by, The method comprises the steps of: obtaining a target medical image to be segmented for a region of interest, and a target segmentation model trained for segmenting the region of interest; inputting the target medical image into the target segmentation model, and obtaining a segmentation result of the target medical image for the region of interest according to an output result of the target segmentation model; wherein the target segmentation model is trained based on a plurality of first training samples, a plurality of second training samples, and a medical image feature library; each of the plurality of first training samples comprises a first medical image containing the region of interest and a target segmentation label of the first medical image for the region of interest; each of the plurality of second training samples comprises a second medical image containing the region of interest; the medical image feature library is determined based on a plurality of third medical images, and is determined according to a mean feature, wherein the mean feature is obtained by performing feature extraction on the plurality of third medical images, performing feature clustering processing on the extracted features, and performing mean feature calculation, and the plurality of third medical images do not contain the region of interest.

15. An electronic device, comprising: The method comprises the steps of: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the region of interest segmentation method according to any one of claims 1-13.

16. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the region of interest segmentation method according to any one of claims 1-13 when executed.

Images