A method and system for automatically delineating pancreatic cancer target area

Abstract

A method and system for automatic delineation of pancreatic cancer target area, belonging to the technical field of target area delineation, solves the problem of low accuracy of automatic delineation in the prior art. The method comprises: obtaining plain scan CT images, enhanced CT images and corresponding labels of pancreatic cancer patients to construct a training sample set; training the constructed organ at risk segmentation model and imaging omics model based on the training sample set; obtaining a trained target area segmentation model based on the training sample set, the trained organ at risk segmentation model and the imaging omics model to train the constructed target area segmentation model; obtaining the organ at risk segmentation result and tumor distribution probability prediction result of the patient to be delineated based on the trained organ at risk segmentation model and the trained imaging omics model, inputting the enhanced CT image, the organ at risk segmentation result and the tumor distribution probability prediction result into the trained target area segmentation model, and obtaining the pancreatic cancer target area delineation result. Efficient and accurate automatic delineation of pancreatic cancer target area is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of target area delineation, and in particular to a method and system for automatically delineating a pancreatic cancer target area. Background Art

[0002] Pancreatic cancer is a highly malignant tumor with no obvious early symptoms. When it is discovered, it is often in the late stage and the treatment effect is poor. Radiotherapy is one of the main treatment methods for pancreatic cancer, and accurate target area delineation is a key link in radiotherapy. However, there are many difficulties in delineating the target area of ​​pancreatic cancer. The main reasons are: 1. The special location environment of pancreatic cancer: the pancreas is deep in the abdominal cavity, surrounded by many important tissues and organs, such as the stomach, duodenum, spleen, kidneys, etc., which surround the entire pancreas, and their density and morphology are also quite different, resulting in a large shape difference of the pancreas itself and making it difficult to distinguish; 2. The particularity of pancreatic cancer tumors: pancreatic cancer is usually an irregular mass of irregular size, and it will appear in different parts of the pancreas, such as the head and tail of the pancreas. The boundary between it and the pancreas is not clear, which increases the difficulty of delineation; 3. Limitations of single modality imaging: due to the unclear boundaries of pancreatic cancer, it is difficult to directly delineate pancreatic cancer in ordinary plain scan CT, and doctors will also use enhanced CT images for auxiliary delineation.

[0003] With the development of artificial intelligence technologies such as machine learning and deep learning, computer-assisted target delineation methods have begun to be used in clinical practice. In current medical practice, the automatic target delineation effect of pancreatic cancer is generally not as good as that of other types of cancer. This is mainly because the pancreas is located in a special position inside the human body, and the limitations of plain scan CT in imaging capabilities make accurate delineation of pancreatic cancer a challenge.

[0004] In the field of radiotherapy, the use of multimodal images for automatic target delineation is an important technological advancement. It can help doctors determine the location and range of tumors more accurately, thereby improving the effectiveness of treatment. However, traditional multimodal image delineation methods usually input different types of image data into deep learning models for training at the same time. This training method does not fully utilize the unique information contained in each modality image, so the delineation accuracy is not high. Summary of the invention

[0005] In view of the above analysis, the embodiments of the present invention aim to provide a method and system for automatically delineating a pancreatic cancer target area, so as to solve the problem of low accuracy of the existing automatic delineation of a pancreatic cancer target area.

[0006] In one aspect, an embodiment of the present invention provides a method for automatically delineating a pancreatic cancer target region, comprising the following steps:

[0007] Acquire multimodal medical image data and corresponding labels of pancreatic cancer patients to construct a training sample set; the multimodal medical image data includes plain scan CT images and enhanced CT images;

[0008] Constructing an organ-at-risk segmentation model, training the organ-at-risk segmentation model based on the plain scan CT images in the training sample set, and obtaining a trained organ-at-risk segmentation model;

[0009] Constructing a radiomics model, wherein the radiomics model is used to predict tumor distribution probability, and training the radiomics model based on enhanced CT images in a training sample set to obtain a trained radiomics model;

[0010] Constructing a target segmentation model, and training the target segmentation model based on a training sample set, a trained organ-at-risk segmentation model, and a trained imaging omics model to obtain a trained target segmentation model;

[0011] The plain CT images and enhanced CT images of the patients to be delineated are respectively input into the trained OAR segmentation model and the trained imaging omics model to obtain the OAR segmentation results and the tumor distribution probability prediction results. The enhanced CT images, OAR segmentation results and the tumor distribution probability prediction results are input into the trained target segmentation model to obtain the pancreatic cancer target delineation results.

[0012] Based on the further improvement of the above method, the risk organ segmentation model includes:

[0013] The first encoding module includes a plurality of downsampling units connected in sequence, and is used to extract multi-layer semantic features from the sample image;

[0014] The middle layer is used to compress the semantic features extracted by the last downsampling unit of the first encoding module and pass the compressed semantic features to the decoding module;

[0015] A decoding module, comprising a plurality of upsampling units connected in sequence; used to recover the feature map layer by layer based on the semantic features extracted by the first encoding module to obtain a final feature map; the number of downsampling units in the first encoding module is the same as the number of upsampling units in the encoding module and corresponds one to one; an attention module is connected between the downsampling unit and the corresponding upsampling unit, used to perform attention extraction on the semantic features extracted by the downsampling unit and pass it to the corresponding upsampling unit;

[0016] The output module includes a plurality of output channels, each of which is used to predict an organ-at-risk region corresponding to the output channel based on the final feature map.

[0017] Based on the further improvement of the above method, the following formula is used to calculate the training loss of the risk organ segmentation model:

[0018] ;

[0019] ;

[0020] ;

[0021] ;

[0022] in, represents the boundary dice loss, represents the boundary distance loss, N represents the number of pixels of the sample, C represents the number of types of organs at risk, and M represents the number of samples in the current training batch. Indicates whether the i-th pixel of the j-th sample belongs to the k-th type of endangered organ. If so, it is 1, otherwise it is 0. represents the probability that the i-th pixel of the j-th sample predicted by the model belongs to the k-th type of endangered organ, Indicates whether the i-th pixel of the j-th sample belongs to an organ at risk. If so, it is 1, otherwise it is 0. It indicates the probability that the i-th pixel of the j-th sample predicted by the model belongs to an organ at risk. , and represents the weight, Indicates borderline loss of organs at risk, Indicates DICE loss of organs at risk, represents the cross entropy loss of organs at risk.

[0023] Based on the further improvement of the above method, the following formula is used to calculate the boundary distance loss:

[0024] ;

[0025] in, represents the transformation matrix from the boundary of the kth class of organs at risk of the jth sample predicted by the organ at risk segmentation model to the true boundary of the kth class of organs at risk of the jth sample, I represents the identity matrix, Represents the F-norm of the matrix.

[0026] Based on the further improvement of the above method, the following formula is used to calculate the boundary dice loss:

[0027] ;

[0028] in, Indicates whether the i-th pixel of the j-th sample is the boundary of the k-th type of endangered organ. If it is the boundary, it is 1, otherwise it is 0. It represents the probability that the i-th pixel of the j-th sample learned by the model is the boundary of the k-th category of endangered organs.

[0029] Based on the further improvement of the above method, the target segmentation model includes:

[0030] The second encoding module includes a first encoding channel and a second encoding channel, the input of the first encoding channel is the enhanced image of the sample and the segmentation result of the organ at risk, and the input of the second encoding channel is the enhanced image of the sample and the prediction result of the tumor distribution probability; each encoding channel includes a plurality of downsampling units connected in sequence; and is used to perform multi-layer semantic feature extraction on the input image;

[0031] The middle layer is used to fuse the semantic features extracted by the last downsampling unit of the first encoding channel and the second encoding channel, and transmit the fused semantic features to the decoding module;

[0032] A decoding module, comprising a plurality of upsampling units connected in sequence; used to restore the feature map layer by layer based on the semantic features extracted by the second encoding module to obtain a target area prediction feature map; the number of downsampling units of the first encoding module is the same as the number of upsampling units of the encoding module and corresponds one to one;

[0033] An output module is used to predict the target area based on the target area prediction feature map.

[0034] Based on the further improvement of the above method, the loss of the target segmentation model is calculated using the following formula:

[0035] ;

[0036] ;

[0037] ;

[0038] ;

[0039] in, represents the boundary dice loss of the target area, represents the boundary distance loss of the target area, N represents the number of pixels of the sample, and M represents the number of samples in the current training batch. Indicates whether the i-th pixel of the j-th sample is the target area, if it is the target area, it is 1, otherwise it is 0; It indicates the probability that the i-th pixel of the j-th sample predicted by the model is the target area, , and represents the weight, represents the boundary loss of the target area, represents the DICE loss in the target area, represents the cross entropy loss of the target area.

[0040] Based on the further improvement of the above method, the boundary distance loss of the target area is calculated using the following formula:

[0041] ;

[0042] in, represents the transformation matrix from the boundary of the target area of ​​the jth sample predicted by the target area segmentation model to the actual boundary of the target area of ​​the jth sample, I represents the unit matrix, Represents the F-norm of the matrix.

[0043] Based on the further improvement of the above method, the following formula is used to calculate the boundary dice loss of the target area:

[0044] ;

[0045] in, Indicates whether the i-th pixel of the j-th sample is the boundary of the target area. If it is the boundary, it is 1, otherwise it is 0. It represents the probability that the i-th pixel of the j-th sample learned by the model is the boundary of the target area.

[0046] On the other hand, an embodiment of the present invention provides a pancreatic cancer target area automatic delineation system, comprising the following modules:

[0047] A sample construction module is used to obtain multimodal medical image data and corresponding labels of pancreatic cancer patients to construct a training sample set; the multimodal medical image data includes plain scan CT images and enhanced CT images;

[0048] The organ-at-risk segmentation model is used to perform organ-at-risk segmentation;

[0049] Radiomics models are used to predict tumor distribution probability;

[0050] Target segmentation model, used for pancreatic cancer target segmentation;

[0051] Training an organ-at-risk segmentation model based on plain scan CT images in a training sample set to obtain a trained organ-at-risk segmentation model; training an imaging omics model based on enhanced CT images in a training sample set to obtain a trained imaging omics model; constructing a target area segmentation model, and training the target area segmentation model based on the training sample set, the trained organ-at-risk segmentation model, and the trained imaging omics model to obtain a trained target area segmentation model;

[0052] The delineation module is used to input the plain scan CT image and enhanced CT image of the patient to be delineated into the trained OAR segmentation model and the trained imaging genomics model respectively to obtain the OAR segmentation results and the tumor distribution probability prediction results, and input the enhanced CT image, OAR segmentation results and the tumor distribution probability prediction results into the trained target segmentation model to obtain the pancreatic cancer target delineation results.

[0053] Compared with the prior art, the present invention can improve the accuracy of pancreatic cancer tumor delineation by acquiring multimodal medical image data and corresponding labels of pancreatic cancer patients, thereby using the multimodal images of plain scan CT combined with enhanced CT for automatic delineation. By constructing an imaging genomics model and an organ at risk segmentation model, information about the pancreas and organs at risk, namely, tumor distribution probability and organ at risk segmentation information, is obtained, and this information is used as a constraint condition, so as to more comprehensively mine and utilize useful information in multimodal images, thereby improving the accuracy and effect of automatic delineation. In delineation training, the introduction of two constraints at the same time can effectively improve the accuracy of delineation, improve the problem of incorrect target area delineation position caused by the lack of effective constraints in traditional methods, and improve the quality of automatic delineation.

[0054] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. In the entire drawings, the same reference symbols represent the same components;

[0056] Figure 1 A flow chart of a method for automatically delineating a pancreatic cancer target region according to an embodiment of the present invention;

[0057] Figure 2 The block diagram of the automatic delineation system for pancreatic cancer target area according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0058] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0059] A specific embodiment of the present invention discloses a method for automatically delineating a pancreatic cancer target region, such as Figure 1 As shown, the following steps are included:

[0060] S1. Acquire multimodal medical image data and corresponding labels of pancreatic cancer patients to construct a training sample set; the multimodal medical image data includes plain scan CT images and enhanced CT images;

[0061] S2, constructing an organ-at-risk segmentation model, and training the organ-at-risk segmentation model based on the plain scan CT images in the training sample set to obtain a trained organ-at-risk segmentation model;

[0062] S3, constructing an imaging omics model, wherein the imaging omics model is used to predict tumor distribution probability, and training the imaging omics model based on enhanced CT images in a training sample set to obtain a trained imaging omics model;

[0063] S4, constructing a target segmentation model, and training the target segmentation model based on the training sample set, the trained organ-at-risk segmentation model and the trained imaging omics model to obtain a trained target segmentation model;

[0064] S5. Input the plain scan CT image and enhanced CT image of the patient to be delineated into the trained OAR segmentation model and the trained imaging genomics model respectively to obtain the OAR segmentation result and the tumor distribution probability prediction result; input the enhanced CT image, the OAR segmentation result and the tumor distribution probability prediction result into the trained target segmentation model to obtain the pancreatic cancer target delineation result.

[0065] Compared with the prior art, the automatic delineation method of the pancreatic cancer target area provided in this embodiment can improve the delineation accuracy of pancreatic cancer tumors by obtaining multimodal medical image data and corresponding labels of pancreatic cancer patients, thereby using the multimodal images of plain scan CT combined with enhanced CT for automatic delineation. By constructing an imaging omics model and an organ at risk segmentation model, the information of the pancreas and organs at risk, that is, the tumor distribution probability and organ at risk segmentation information, is obtained, and this information is used as a constraint condition, so as to more comprehensively mine and utilize the useful information in the multimodal image, and improve the accuracy and effect of automatic delineation. In the delineation training, the introduction of two constraints at the same time can effectively improve the accuracy of delineation, and improve the problem of target area delineation position error caused by the lack of effective constraints in the traditional method. This method not only improves the quality of automatic delineation, but also expands the traditional automatic delineation training method. With the development of medical technology, the demand for precision medicine is increasing. As a typical precision medical demand, the treatment of pancreatic cancer has a growing demand for accurate target area delineation, so this technology will have a relatively broad application prospect in the field of radiotherapy planning for pancreatic cancer tumors.

[0066] During implementation, before starting model training, the first task is to collect medical image data of pancreatic cancer patients. These data include plain scan images and enhanced images, which provide a key information basis for subsequent analysis and training. In the collected plain scan CT images, doctors or radiologists will be responsible for outlining nearby organs at risk (OAR), including the stomach, liver, small intestine, duodenum, etc., and obtain OAR labels. Similarly, in the enhanced CT images, outline the location of the tumor.

[0067] The plain CT images, enhanced CT images and corresponding OAR labels and target area labels of pancreatic cancer patients constitute a training sample, thereby constructing a training sample set.

[0068] During implementation, the CT values ​​of the images in the training sample set are counted, and the CT values ​​are sorted from small to large, with the 5th percentile of the CT value sequence as the minimum value I min , the 995th percentile is the maximum value I max , normalize the image, the formula is as follows:

[0069] ;

[0070] Among them, I image is the original CT value, and uniformly will be higher than the maximum value I max C T-value Set to I max , will be lower than the minimum value I min The CT value is set to I min , which can improve the stability of model training.

[0071] During implementation, the present invention adopts a staged training deep learning model. First, the organ-at-risk segmentation model and the imaging omics model are trained based on the samples of the training set, that is, the organ-at-risk segmentation model is trained based on the plain CT images of the training samples and the corresponding labels, and then the imaging omics model is trained based on the enhanced CT images of the training samples and the corresponding labels.

[0072] The organs at risk segmented by the organ at risk segmentation model will serve as OAR constraints for subsequent training, which is equivalent to a priori knowledge constraint. By using the labels in the OAR to guide the subsequent target segmentation model, we can identify which areas are OARs and which areas are target areas, thereby further improving the accuracy and reliability of the model. The distribution probability predicted by the imaging omics model describes the possibility of tumors appearing in different locations within the pancreas. It can be used as a constraint for subsequent training to help us better predict the location and size of the tumor.

[0073] Specifically, the risk organ segmentation model adopts an encoding-decoding symmetric structure, which includes:

[0074] The first encoding module includes a plurality of downsampling units connected in sequence, and is used to extract multi-layer semantic features from the sample image;

[0075] The middle layer is used to compress the semantic features extracted by the last downsampling unit of the first encoding module and pass the compressed semantic features to the decoding module;

[0076] A decoding module, comprising a plurality of upsampling units connected in sequence; used to recover the feature map layer by layer based on the semantic features extracted by the first encoding module to obtain a final feature map; the number of downsampling units in the first encoding module is the same as the number of upsampling units in the encoding module and corresponds one to one; an attention module is connected between the downsampling unit and the corresponding upsampling unit, used to perform attention extraction on the semantic features extracted by the downsampling unit and pass it to the corresponding upsampling unit;

[0077] The output module includes a plurality of output channels, each of which is used to predict an organ-at-risk region corresponding to the output channel based on the final feature map.

[0078] During implementation, the endangered organ segmentation model adopts an encoder-decoder symmetrical structure, which can obtain medical image spatial information for image semantic segmentation and improve the accuracy of medical image delineation. At the same time, an attention module is added to the skip link structure. The convolutional attention module can weight the feature map, enhance the weight of important features, and reduce the influence of irrelevant and minor features at the same time, improving the ability to capture important details, allowing the model to pay more attention to the area to be segmented, and improving the accuracy of segmentation.

[0079] During implementation, semantic features are extracted through a first encoding module, which includes four sequentially connected downsampling units, gradually reducing the size of the feature map and increasing the number of channels of the feature map, thereby capturing image features of different scales and improving the accuracy of the segmentation result.

[0080] When implemented, the downsampling unit includes a convolutional layer, an activation function, and a maximum pooling layer connected in sequence. The convolutional layer is used for feature extraction, and each downsampling unit usually contains two consecutive convolution operations; each convolutional layer is followed by an activation function to increase the nonlinear expression ability of the model; finally, downsampling is performed through the maximum pooling layer to reduce the spatial dimension of the image by half, thereby reducing the computational complexity and obtaining more abstract features.

[0081] The last downsampling unit of the first encoding module is connected to the middle layer, which is used to compress and refine the semantic features output by the downsampling unit, while maintaining the semantic information of the features. In implementation, the middle layer is a convolutional layer.

[0082] During implementation, the output module is a convolutional layer used to generate the final OAR segmentation result. The number of channels of the output module corresponds one-to-one to the type of OAR, that is, the first output channel is used to predict the area of ​​the first type of OAR, the second output channel is used to predict the area and of the second type of OAR, and so on.

[0083] Since segmentation errors are more likely to occur near the boundaries, in order to improve the segmentation accuracy of the model, the organ at risk segmentation model not only predicts the area of ​​the OAR, but also predicts the boundary of the OAR, taking into account the OAR boundary loss, thereby improving the prediction accuracy.

[0084] Specifically, the following formula is used to calculate the training loss of the organ at risk segmentation model:

[0085] ;

[0086] ;

[0087] ;

[0088] ;

[0089] in, represents the boundary dice loss, represents the boundary distance loss, N represents the number of pixels of the sample, C represents the number of types of organs at risk, and M represents the number of samples in the current training batch. is whether the i-th pixel of the j-th sample belongs to the k-th type of endangered organ, if yes, it is 1, otherwise it is 0; It represents the probability that the i-th pixel of the j-th sample belongs to the k-th category of endangered organs, and its value range is (0,1). Indicates whether the i-th pixel of the j-th sample belongs to an organ at risk. If so, it is 1, otherwise it is 0. It indicates the probability that the i-th pixel of the j-th sample predicted by the model belongs to an organ at risk. , and Represents weight. Indicates borderline loss of organs at risk, Indicates DICE loss of organs at risk, represents the cross entropy loss of organs at risk.

[0090] Specifically, the boundary dice loss is calculated using the following formula:

[0091] ;

[0092] in, Indicates whether the i-th pixel of the j-th sample is the boundary of the k-th type of endangered organ. If it is the boundary, it is 1, otherwise it is 0. It represents the probability that the i-th pixel of the j-th sample learned by the model is the boundary of the k-th type of endangered organ. are the parameters learned by the model.

[0093] Specifically, the boundary distance loss is calculated using the following formula:

[0094] ;

[0095] in, represents the transformation matrix from the boundary of the kth class of organs at risk of the jth sample predicted by the organ at risk segmentation model to the true boundary of the kth class of organs at risk of the jth sample, I represents the identity matrix, Represents the F-norm of the matrix.

[0096] A multi-label strategy is used in segmentation training. The organs at risk are labeled separately according to their anatomical structure and then trained together. There is no need to train a separate model for each organ.

[0097] Radiomics models can use vector space models (VSM) or other machine learning models.

[0098] When implemented, the input of the radiomics model is to segment the enhanced CT image into several small regions, for example, each layer of the image is segmented into The target area mask is used to identify the target area. ...

[0099] The radiomics model is used to predict the probability of each small area being a tumor area, thereby obtaining a tumor distribution probability map. The probability distribution map gives a 0~1 value to each pixel in each small area. The larger the value, the higher the probability of the presence of a tumor. The probability value of the pixels in the content of the small area is the same. During implementation, cross entropy is used as the loss function of the radiomics model. In this way, we can establish an radiomics model that can predict the probability of tumor distribution. The distribution probability describes the possibility of tumors appearing in different locations in the pancreas. It can be used as a constraint for subsequent training to help us better predict the location and size of the tumor.

[0100] During implementation, the training losses of the organ-at-risk segmentation model and the radiomics model were calculated, and the gradient descent method was used to update the parameters of the organ-at-risk segmentation model and the radiomics model, respectively.

[0101] After the OAR segmentation model and the radiomics model are trained, the target segmentation model is trained based on the training sample set, the trained OAR segmentation model, and the trained radiomics model.

[0102] During implementation, one or more batches of samples are collected, and the plain CT images of the samples are input into the OAR segmentation model to obtain the OAR segmentation results. The enhanced CT images are input into the imaging omics model to obtain the tumor distribution probability prediction results. The OAR segmentation results, tumor distribution probability prediction results, and enhanced CT images are used as inputs of the target segmentation model. Features are extracted for target segmentation. The gold standard (i.e., label) of the target area is combined to calculate the loss, and the parameters of the target segmentation model are updated by back propagation using the gradient descent method.

[0103] Specifically, the target area segmentation model adopts an encoding-decoding symmetric structure, which specifically includes:

[0104] The second encoding module includes a first encoding channel and a second encoding channel, the input of the first encoding channel is the enhanced image of the sample and the segmentation result of the organ at risk, and the input of the second encoding channel is the enhanced image of the sample and the prediction result of the tumor distribution probability; each encoding channel includes a plurality of downsampling units connected in sequence; and is used to perform multi-layer semantic feature extraction on the input image;

[0105] The middle layer is used to fuse the semantic features extracted by the last downsampling unit of the first encoding channel and the second encoding channel, and transmit the fused semantic features to the decoding module;

[0106] A decoding module, comprising a plurality of upsampling units connected in sequence; used to restore the feature map layer by layer based on the semantic features extracted by the second encoding module to obtain a target area prediction feature map; the number of downsampling units of the first encoding module is the same as the number of upsampling units of the encoding module and corresponds one to one;

[0107] An output module is used to predict the target area based on the target area prediction feature map.

[0108] It should be noted that the segmentation result of the organ at risk input by the first coding channel is the set of segmentation results of all types of organs at risk output by the organ at risk segmentation model. The segmentation result of each type of organ at risk is represented by a 0,1 mask, where 1 represents that the pixel belongs to the type of organ at risk, and 0 represents that the pixel does not belong to the type of organ at risk. The tumor distribution probability prediction result input by the second coding channel is the tumor distribution probability prediction result output by the imaging omics model.

[0109] During implementation, the structure of each encoding channel of the second encoding module is the same as that of the first encoding module. The decoding module of the target area segmentation model has the same structure as that of the decoding module of the organ at risk segmentation model.

[0110] During implementation, the semantic features extracted by the last downsampling unit of the first encoding channel and the second encoding channel are fused by splicing.

[0111] The tumor distribution probability obtained by enhanced images and radiomics can tell the target segmentation model which ranges of the pancreas have a higher probability of tumor presence, thereby improving the accuracy of target semantic segmentation. At the same time, in order to further reduce segmentation errors and improve the accuracy of image semantic segmentation, the target segmentation model not only predicts the tumor area, but also predicts the boundary of the target area, taking into account the boundary loss of the target area, thereby improving the prediction accuracy.

[0112] During implementation, the loss of the target segmentation model is calculated using the following formula:

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] in, represents the boundary dice loss of the target area, represents the boundary distance loss of the target area, N represents the number of pixels of the sample, M represents the number of samples in the current training batch, Indicates whether the i-th pixel of the j-th sample is the target area, if it is the target area, it is 1, otherwise it is 0; It indicates the probability that the i-th pixel of the j-th sample predicted by the model is the target area, , and represents the weight. represents the boundary loss of the target area, represents the DICE loss in the target area, represents the cross entropy loss of the target area.

[0118] Specifically, the following formula is used to calculate the boundary dice loss of the target area:

[0119] ;

[0120] in, Indicates whether the i-th pixel of the j-th sample is the boundary of the target area. If it is the boundary, it is 1, otherwise it is 0. Indicates the probability that the i-th pixel of the j-th sample learned by the model is the boundary of the target area. When implemented, are the parameters learned by the model.

[0121] Specifically, the boundary distance loss of the target area is calculated using the following formula:

[0122] ;

[0123] in, represents the transformation matrix from the boundary of the target area of ​​the jth sample predicted by the target area segmentation model to the actual boundary of the target area of ​​the jth sample, I represents the unit matrix, Represents the F-norm of the matrix.

[0124] According to the loss function, the parameters of the target segmentation model are updated until convergence, and the training is terminated to obtain the trained target segmentation model.

[0125] For patients to be delineated, their plain scan CT images and enhanced CT images are input into the trained OAR segmentation model and the trained imaging omics model, that is, the plain scan CT images are input into the OAR segmentation model to obtain the OAR segmentation results, and the enhanced CT images are input into the imaging omics model to obtain the tumor distribution probability prediction results. Then, the OAR segmentation results, tumor distribution probability prediction results and enhanced CT images are input into the target segmentation model to obtain the target segmentation results, that is, the target delineation results.

[0126] A specific embodiment of the present invention discloses a pancreatic cancer target area automatic delineation system, such as Figure 2 As shown, it includes the following modules:

[0127] A sample construction module is used to obtain multimodal medical image data and corresponding labels of pancreatic cancer patients to construct a training sample set; the multimodal medical image data includes plain scan CT images and enhanced CT images;

[0128] The organ-at-risk segmentation model is used to perform organ-at-risk segmentation;

[0129] Radiomics models are used to predict tumor distribution probability;

[0130] Target segmentation model, used for pancreatic cancer target segmentation;

[0131] Training an organ-at-risk segmentation model based on plain scan CT images in a training sample set to obtain a trained organ-at-risk segmentation model; training an imaging omics model based on enhanced CT images in a training sample set to obtain a trained imaging omics model; constructing a target area segmentation model, and training the target area segmentation model based on the training sample set, the trained organ-at-risk segmentation model, and the trained imaging omics model to obtain a trained target area segmentation model;

[0132] The delineation module is used to input the plain scan CT image and enhanced CT image of the patient to be delineated into the trained OAR segmentation model and the trained imaging genomics model respectively to obtain the OAR segmentation results and the tumor distribution probability prediction results, and input the enhanced CT image, OAR segmentation results and the tumor distribution probability prediction results into the trained target segmentation model to obtain the pancreatic cancer target delineation results.

[0133] The above method embodiments and system embodiments are based on the same principle, and their related parts can be used for reference, and can achieve the same technical effect. The specific implementation process refers to the above embodiment, which will not be repeated here.

[0134] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0135] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for automatically delineating a pancreatic cancer target region, characterized in that: The following steps are involved: Acquire multimodal medical image data and corresponding labels of pancreatic cancer patients to construct a training sample set; the multimodal medical image data includes plain scan CT images and enhanced CT images; Constructing an organ-at-risk segmentation model, wherein the organ-at-risk segmentation model adopts an encoder-decoder symmetric structure, and training the organ-at-risk segmentation model based on plain scan CT images in a training sample set to obtain a trained organ-at-risk segmentation model; Constructing a radiomics model, wherein the radiomics model is used to predict tumor distribution probability, and training the radiomics model based on enhanced CT images in a training sample set to obtain a trained radiomics model; Constructing a target segmentation model, wherein the target segmentation model adopts an encoder-decoder symmetric structure, and training the target segmentation model based on a training sample set, a trained organ-at-risk segmentation model, and a trained imaging omics model to obtain a trained target segmentation model; The plain scan CT image and enhanced CT image of the patient to be delineated are respectively input into the trained OAR segmentation model and the trained imaging omics model to obtain the OAR segmentation result and the tumor distribution probability prediction result, and the enhanced CT image, the OAR segmentation result and the tumor distribution probability prediction result are input into the trained target segmentation model to obtain the pancreatic cancer target delineation result; The organ at risk segmentation model includes: The first encoding module includes a plurality of downsampling units connected in sequence, and is used to extract multi-layer semantic features from the sample image; The middle layer is used to compress the semantic features extracted by the last downsampling unit of the first encoding module and pass the compressed semantic features to the decoding module; A decoding module, comprising a plurality of upsampling units connected in sequence; used to recover the feature map layer by layer based on the semantic features extracted by the first encoding module to obtain a final feature map; the number of downsampling units in the first encoding module is the same as the number of upsampling units in the decoding module and corresponds one to one; an attention module is connected between the downsampling unit and the corresponding upsampling unit, used to perform attention extraction on the semantic features extracted by the downsampling unit and transfer it to the corresponding upsampling unit; an output module, comprising a plurality of output channels, each output channel being used to predict an organ-at-risk region corresponding to the output channel based on the final feature map; The target area segmentation model includes: The second encoding module includes a first encoding channel and a second encoding channel, the input of the first encoding channel is the enhanced CT image of the sample and the segmentation result of the organ at risk, and the input of the second encoding channel is the enhanced CT image of the sample and the prediction result of the tumor distribution probability; each encoding channel includes a plurality of downsampling units connected in sequence; and is used to perform multi-layer semantic feature extraction on the input image; The middle layer is used to fuse the semantic features extracted by the last downsampling unit of the first encoding channel and the second encoding channel, and transmit the fused semantic features to the decoding module; A decoding module, comprising a plurality of upsampling units connected in sequence; used to restore the feature map layer by layer based on the semantic features extracted by the second encoding module to obtain a target area prediction feature map; the number of downsampling units in the second encoding module is the same as the number of upsampling units in the decoding module and corresponds one to one; An output module, used for predicting a target area based on the target area prediction feature map; The loss of the target segmentation model is calculated using the following formula: ； ； ； ； in, represents the boundary dice loss of the target area, represents the boundary distance loss of the target area, N represents the number of pixels of the sample, and M represents the number of samples in the current training batch. Indicates whether the i-th pixel of the j-th sample is the target area, if it is the target area, it is 1, otherwise it is 0; It indicates the probability that the i-th pixel of the j-th sample predicted by the target segmentation model is the target area. , and represents the weight, represents the boundary loss of the target area, represents the dice loss of the target area, represents the cross entropy loss of the target area; The boundary distance loss of the target area is calculated using the following formula: ； in, represents the transformation matrix from the boundary of the target area of ​​the jth sample predicted by the target area segmentation model to the actual boundary of the target area of ​​the jth sample, I represents the unit matrix, Represents the F-norm of the matrix.

2. The method for automatic delineation of pancreatic cancer target area according to claim 1, characterized in that: The training loss of the organ at risk segmentation model is calculated using the following formula: ； ； ； ； in, represents the boundary dice loss, represents the boundary distance loss, N represents the number of pixels of the sample, C represents the number of types of organs at risk, and M represents the number of samples in the current training batch. Indicates whether the i-th pixel of the j-th sample belongs to the k-th type of endangered organ. If so, it is 1, otherwise it is 0. represents the probability that the i-th pixel of the j-th sample belongs to the k-th type of organ at risk predicted by the organ at risk segmentation model, Indicates whether the i-th pixel of the j-th sample belongs to an organ at risk. If so, it is 1, otherwise it is 0. represents the probability that the i-th pixel of the j-th sample belongs to an organ at risk, predicted by the organ at risk segmentation model. , and represents the weight, Indicates borderline loss of organs at risk, Indicates dice loss of organs at risk, represents the cross entropy loss of organs at risk.

3. The method for automatic delineation of pancreatic cancer target area according to claim 2, characterized in that: The boundary distance loss is calculated using the following formula: ； in, represents the transformation matrix from the boundary of the kth class of organs at risk of the jth sample predicted by the organ at risk segmentation model to the true boundary of the kth class of organs at risk of the jth sample, I represents the identity matrix, Represents the F-norm of the matrix.

4. The method for automatic delineation of pancreatic cancer target area according to claim 2, characterized in that: The boundary dice loss is calculated using the following formula: ； in, Indicates whether the i-th pixel of the j-th sample is the boundary of the k-th type of endangered organ. If it is the boundary, it is 1, otherwise it is 0. It represents the probability that the i-th pixel of the j-th sample learned by the OAR segmentation model is the boundary of the k-th OAR.

5. The method for automatic delineation of pancreatic cancer target area according to claim 1, characterized in that: The following formula is used to calculate the boundary dice loss of the target area: ； in, Indicates whether the i-th pixel of the j-th sample is the boundary of the target area. If it is the boundary, it is 1, otherwise it is 0. It represents the probability that the i-th pixel of the j-th sample learned by the target segmentation model is the boundary of the target area.

6. A pancreatic cancer target area automatic delineation system, characterized in that: Includes the following modules: A sample construction module is used to obtain multimodal medical image data and corresponding labels of pancreatic cancer patients to construct a training sample set; the multimodal medical image data includes plain scan CT images and enhanced CT images; An organ-at-risk segmentation model construction module is used to construct an organ-at-risk segmentation model, and the organ-at-risk segmentation model is used to perform organ-at-risk segmentation; the organ-at-risk segmentation model adopts an encoder-decoder symmetric structure; An imaging omics model construction module, used to construct an imaging omics model, wherein the imaging omics model is used to predict tumor distribution probability; A target segmentation model construction module is used to construct a target segmentation model, wherein the target segmentation model is used to perform pancreatic cancer target segmentation; the target segmentation model adopts an encoder-decoder symmetric structure; A training module is used to train an organ-at-risk segmentation model based on plain scan CT images in a training sample set to obtain a trained organ-at-risk segmentation model; to train an imaging omics model based on enhanced CT images in a training sample set to obtain a trained imaging omics model; and to construct a target segmentation model, and to train the target segmentation model based on the training sample set, the trained organ-at-risk segmentation model and the trained imaging omics model to obtain a trained target segmentation model; A delineation module is used to input the plain scan CT image and the enhanced CT image of the patient to be delineated into the trained OAR segmentation model and the trained imaging omics model respectively to obtain the OAR segmentation result and the tumor distribution probability prediction result, and input the enhanced CT image, the OAR segmentation result and the tumor distribution probability prediction result into the trained target segmentation model to obtain the pancreatic cancer target delineation result; The organ at risk segmentation model includes: The first encoding module includes a plurality of downsampling units connected in sequence, and is used to extract multi-layer semantic features from the sample image; The middle layer is used to compress the semantic features extracted by the last downsampling unit of the first encoding module and pass the compressed semantic features to the decoding module; A decoding module, comprising a plurality of upsampling units connected in sequence; used to recover the feature map layer by layer based on the semantic features extracted by the first encoding module to obtain a final feature map; the number of downsampling units in the first encoding module is the same as the number of upsampling units in the decoding module and corresponds one to one; an attention module is connected between the downsampling unit and the corresponding upsampling unit, used to perform attention extraction on the semantic features extracted by the downsampling unit and transfer it to the corresponding upsampling unit; an output module, comprising a plurality of output channels, each output channel being used to predict an organ-at-risk region corresponding to the output channel based on the final feature map; The target area segmentation model includes: The second encoding module includes a first encoding channel and a second encoding channel, the input of the first encoding channel is the enhanced CT image of the sample and the segmentation result of the organ at risk, and the input of the second encoding channel is the enhanced CT image of the sample and the prediction result of the tumor distribution probability; each encoding channel includes a plurality of downsampling units connected in sequence; and is used to perform multi-layer semantic feature extraction on the input image; The middle layer is used to fuse the semantic features extracted by the last downsampling unit of the first encoding channel and the second encoding channel, and transmit the fused semantic features to the decoding module; A decoding module, comprising a plurality of upsampling units connected in sequence; used to restore the feature map layer by layer based on the semantic features extracted by the second encoding module to obtain a target area prediction feature map; the number of downsampling units in the second encoding module is the same as the number of upsampling units in the decoding module and corresponds one to one; An output module, used for predicting a target area based on the target area prediction feature map; The loss of the target segmentation model is calculated using the following formula: ； ； ； ； in, represents the boundary dice loss of the target area, represents the boundary distance loss of the target area, N represents the number of pixels of the sample, and M represents the number of samples in the current training batch. Indicates whether the i-th pixel of the j-th sample is the target area, if it is the target area, it is 1, otherwise it is 0; It indicates the probability that the i-th pixel of the j-th sample predicted by the target segmentation model is the target area. , and represents the weight, represents the boundary loss of the target area, represents the dice loss of the target area, represents the cross entropy loss of the target area; The boundary distance loss of the target area is calculated using the following formula: ； in, represents the transformation matrix from the boundary of the target area of ​​the jth sample predicted by the target area segmentation model to the actual boundary of the target area of ​​the jth sample, I represents the unit matrix, Represents the F-norm of the matrix.

Images