A small sample segmentation system for CT image tumors based on background class recognition

Abstract

The present invention discloses a small-sample tumor segmentation system for CT images based on background class recognition. The system comprises a data acquisition module, a region of interest delineation module, a small-sample tumor segmentation model training module, and a model testing module. This system addresses the problem of traditional meta-learning frameworks incorrectly segmenting tumor background classes as tumors in small-sample medical image segmentation tasks. By introducing a tumor-background class recognition branch, the system utilizes the output of the tumor-background class segmentation network to guide tumor segmentation and prevent incorrect segmentation of the tumor background class. An integrated optimizer is designed to better integrate the output of the tumor background class and tumor segmentation, producing more accurate segmentation results. Furthermore, to address the spatial unevenness of background in medical image data, the system uses support set labels and label inverses to guide the learner in extracting foreground and background features, and separately calculates feature prototypes, fully utilizing foreground and background information for segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a CT image tumor small sample segmentation system based on background class recognition. Background Art

[0002] Nasopharyngeal carcinoma (NPC) is a highly prevalent malignant tumor, with the highest incidence among otolaryngological malignancies. Failure to strictly control the radiation target volume during radiotherapy for this type of tumor can result in significant damage to normal organs and tissues, adversely affecting patient health. CT images are the standard image resource for target delineation in radiotherapy. Manual delineation of the target volume by experienced physicians clearly illustrates the location of each target, ensuring that the radiation treatment area is strictly contained within the target volume, minimizing or preventing damage to normal organs. Organ segmentation in CT images plays a crucial role in clinical practice. However, manual delineation is labor-intensive and time-consuming, and due to varying experience among physicians, there is significant variability between manual segmentations, impacting delineation accuracy. Therefore, designing a system for automatic segmentation of NPC tumors in CT images is of great practical significance. This system not only frees up hands and reduces physician workload, but also significantly accelerates delineation time compared to manual delineation, enabling timely tumor segmentation and saving valuable patient treatment time.

[0003] In medical image segmentation tasks, especially tumor segmentation, researchers often face a shortage of high-quality annotated data for medical images, or even a lack of general annotated data. This is the small sample problem in medical image segmentation research. Furthermore, the complex anatomical structure leads to spatially uneven backgrounds in medical image data, which poses a significant challenge to small-sample learning, which already faces data shortages. Given that the annotation quality of cases with tumor annotations is insufficient, and the number of annotated slices is insufficient to support supervised learning, we need to employ small-sample learning to address the automatic segmentation of nasopharyngeal carcinoma tumors.

[0004] Existing CT image tumor segmentation systems based on small-sample learning are generally based on metric learning. They use a meta-learning structure to learn feature prototype extraction from labeled data, which is easier to obtain and more abundant, and then migrate to unlabeled data, which is more difficult to obtain and more scarce, for segmentation. However, they still have the following shortcomings:

[0005] 1. Although previous meta-learning frameworks have a certain degree of generalization, these models are still more likely to identify new categories appearing in the test set as known categories that have been learned multiple times in the training set due to the different data distributions in the training set and the test set, which is not conducive to the generalization of the model.

[0006] 2. The distinction between tumors and background tissues in CT images is low, and the boundaries between them are fuzzy and difficult to define. It requires professional doctors to make accurate judgments. This has led to the inability of previous meta-learning models to distinguish tumors from other normal tissues well, resulting in low accuracy of the final segmentation results.

[0007] In summary, in the CT image tumor segmentation system based on small sample learning, how to avoid data bias that leads to reduced model generalization performance and how to clearly define the foreground and background features of the tumor that are difficult to distinguish are key issues that need to be solved urgently. Summary of the Invention

[0008] The purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology. A small sample segmentation system for CT image tumors based on background class recognition is proposed. A background class learning branch is additionally added to the traditional meta-learning framework. In conjunction with an integrated optimizer, auxiliary information can be provided during tumor segmentation to prevent the tumor from being mistakenly segmented as a background class target of the tumor. A feature prototype extraction layer is designed, and support set labels are used to guide the learner to extract foreground and background features, and feature prototypes are calculated separately to make full use of foreground and background information for segmentation.

[0009] To achieve the above objectives, the present invention provides a technical solution: a CT image tumor small sample segmentation system based on background class recognition, comprising:

[0010] The data acquisition module is used to collect CT image data sets, randomly divide the CT image data sets into training sets and test sets, and then divide the support set and query set for small sample learning based on the training set and test set.

[0011] A region of interest delineation module is used to delineate the region of interest in the CT image data set and use the delineated image data as a reference label;

[0012] The small tumor sample segmentation model training module is used to iteratively train the small tumor sample segmentation model using image data in the training set. The small tumor sample segmentation model includes a shared encoder S, a segmentation network A, a segmentation network B, and an integrated optimizer O. The shared encoder S is composed of the encoder part of a pyramid network P with multi-scale global information fusion, the segmentation network A is composed of a meta-learning network M for modeling foreground and background information, the segmentation network B is composed of the decoder part of the pyramid network P with multi-scale global information fusion, and the integrated optimizer O is composed of two convolutional connection structures. During training, the support set and query set features are first extracted through the shared encoder S. Then, supervised learning is used to optimize only the loss function of the segmentation network B for training to identify background class targets of the tumor. After training is completed, the parameters of the shared encoder S and the segmentation network B are fixed, and the loss function of the segmentation network A is optimized using the image data in the training set to complete the segmentation of the tumor background class. The output of the segmentation network B is used in the integrated optimizer O to provide auxiliary information for the segmentation result of the segmentation network A to obtain a more accurate tumor background class segmentation result.

[0013] The model testing module is used to input all sample CT images in the test set into the tumor small sample segmentation model of each iterative training round obtained by the tumor small sample segmentation model training module. In the testing phase, the segmentation network A segments the tumor, and the segmentation network B provides auxiliary information for tumor background identification. The integrated optimizer O is used to prevent the segmentation network A from incorrectly segmenting the tumor background as a tumor. The model with the highest tumor segmentation accuracy among the models obtained in each round of iterative training is selected as the optimal segmentation model, and the final complete tumor binary segmentation image is output.

[0014] Furthermore, the CT image dataset includes CT images containing nasopharyngeal carcinoma tumors and multiple organs at risk of nasopharyngeal carcinoma, wherein the organs at risk of nasopharyngeal carcinoma belong to the background class targets of the nasopharyngeal carcinoma tumor; the data acquisition module uses the software Matlab to read the dicom file containing the CT image data of the organs at risk of nasopharyngeal carcinoma, and then makes the read data into images to obtain the corresponding CT image dataset; the CT image dataset is randomly sorted, and the data in the randomly sorted dataset is divided into a training set and a test set in proportion, wherein the CT images in the training set only contain information on organs at risk of nasopharyngeal carcinoma, and the CT images in the test set contain both nasopharyngeal carcinoma tumor information and information on organs at risk of nasopharyngeal carcinoma; a 1-way-1-shot method is selected in the support set and query set settings, that is, one category is selected each time in each scenario training on the support set, and the number of samples in each category is 1, and the number of slices in each category on the query set is also 1.

[0015] Furthermore, the region of interest delineation module uses the software Matlab to manually outline the segmentation result data in the dicom file corresponding to the reference image, uses the outlined image data as a reference label, and produces a corresponding label map as the target area of ​​the segmentation system.

[0016] Furthermore, the tumor small sample segmentation model training module divides the training set into n batches of data with a size of m and trains them batch by batch; in the training phase, the CT images on the support set of the current batch are trained with I support_train Indicates that the CT images on the query set are represented by I query_train The label of the manually segmented tumor background class is represented by gtb; the training process includes the following steps:

[0017] 1) Will I support_train with I query_train Input the shared encoder S to obtain feature information F support With F query , where the calculation result of the characteristic information is determined by the following expression:

[0018] F support =S(I support_train )

[0019] F query =S(I query_train )

[0020] 2) F query Input segmentation network B to obtain tumor background segmentation results Calculate the corresponding loss term of the segmentation network B And update the parameters of the segmentation network B; the segmentation result is determined by the following expression:

[0021]

[0022] The corresponding loss term of the segmentation network B It is composed of multi-class cross entropy loss and is determined by the following formula:

[0023]

[0024] in, The smaller it is, the higher the accuracy of the segmentation network B;

[0025] 3) Replace F in step 1) support With F query Input segmentation network A to obtain the foreground and background segmentation results of tumor background class targets and The segmentation result is determined by the following expression:

[0026]

[0027]

[0028] 4) Change the and Input the integrated optimizer O to get the optimized segmentation result The segmentation result is determined by the following expression:

[0029]

[0030] The corresponding loss term of the segmentation network A It is composed of multi-class cross entropy loss and is determined by the following formula:

[0031]

[0032] Where, The smaller it is, the higher the accuracy of the segmentation network A;

[0033] When training the shared encoder S, segmentation network A and segmentation network B, the segmentation network B is first trained alone until convergence, and then the parameters of the shared encoder S and segmentation network B are fixed, and the segmentation network A is added to train the entire tumor small sample segmentation model.

[0034] Furthermore, the shared encoder S is composed of the encoder part of a pyramid network P that fuses multi-scale global information. The network P is a pyramid network embedded with an SCP (SE-ConnectionPyramid) module that extracts multi-scale information and integrates global information.

[0035] The encoder part consists of four encoder submodules with the same structure; each encoder submodule contains two consecutive 2D convolutional layers and one maximum pooling layer; each 2D convolutional layer is followed by an instance normalization layer, and ReLU is used as the activation function; wherein, the convolution kernel size of each convolutional layer is set to 3×3, the stride is set to 1, and the same convolution method is used, and the padding is set to 1, the purpose is to expand the network receptive field while strengthening the extraction of image edge information, ensuring that the size of the feature map remains unchanged during the convolution process, making the subsequent size change calculation of the network more convenient; the maximum pooling layer is placed after the second 2D convolutional layer, the purpose is to extract the main information of the feature map and reduce the size of the feature map; in addition to being transmitted to the encoder submodule of the next level, the output of each encoder submodule will also be transmitted to the SCP module at the jump connection position, and will be spliced ​​and fused with the output of the SCP module from the next level in the channel dimension to achieve global multi-scale information fusion. In addition to being the input of the SCP module of the previous level, the fused information will also be input into the decoder submodule of the corresponding level to obtain more accurate segmentation results;

[0036] The SCP module consists of a concatenation layer, a convolution layer, and an attention mechanism module. As the encoder performs multiple convolution and pooling operations on the input CT image, the semantic information of the obtained feature map information is gradually enhanced, and the scale of the extracted features is gradually increased. The feature map output by the encoder submodule at the same level as an SCP module is concatenated with the feature map output by the SCP module at the next level in the channel dimension through a concatenation operation at the concatenation layer, and the concatenated feature information is fused through a convolution layer, thereby obtaining a multi-scale information flow. The information output by the encoder submodule at the deepest level of the pyramid network P only passes through an attention mechanism module that is the same as that in the SCP module.

[0037] The multi-scale information flow obtained next passes through the attention mechanism module, and the channel attention mechanism is used to assign higher weights to important features that contribute more to the segmentation task, weaken unimportant feature channels, and thus improve the overall performance of the segmentation network A; the attention mechanism module is composed of a global average pooling layer and two consecutive fully connected layers in sequence, wherein the global average pooling layer is first used to generate channel statistics, and the global spatial information of the feature map is compressed into a channel descriptor, and then the channel statistics are reduced to 1 / 16 of the original dimension in the first fully connected layer and activated by the ReLU function, and then the output of the first fully connected layer is increased to the original dimension in the second fully connected layer, and then a Sigmoid function is used to obtain normalized weights between 0 and 1, and finally the normalized weights are weighted to the features of each channel through the Scale operation.

[0038] Furthermore, the segmentation network A is composed of a meta-learning network M that models foreground and background information, which is a classic network structure used in small sample segmentation tasks of medical images;

[0039] The meta-learning network M consists of two convolutional layers, one upsampling layer, one feature prototype extraction layer and one metric module. After being extracted by the shared encoder S, the feature information F from the support set and the query set is support With F query After convolution and upsampling operations, where the convolution kernel size is 1×1, the dimension size is the same as the support set label M support Match, then use M support With M supportThe inverse code is then ANDed with the feature information after dimension matching, paying more attention to the area of ​​interest to be segmented in the feature matrix, and performing a convolution operation with a convolution kernel size of 3×3 to obtain the foreground feature and background feature matrices respectively. Then, in the feature prototype extraction layer, the mask average pooling method is used to calculate the foreground feature prototype and background feature prototype in the support set respectively. Finally, the foreground and background feature prototypes are respectively input into the measurement module. In the image data of the query set, the cosine distance measurement method is used to measure and classify the foreground and background feature prototypes of the tumor background class targets pixel by pixel, respectively, completing the foreground and background segmentation of the tumor background class targets, and outputting the corresponding foreground segmentation results and background segmentation results.

[0040] Furthermore, the segmentation network B is composed of the decoder part of the pyramid network P with multi-scale global information fusion;

[0041] The decoder part consists of 4 decoder submodules and 1 output layer, wherein the structures of the 4 decoder submodules are the same; the 4 decoder submodules all contain 2 consecutive 2D convolution layers and 1 upsampling layer, arranged in order; each 2D convolution layer is followed by an instance normalization layer, and ReLU is used as the activation function; wherein, the convolution kernel size of each convolution layer is set to 3×3, the step size is set to 1, and the same convolution method is used, and the padding is set to 1. While expanding the network receptive field, the extraction of image edge information is strengthened, ensuring that the size of the feature map remains unchanged during the convolution process, making the calculation of subsequent network size changes more convenient; the upsampling layer After the second 2D convolutional layer, the convolutional layers and upsampling layers are arranged in this order to restore the image size while eliminating the checkerboard effect. The output layer contains two consecutive 2D convolutional layers, which are the same as the convolutional layer settings in the four decoder submodules. The output of each decoder submodule is concatenated with the output of the SCP module in the jump connection from the corresponding depth in the channel dimension, and then input into the next decoder submodule and iterated to the output layer. The operation of feeding the global multi-scale information flow to the decoder through the jump connection can help the network recover the detailed feature information lost during the downsampling process, thereby obtaining a more structurally accurate generation result.

[0042] Furthermore, the integrated optimizer O is composed of two convolutional connection structures; first, the background segmentation result of the tumor background target output by the segmentation network A is spliced ​​with the output result of the segmentation network B in the channel dimension; then, a 1×1 convolution kernel is used to obtain the target background mask guided by the output result of the segmentation network B, and the incorrectly segmented pixels in the background segmentation result of the tumor background target by the segmentation network A are corrected to improve the rough segmentation result of the segmentation network A; then, the target background mask and the foreground segmentation result of the tumor background target output by the segmentation network A are spliced ​​in the channel dimension, and the final refined segmentation result is obtained through a 1×1 convolution.

[0043] Furthermore, the model testing module inputs the CT images of each sample in the test set into the tumor small sample segmentation model training module, generates a tumor small sample segmentation model for each iterative training round, outputs the corresponding nasopharyngeal carcinoma tumor segmentation result, and evaluates the accuracy of the segmentation result obtained for each sample. The model with the best nasopharyngeal carcinoma tumor segmentation result evaluation result among the models obtained in each round of iterative training is selected as the optimal segmentation model;

[0044] In the testing phase, the CT images on the support set of the current batch are represented by I support_test Indicates that the CT images on the query set are represented by I query_test The label of the manually segmented tumor class is represented by gt; the testing process includes the following steps:

[0045] 1) Will I support_test with I query_test Input to the shared encoder S that has completed training and has fixed parameters to obtain feature information F support_test With F query_test , where the calculation result of the characteristic information is determined by the following expression:

[0046] F support_test =S(I support_test )

[0047] F query_test =S(I query_test )

[0048] 2) F query_test Input segmentation network B to obtain tumor background segmentation results The segmentation result is determined by the following expression:

[0049]

[0050] 3) Replace F in step 1) support_test With F query_test Input segmentation network A to obtain the foreground and background segmentation results of the tumor and The segmentation result is determined by the following expression:

[0051]

[0052]

[0053] 4) Change the and Input the integrated optimizer O to get the optimized segmentation result The segmentation result is determined by the following expression:

[0054]

[0055] Segmentation results The output is in the form of a matrix with a channel dimension of 1, which is a binary 0-1 image. The tumor segmentation result is 1, and the tumor background target segmentation result is 0. A complete and clear segmentation image is output.

[0056] In the selection of evaluation indicators, the Dice coefficient is selected as the evaluation indicator, denoted as E Dice , determined by the following formula:

[0057]

[0058] The model with the best tumor segmentation result evaluation result among the models obtained in each round of iterative training is selected, which means that for the models obtained in each round of iterative training, the E calculated by selecting the nasopharyngeal carcinoma tumor segmentation result and the corresponding tumor segmentation label gt Dice The highest model is taken as the optimal segmentation model.

[0059] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0060] 1. To address the problem that traditional meta-learning frameworks mistakenly segment tumor background classes as tumors in small-sample medical image segmentation tasks, we introduced an additional approach to tumor background class identification. During tumor segmentation, we use the output of the tumor background class segmentation network to guide tumor background segmentation and prevent incorrect segmentation of the tumor background class. We also designed an integrated optimizer to better fuse the output results of the tumor background class segmentation network and the tumor segmentation meta-learner to produce more accurate segmentation results.

[0061] 2. In view of the uneven background space of medical image data, a meta-learner is designed to segment the foreground and background information of the target. The support set label and label inverse code are used to guide the learner to extract the foreground and background features, and the feature prototypes are calculated separately to make full use of the foreground and background information for segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is an architectural diagram of the system of the present invention.

[0063] Figure 2 This is a structural diagram of the tumor small sample segmentation model of the system of the present invention.

[0064] Figure 3 It is the structural diagram of the pyramid network P.

[0065] Figure 4 It is the structural diagram of the SCP module in the shared encoder S.

[0066] Figure 5 It is the structural diagram of the segmentation network A.

[0067] Figure 6 It is the structural diagram of the integrated optimizer O. DETAILED DESCRIPTION

[0068] The present invention will be described in further detail below with reference to the embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0069] like Figure 1 As shown, this embodiment discloses a CT image tumor small sample segmentation system based on background class recognition, including: a data acquisition module, a region of interest delineation module, a tumor small sample segmentation model training module and a model testing module.

[0070] The data acquisition module is used to collect a CT image dataset, which includes CT images of nasopharyngeal carcinoma tumors and multiple organs at risk. In this embodiment, the target sample location is concentrated in the head and neck due to pathological reasons. The CT image dataset is randomly divided into a training set and a test set in a 4:1 ratio. The CT images in the training set do not contain tumor information, while the CT images in the test set contain both organ at risk and tumor information.

[0071] The region of interest delineation module is used to delineate the region of interest in the CT image data set. After the radiologist delineates the nasopharyngeal carcinoma tumor target area and the organ at risk area, the tumor and organ at risk area are used as the region of interest, and the information of the region of interest is stored in a dicom file closely related to the CT image data. Matlab software is used to read it and produce a visual label.

[0072] The tumor small sample segmentation model training module divides the training set into n batches of data with a size of m and trains them batch by batch. The specific value of the batch size m can be adjusted according to the actual hardware conditions, mainly depending on the available GPU memory size. In this embodiment, the value of m is 16. In the training stage, the CT images on the support set of the current batch are used as Isupport_train Indicates that the CT images on the query set are represented by I query_train The tumor background class is manually segmented by a physician and represented by gtb. The tumor small sample segmentation model includes a shared encoder S, a segmentation network A, a segmentation network B, and an integrated optimizer O. The shared encoder S is composed of the encoder part of the pyramid network P with multi-scale global information fusion, the segmentation network A is composed of the meta-learning network M for modeling foreground and background information, the segmentation network B is composed of the decoder part of the pyramid network P with multi-scale global information fusion, and the integrated optimizer O is composed of two convolutional connection structures. The training process includes the following steps:

[0073] 1) Will I support_train with I query_train Input the shared encoder S to obtain feature information F support With F query , where the calculation result of the characteristic information is determined by the following expression:

[0074] F support =S(I support_train )

[0075] F query =S(I query_train )

[0076] 2) F query Input segmentation network B to obtain tumor background segmentation results Calculate the corresponding loss term of the segmentation network B And update the parameters of the segmentation network B; the segmentation result is determined by the following expression:

[0077]

[0078] The corresponding loss term of the segmentation network B It is composed of multi-class cross entropy loss and is determined by the following formula:

[0079]

[0080] in, The smaller it is, the higher the accuracy of the segmentation network B;

[0081] 3) Replace F in step 1) support With F query Input segmentation network A to obtain the foreground and background segmentation results of tumor background class targets and The segmentation result is determined by the following expression:

[0082]

[0083]

[0084] 4) Change the and Input the integrated optimizer O to get the optimized segmentation result The segmentation result is determined by the following expression:

[0085]

[0086] The corresponding loss term of the segmentation network A It is composed of multi-class cross entropy loss and is determined by the following formula:

[0087]

[0088] Where, The smaller it is, the higher the accuracy of the segmentation network A.

[0089] When training the shared encoder S, segmentation network A and segmentation network B, the segmentation network B is first trained alone until convergence, and then the parameters of the shared encoder S and segmentation network B are fixed, and the segmentation network A is added to train the entire tumor small sample segmentation model.

[0090] The model testing module inputs the CT images of each sample in the test set into the tumor small sample segmentation model training module, generates a nasopharyngeal carcinoma tumor segmentation model for each iterative training round, outputs the corresponding nasopharyngeal carcinoma tumor segmentation result, and evaluates the accuracy of the segmentation result obtained for each sample. The model with the best nasopharyngeal carcinoma tumor segmentation result evaluation result among the models obtained in each round of iterative training is selected as the optimal segmentation model.

[0091] In the testing phase, the CT images on the support set of the current batch are represented by I support_test Indicates that the CT images on the query set are represented by I query_test The label of the tumor class manually segmented by the physician is represented by gt; the testing process includes the following steps:

[0092] 1) Will I support_test with I query_test Input to the shared encoder S that has completed training and has fixed parameters to obtain feature information F support_test With F query_test , where the calculation result of the characteristic information is determined by the following expression:

[0093] F support_test =S(I support_test )

[0094] F query_test =S(I query_test )

[0095] 2) Fquery_test Input segmentation network B to obtain tumor background segmentation results The segmentation result is determined by the following expression:

[0096]

[0097] 3) Replace F in step 1) support_test With F query_test Input segmentation network A to obtain the foreground and background segmentation results of the tumor and The segmentation result is determined by the following expression:

[0098]

[0099]

[0100] 4) Change the and Input the integrated optimizer O to get the optimized segmentation result The segmentation result is determined by the following expression:

[0101]

[0102] Segmentation results The output is in the form of a matrix with a channel dimension of 1, which is a binary 0-1 image. The tumor segmentation result is 1 and the tumor background target segmentation result is 0. A complete and clear segmentation image is output.

[0103] In the selection of evaluation indicators, the Dice coefficient is selected as the evaluation indicator, denoted as E Dice , determined by the following formula:

[0104]

[0105] Among them, the model with the best tumor segmentation result evaluation result among the models obtained in each round of iterative training is selected, which means that for the models obtained in each round of iterative training, the E calculated by selecting the nasopharyngeal carcinoma tumor segmentation result and the corresponding tumor segmentation label gt Dice The highest model is taken as the optimal segmentation model.

[0106] like Figure 2As shown, the small-sample tumor segmentation model consists of a shared encoder S, a segmentation network B for tumor-background segmentation, a tumor segmentation network A based on a meta-learning architecture, and an ensemble optimizer O. Segmentation network B is used to identify tumor-background objects, while segmentation network A is used to identify tumor-like objects. The two segmentation network encoders are shared to extract information from CT images in the support and query sets. Multi-scale information from different encoder depths is jointly fed into segmentation network A, which leverages this multi-scale information to segment the foreground and background of query objects, obtaining coarse foreground and background segmentation results, respectively. The information extracted through layer-by-layer encoding is fed into segmentation network B. The encoder and segmentation network B form a U-shaped encoder-decoder network structure, outputting a probability distribution for the background class in the query set. An adaptive ensemble module adjusts the probability distribution output by segmentation network B and the coarse segmentation mask output by segmentation network A, suppressing pixels in the mask that are misclassified as tumor-background, to obtain the final prediction for the query set.

[0107] like Figure 3 As shown in the figure, the shared encoder S is composed of the encoder part of the pyramid network P that fuses multi-scale global information, and the segmentation network B is composed of the decoder part of the pyramid network P. The pyramid network P is a pyramid network based on the U-Net network structure, embedded with an SCP module that extracts multi-scale information and integrates global information. The pyramid network includes an encoder part and a decoder part with skip connections.

[0108] The encoder part consists of 4 encoder submodules with the same structure, and the number of output channels is 64, 128, 256, and 512 respectively; each encoder submodule contains 2 consecutive 2D convolution layers and 1 maximum pooling layer; each 2D convolution layer is followed by an instance normalization layer, and ReLU is used as the activation function; the convolution kernel size of each convolution layer is set to 3×3, the step size is set to 1, and the "same" convolution method is used, and the padding is set to 1. The purpose is to expand the network receptive field while strengthening the extraction of image edge information, ensuring that the size of the feature map remains unchanged during the convolution process, making the subsequent size change calculation of the network more convenient. The maximum pooling layer is placed after the second 2D convolutional layer to extract the main information of the feature map and reduce the size of the feature map. In addition to being sent to the encoder sub-module of the next level, the output of each encoder sub-module is also sent to the SCP module at the jump connection position, and is spliced ​​and fused with the output of the SCP module from the next level in the channel dimension to achieve global multi-scale information fusion. In addition to being the input of the SCP module of the previous level, the fused information is also input into the decoder sub-module of the corresponding level to obtain more accurate segmentation results. There are 4 SCP modules in total, and the number of output channels is 64, 128, 256, and 512 respectively.

[0109] like Figure 4 As shown in the figure, the SCP module consists of a splicing layer, a convolutional layer and an attention mechanism module; as the encoder part performs multiple convolution and pooling operations on the input CT image, the semantic information of the obtained feature map information is gradually enhanced, and the scale of the extracted features is gradually increased; the feature map output by the encoder submodule at the same level as an SCP module is spliced ​​with the feature map output by the SCP module at the next level in the channel dimension through the concatenate operation at the splicing layer, where the output feature map of the SCP module at the next level will first undergo convolution and upsampling to be the same as the output feature map of the encoder submodule at this level in terms of the number of channels and width × height. The spliced ​​feature information is fused through the convolution layer to obtain a multi-scale information flow; the information output by the encoder submodule at the deepest level of the pyramid network P only passes through an attention mechanism module that is the same as that in the SCP module.

[0110] The multi-scale information flow obtained next passes through the attention mechanism module, and the channel attention mechanism is used to assign higher weights to important features that contribute more to the segmentation task, weaken unimportant feature channels, and thus improve the overall performance of the segmentation network A; the attention mechanism module is composed of a global average pooling layer and two consecutive fully connected layers in sequence, wherein the channel statistics are first generated in the width × height dimension through the global average pooling layer, and the global spatial information of the feature map is compressed into a channel descriptor, and then the channel statistics are reduced to 1 / 16 of the original dimension in the first fully connected layer and activated by the ReLU function, and then the output of the first fully connected layer is increased to the original dimension in the second fully connected layer, and then a Sigmoid function is used to obtain a normalized weight between 0 and 1, and finally the normalized weight is weighted to the features of each channel through the Scale operation, so that the attention mechanism can complete the selection of important feature information.

[0111] The decoder part consists of 4 decoder submodules and 1 output layer, wherein the structures of the 4 decoder submodules are the same; the 4 decoder submodules all contain 2 consecutive 2D convolution layers and 1 upsampling layer, arranged in sequence, and the number of output channels of each decoder submodule is 1024, 512, 256, and 128; each 2D convolution layer is followed by an instance normalization layer, and ReLU is used as the activation function; wherein the convolution kernel size of each convolution layer is set to 3×3, the step size is set to 1, and the "same" convolution method is used, and the padding is set to 1. While expanding the network receptive field, the extraction of image edge information is strengthened, ensuring that the size of the feature map remains unchanged during the convolution process, making the calculation of subsequent size changes of the network more convenient; the upsampling layer is in the first After the two 2D convolutional layers, the convolutional layers and upsampling layers are arranged in this order to restore the image size while eliminating the checkerboard effect. The output layer also contains two consecutive 2D convolutional layers, with the same convolutional layer configuration as the four decoder submodules. The number of output channels is 14, which is the sum of the types of organs at risk for nasopharyngeal carcinoma and the image background categories selected in this embodiment. The output of each decoder submodule is concatenated with the output of the SCP module in the skip connection at the corresponding depth in the channel dimension, and then input into the next decoder submodule and iterated to the output layer. The global multi-scale information flow fed to the decoder via the skip connection can help the network recover the detailed feature information lost during the downsampling process, thereby obtaining a more structurally accurate generation result.

[0112] like Figure 5 As shown in FIG, the segmentation network A is composed of a meta-learning network M that models foreground and background information, and is a classic network structure used in small sample segmentation tasks of medical images.

[0113] The meta-learning network M consists of two convolutional layers, one upsampling layer, one feature prototype extraction layer and one metric module. After being extracted by the shared encoder S, the feature information F from the support set and the query set is support With F query After convolution and upsampling operations, where the convolution kernel size is 1×1, the dimension size is the same as the support set label M support Match, then use M support With M support The inverse code is then ANDed with the feature information after dimension matching, paying more attention to the area of ​​interest to be segmented in the feature matrix, and performing a convolution operation with a convolution kernel size of 3×3 to obtain the foreground feature and background feature matrices respectively; then, in the feature prototype extraction layer, the mask average pooling method is used to calculate the foreground feature prototype and background feature prototype in the support set respectively; finally, the foreground and background feature prototypes are respectively input into the measurement module, and in the query set image data, the cosine distance measurement method is used to measure and classify the foreground and background feature prototypes of the tumor background class target pixel by pixel, respectively, completing the foreground and background segmentation of the tumor background class target, and outputting the corresponding foreground segmentation results and background segmentation results.

[0114] like Figure 6 As shown in the figure, the integrated optimizer O consists of two convolutional connection structures. First, the background segmentation results of the tumor background target output by segmentation network A are spliced ​​with the output results of segmentation network B in the channel dimension. Then, a 1×1 convolution kernel is used to obtain the target background mask guided by the output results of segmentation network B. The pixels that are incorrectly segmented in the background segmentation results of the tumor background target by segmentation network A are corrected, thereby improving the rough segmentation results of segmentation network A. The target background mask is then spliced ​​with the foreground segmentation results of the tumor background target output by segmentation network A in the channel dimension, and a 1×1 convolution is performed to obtain the final refined segmentation result.

[0115] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A CT image tumor small sample segmentation system based on background class recognition, characterized by: include: The data acquisition module is used to collect CT image data sets, randomly divide the CT image data sets into training sets and test sets, and then divide the support set and query set for small sample learning based on the training set and test set. A region of interest delineation module is used to delineate the region of interest in the CT image data set and use the delineated image data as a reference label; The small tumor sample segmentation model training module is used to iteratively train the small tumor sample segmentation model using image data in the training set. The small tumor sample segmentation model includes a shared encoder S, a segmentation network A, a segmentation network B, and an integrated optimizer O. The shared encoder S is composed of the encoder part of a pyramid network P with multi-scale global information fusion, the segmentation network A is composed of a meta-learning network M for modeling foreground and background information, the segmentation network B is composed of the decoder part of the pyramid network P with multi-scale global information fusion, and the integrated optimizer O is composed of two convolutional connection structures. During training, the support set and query set features are first extracted through the shared encoder S. Then, supervised learning is used to optimize only the loss function of the segmentation network B for training to identify background class targets of the tumor. After training is completed, the parameters of the shared encoder S and the segmentation network B are fixed, and the loss function of the segmentation network A is optimized using the image data in the training set to complete the segmentation of the tumor background class. The output of the segmentation network B is used in the integrated optimizer O to provide auxiliary information for the segmentation result of the segmentation network A to obtain a more accurate tumor background class segmentation result. The model testing module is used to input all sample CT images in the test set into the tumor small sample segmentation models obtained from each iterative training round by the tumor small sample segmentation model training module. During the testing phase, segmentation network A segments the tumor, and segmentation network B provides auxiliary information for identifying the tumor background class. An integrated optimizer O is used to prevent segmentation network A from incorrectly segmenting the tumor background class as a tumor. The model with the highest tumor segmentation accuracy among the models obtained in each round of iterative training is selected as the optimal segmentation model, and the final complete tumor binary segmentation image is output; The integrated optimizer O is composed of two convolutional connection structures. First, the foreground segmentation result of the tumor background target output by the segmentation network A is spliced ​​with the output result of the segmentation network B in the channel dimension. Then, a 1×1 convolution kernel is used to obtain the target background mask adjusted by the output result of the segmentation network B. The erroneously segmented pixels in the background segmentation result of the tumor background target by the segmentation network A are corrected to improve the rough segmentation result of the segmentation network A. Then, the target background mask is spliced ​​with the background segmentation result of the tumor background target output by the segmentation network A in the channel dimension, and a 1×1 convolution is performed to obtain the final refined segmentation result.

2. The CT image tumor small sample segmentation system based on background class recognition according to claim 1 is characterized by: The CT image dataset includes CT images containing nasopharyngeal carcinoma tumors and multiple organs at risk of nasopharyngeal carcinoma, wherein the organs at risk of nasopharyngeal carcinoma belong to the background class targets of the nasopharyngeal carcinoma tumor; the data acquisition module uses the software Matlab to read the dicom file containing the CT image data of the organs at risk of nasopharyngeal carcinoma, and then converts the read data into images to obtain the corresponding CT image dataset; the CT image dataset is randomly sorted, and the data in the randomly sorted dataset is divided into a training set and a test set in proportion, wherein the CT images in the training set only contain information on the organs at risk of nasopharyngeal carcinoma, and the CT images in the test set contain both nasopharyngeal carcinoma tumor information and information on the organs at risk of nasopharyngeal carcinoma; a 1-way-1-shot method is selected in the support set and query set settings, that is, one category is selected each time in each scenario training on the support set, and the number of samples in each category is 1, and the number of slices in each category on the query set is also 1.

3. The CT image tumor small sample segmentation system based on background class recognition according to claim 2, characterized in that: The region of interest delineation module uses the software Matlab to generate the corresponding label map based on the manually delineated segmentation result data in the reference image corresponding to the DICOM file, and uses the delineated image data as the reference label to serve as the target area of ​​the segmentation system.

4. The CT image tumor small sample segmentation system based on background class recognition according to claim 3 is characterized by: The tumor small sample segmentation model training module divides the training set into n batches of data with a size of m and trains them batch by batch; in the training phase, the CT images on the support set of the current batch are trained with I support_train Indicates that the CT images on the query set are represented by I query_train Indicates that the label of the manually segmented tumor background class is represented by gtb; The training process consists of the following steps: 1) Will I support_train with I query_train Input the shared encoder S to obtain feature information F support With F query , where the calculation result of the characteristic information is determined by the following expression: F support =S(I support_train ) F query =S(I query_train ) 2) F query Input segmentation network B to obtain tumor background segmentation results Calculate the corresponding loss term of the segmentation network B And update the parameters of the segmentation network B; the segmentation result is determined by the following expression: The corresponding loss term of the segmentation network B It is composed of multi-class cross entropy loss and is determined by the following formula: in, The smaller it is, the higher the accuracy of the segmentation network B; 3) Replace F in step 1) support With F query Input segmentation network A to obtain the foreground and background segmentation results of tumor background class targets and The segmentation result is determined by the following expression: 4) Change the and Input the integrated optimizer O to get the optimized segmentation result The segmentation result is determined by the following expression: The corresponding loss term of the segmentation network A It is composed of multi-class cross entropy loss and is determined by the following formula: Where, The smaller it is, the higher the accuracy of the segmentation network A; When training the shared encoder S, segmentation network A and segmentation network B, the segmentation network B is first trained alone until convergence, and then the parameters of the shared encoder S and segmentation network B are fixed, and the segmentation network A is added to train the entire tumor small sample segmentation model.

5. The CT image tumor small sample segmentation system based on background class recognition according to claim 4 is characterized by: The shared encoder S is composed of the encoder part of a pyramid network P that fuses multi-scale global information. The network P is a pyramid network embedded with an SCP module that extracts multi-scale information and integrates global information. The encoder part consists of four encoder submodules with the same structure; each encoder submodule contains two consecutive 2D convolutional layers and one maximum pooling layer; each 2D convolutional layer is followed by an instance normalization layer, and ReLU is used as the activation function; wherein, the convolution kernel size of each convolutional layer is set to 3×3, the stride is set to 1, and the same convolution method is used, and the padding is set to 1, the purpose is to expand the network receptive field while strengthening the extraction of image edge information, ensuring that the size of the feature map remains unchanged during the convolution process, making the subsequent size change calculation of the network more convenient; the maximum pooling layer is placed after the second 2D convolutional layer, the purpose is to extract the main information of the feature map and reduce the size of the feature map; in addition to being transmitted to the encoder submodule of the next level, the output of each encoder submodule will also be transmitted to the SCP module at the jump connection position, and will be spliced ​​and fused with the output of the SCP module from the next level in the channel dimension to achieve global multi-scale information fusion. In addition to being the input of the SCP module of the previous level, the fused information will also be input into the decoder submodule of the corresponding level to obtain more accurate segmentation results; The SCP module consists of a concatenation layer, a convolution layer, and an attention mechanism module. As the encoder performs multiple convolution and pooling operations on the input CT image, the semantic information of the obtained feature map information is gradually enhanced, and the scale of the extracted features is gradually increased. The feature map output by the encoder submodule at the same level as an SCP module is concatenated with the feature map output by the SCP module at the next level in the channel dimension through a concatenation operation at the concatenation layer, and the concatenated feature information is fused through a convolution layer, thereby obtaining a multi-scale information flow. The information output by the encoder submodule at the deepest level of the pyramid network P only passes through an attention mechanism module that is the same as that in the SCP module. The multi-scale information flow obtained next passes through the attention mechanism module, and the channel attention mechanism is used to assign higher weights to important features that contribute more to the segmentation task, weaken unimportant feature channels, and thus improve the overall performance of the segmentation network A; the attention mechanism module is composed of a global average pooling layer and two consecutive fully connected layers in sequence, wherein the global average pooling layer is first used to generate channel statistics, and the global spatial information of the feature map is compressed into a channel descriptor, and then the channel statistics are reduced to 1 / 16 of the original dimension in the first fully connected layer and activated by the ReLU function, and then the output of the first fully connected layer is increased to the original dimension in the second fully connected layer, and then a Sigmoid function is used to obtain normalized weights between 0 and 1, and finally the normalized weights are weighted to the features of each channel through the Scale operation.

6. The CT image tumor small sample segmentation system based on background class recognition according to claim 5, characterized in that: The segmentation network A is composed of a meta-learning network M that models foreground and background information, and is a classic network structure used in small-sample segmentation tasks of medical images. The meta-learning network M consists of two convolutional layers, one upsampling layer, one feature prototype extraction layer and one metric module. After being extracted by the shared encoder S, the feature information F from the support set and the query set is support With F query After convolution and upsampling operations, where the convolution kernel size is 1×1, the dimension size is the same as the support set label M support Match, then use M support With M support The inverse code is then ANDed with the feature information after dimension matching to pay more attention to the area of ​​interest to be segmented in the feature matrix, and a convolution operation is performed with a convolution kernel size of 3×3 to obtain the foreground feature matrix and the background feature matrix respectively. In the feature prototype extraction layer, the mask average pooling method is used to calculate the foreground feature prototype and background feature prototype in the support set respectively; finally, the foreground and background feature prototypes are input into the measurement module respectively. In the image data of the query set, the foreground and background feature prototypes of the tumor background class targets are measured and classified pixel by pixel using the cosine distance measurement method, and the foreground and background segmentation of the tumor background class targets are completed respectively, and the corresponding foreground segmentation results and background segmentation results are output.

7. The CT image tumor small sample segmentation system based on background class recognition according to claim 6, characterized in that: The segmentation network B is composed of the decoder part of the pyramid network P with multi-scale global information fusion; The decoder part consists of 4 decoder submodules and 1 output layer, wherein the structures of the 4 decoder submodules are the same; the 4 decoder submodules all contain 2 consecutive 2D convolution layers and 1 upsampling layer, arranged in sequence; each 2D convolution layer is followed by an instance normalization layer, and ReLU is used as the activation function; wherein, the convolution kernel size of each convolution layer is set to 3×3, the step size is set to 1, and the same convolution method is used, and the padding is set to 1, while expanding the network receptive field, the extraction of image edge information is strengthened, and the size of the feature map is guaranteed to remain unchanged during the convolution process, making the calculation of subsequent size changes of the network more convenient; The sampling layer is placed after the second 2D convolutional layer. Arranging the convolutional layers and upsampling layers in this order can restore the image size while eliminating the checkerboard effect. The output layer contains two consecutive 2D convolutional layers, which are the same as the convolutional layer settings in the four decoder sub-modules. The output of each decoder sub-module is spliced ​​with the output of the SCP module in the skip connection from the corresponding depth in the channel dimension, and then input into the next decoder sub-module and iterated to the output layer. The operation of feeding the global multi-scale information flow to the decoder through the skip connection can help the network recover the detailed feature information lost during the downsampling process, thereby obtaining a more structurally accurate generation result.

8. The CT image tumor small sample segmentation system based on background class recognition according to claim 7, characterized in that: The model testing module inputs the CT images of each sample in the test set into the tumor small sample segmentation model training module, generates a tumor small sample segmentation model for each iterative training round, outputs the corresponding nasopharyngeal carcinoma tumor segmentation result, and evaluates the accuracy of the segmentation result obtained for each sample. The model with the best nasopharyngeal carcinoma tumor segmentation result evaluation result among the models obtained in each round of iterative training is selected as the optimal segmentation model; In the testing phase, the CT images on the support set of the current batch are represented by I support_test Indicates that the CT images on the query set are represented by I query_test Indicates that the label of the manually segmented tumor class is represented by gt; The testing process includes the following steps: 1) Will I support_test with I query_test Input to the shared encoder S that has completed training and has fixed parameters to obtain feature information F support_test With F query_test , where the calculation result of the characteristic information is determined by the following expression: F support_test =S(I support_test ) F query_test =S(I query_test ) 2) F query_test Input segmentation network B to obtain tumor background segmentation results The segmentation result is determined by the following expression: 3) Replace F in step 1) support_test With F query_test Input segmentation network A to obtain the foreground and background segmentation results of the tumor and The segmentation result is determined by the following expression: 4) Change the and Input the integrated optimizer O to get the optimized segmentation result The segmentation result is determined by the following expression: Segmentation results The output is in the form of a matrix with a channel dimension of 1, which is a binary 0-1 image. The tumor segmentation result is 1, and the tumor background target segmentation result is 0. A complete and clear segmentation image is output. In the selection of evaluation indicators, the Dice coefficient is selected as the evaluation indicator, denoted as E Dice , determined by the following formula: The model with the best tumor segmentation result evaluation result among the models obtained in each round of iterative training is selected, which means that for the models obtained in each round of iterative training, the E calculated by selecting the nasopharyngeal carcinoma tumor segmentation result and the corresponding tumor segmentation label gt Dice The highest model is taken as the optimal segmentation model.

Images