Unpaired multimodal segmentation method and system based on curvature consistency

Through the non-paired multimodal segmentation method based on curvature consistency, the multi-task feature complementation enhancement and symbol distance field constraints are used to solve the problems of multi-stage CT image fusion and medical image registration in pancreatic disease diagnosis, and realize the single period High-precision pancreas and its lesions segmentation on CT images improves the accuracy and efficiency of diagnosis.

CN118261927BActive Publication Date: 2024-11-22SUZHOU UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410695930.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-31
Publication Date
2024-11-22
Estimated Expiration
2044-05-31

AI Technical Summary

Technical Problem

In the diagnosis of pancreatic diseases, there is a lack of a multi-phase fusion algorithm model that can use multi-phase CT images during training, and only a single-phase CT images are needed for reasoning. It is difficult to register medical images, resulting in accurate segmentation of pancreas and its lesions. Sexual limitation.

Method used

Design a non-paired multimodal segmentation method based on curvature consistency, and fuse pancreatic high-density foci segmentation subnet (SDFCCFF) through multi-task feature complementary enhancement and curvature consistency pancreatic segmentation subnet (MFCECC) and symbol distance field constraints and curvature characteristics. , multi-stage CT images are used during training, and only a single-stage CT images are needed during inference to alleviate the loss of modal invariant information during modal conversion, and assist in the segmentation of high-density pancreatic lesions through symbol distance field and curvature characteristics.

Benefits of technology

It improves the segmentation accuracy and robustness of the pancreas and its high-density lesions, enhances the adaptability and generalization ability of the model, improves the recognition ability of edges and shapes, and improves the accuracy and efficiency of clinical diagnosis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118261927B_ABST
    Figure CN118261927B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of image processing technology, and in particular to an unpaired multimodal segmentation method and system based on curvature consistency. Two stages of a UCPSCC segmentation network are constructed, and the UCPSCC segmentation network is trained in two stages. The MFCE module is used to constrain each other in the process of generating prediction results and synthetic images; the segmentation prediction results of the real image and the synthetic image are aligned according to the curvature consistency loss, and accurate pancreatic segmentation prediction results are obtained. The discriminator of the auxiliary modality for the auxiliary modality synthetic image and the discriminator of the main modality for the main modality synthetic image are introduced for adversarial learning, and the modality conversion task and the segmentation task are unified in the same network. In view of the problem of the loss of modality-invariant information extracted during the modality conversion task and the target organ information in the segmentation task, a multi-task feature complementary enhancement module and a lightweight multi-scale feature fusion module are designed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an unpaired multimodal segmentation method and system based on curvature consistency. Background Art

[0002] Chronic pancreatitis (CP) is a persistent inflammation of the pancreas, which can lead to permanent structural destruction of the pancreas, such as fibrosis and ductal stenosis, and is often accompanied by the formation of stones. It may eventually lead to pancreatic sclerosis, calcification foci, and even pancreatic cancer, which seriously affects the patient's quality of life and health. Computed tomography is currently the most important examination method for pancreatic diseases. Non-contrast-enhanced CT (NCECT) scan images help to identify the structure and density of the pancreas, and the high-density lesion area (PHDLA) in the pancreas appears as a high-density shadow on the NCE image, which forms a clear contrast with the surrounding tissue and is conducive to observation. Contrast-enhanced CT (CECT) scan images can better observe the blood vessels and other tissue structures around the pancreas. Therefore, the combination of NCECT and CECT dual-phase diagnosis of the pancreas and its high-density lesions can provide doctors with important anatomical and pathological information. Many existing studies choose to focus on the segmentation task of the pancreas and its lesion area in CECT images. At the same time, a large number of studies have shown that the combination of NCECT and CECT images can provide more comprehensive and accurate information. Considering that in the actual clinical diagnosis process, it may be impossible to obtain CECT images of patients due to factors such as cost and specific allergic reactions to contrast agents. In addition, since the density of blood vessels and other tissues and organs in CECT images is enhanced by contrast agents, it is also very difficult for professional doctors to label high-density lesions inside the pancreas, which leads to the lack of its gold standard, which also brings certain challenges to cross-period segmentation. Therefore, there is an urgent need for a multi-phase fusion algorithm model that can use multi-phase CT images in the training phase and only requires a single-phase CT image in the inference phase. For example, by combining NCECT and CECT for training, accurate segmentation results of the pancreas and its lesions can be obtained on NCECT images. This method has important clinical value for diseased pancreas. Therefore, considering the difficulty of obtaining labels, it is chosen to use the better obtained venous phase labels to guide the segmentation of plain scan images, and the venous phase is defined as the auxiliary mode and the plain scan period is defined as the main mode.

[0003] In addition, obtaining paired multimodal images in medical research is also a very challenging task. It not only requires a lot of cost, time and time-consuming annotation process, but also usually requires medical image registration technology to align images in multiple modalities in special cases. However, the effect of registration also has a great impact on the results. Therefore, most methods perform poorly in dealing with missing modalities and unpaired situations, which brings huge challenges to medical research. In the process of collecting images of patients at different stages to accurately obtain lesion information, due to various reasons such as organ deformation, patient movement, gastrointestinal movement or breathing, the images of different modalities taken at different times are quite different, so it is impossible to obtain fully paired multi-stage images. Summary of the invention

[0004] In view of the above problems existing in the prior art, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is how to design a multi-period fusion algorithm model, using multi-period CT images during training and only using a single-period CT image during reasoning.

[0006] In order to solve the above technical problems, the present invention provides the following technical solution: an unpaired multimodal segmentation method based on curvature consistency, comprising: and auxiliary modality images Input into the multi-task feature complementary enhancement and curvature consistency pancreas segmentation sub-network MFCECC to generate the main modality synthetic image , auxiliary modality reconstruction image , auxiliary modality synthetic image , Main modality reconstruction image , Pancreas segmentation prediction map of main modality image , Main modality synthetic image pancreas segmentation prediction map , auxiliary modality image pancreas segmentation prediction map , auxiliary modality synthetic image pancreas segmentation prediction map ; Input the main modality image into the pancreatic high-density lesion segmentation subnetwork SDFCCFF fused with signed distance field constraint and curvature feature to obtain the signed distance prediction map of the pancreatic area And the pancreatic high-density lesion segmentation prediction map ; In the first sub-network MFCECC, the pancreas segmentation prediction map for the main modality image Pancreas segmentation prediction map of auxiliary modality synthetic image Design the main modality curvature consistency loss for the auxiliary modality image pancreas segmentation prediction map Pancreas segmentation prediction map of the main modality composite image An auxiliary modal curvature consistency loss is designed to alleviate the loss of modal invariant information during modal conversion. In the second sub-network SDFCCFF, a signed distance loss is designed to solve the over-segmentation phenomenon in non-pancreatic areas and the missed segmentation phenomenon in pancreatic areas.

[0007] As a preferred solution of the unpaired multimodal segmentation method based on curvature consistency described in the present invention, wherein: the multi-task feature complementary enhancement and curvature consistency pancreas segmentation subnetwork MFCECC comprises 11 submodules, including 3 encoders, a modality-invariant content feature encoder , feature encoder for the main modality style , feature encoder for auxiliary modality style , decoder for the main modality synthetic image , decoder for auxiliary modality synthetic images , segmentation head for auxiliary modality and the segmentation head for the main modality , two discriminators for auxiliary modality and main modality synthetic images, one multi-task feature complementary enhancement (MFCE) module for auxiliary modality and one for main modality, where the subscripts are The representation is for auxiliary mode, the subscript is The representation is for the main mode;

[0008] The signed distance field constraint and curvature feature fusion pancreatic high-density lesion segmentation sub-network SDFCCFF contains 5 modules, including an independent encoder and a modality-invariant content feature encoder shared with MFCECC , 1 signed distance field fusion curvature feature fusion module, 1 signed distance constraint branch and a segmentation head for pancreatic high-density lesion areas ;

[0009] It is the backbone network of ResNet34. and Adoption and The same architecture, and A network with 5 convolutional layers is used. The first two convolutional layers are composed of residual blocks, and each of the last three layers is composed of transposed convolution, BN normalization layer and ReLU activation layer. and Adoption and The same architecture, and The last layer output is 3 channels, and The last layer output is 2 channels; the discriminator for the auxiliary modality and main modality synthetic images uses a network with four convolutional layers and one fully connected layer. The convolution in each convolutional layer is followed by an IN normalization layer and a LeakyReLU activation layer, and finally outputs a single-channel feature map; the independent encoder in the second sub-network The segmentation head uses the UNet architecture. The number of feature channels in each layer is {64, 128, 256, 512, 1024} respectively.

[0010] As a preferred solution of the unpaired multimodal segmentation method based on curvature consistency described in the present invention, wherein: the multi-task feature complementary enhancement and curvature consistency pancreas segmentation subnetwork MFCECC includes: and the main modal image Sent to the modality-invariant content feature encoder Extract content features for auxiliary modality images and content features for the main modality image ; then and Send to and In the above example, the style features of the auxiliary modality image are extracted. and style features for the main modality image ; By and Sent to two specific split heads respectively and The pancreas segmentation prediction results of the auxiliary modality image and the main modality image are obtained respectively. and ,Will and Feed generate ,Will and Feed generate , after obtaining the composite image, Feed and In the above example, we obtain the content features of the corresponding main modality synthetic image. and style features ,Will Feed and In the above example, we obtain the content features of the corresponding auxiliary modality synthetic image. and style features ;Will and Feed generate ,Will and Feed generate , considering the synthesized image as a data augmentation sample from the auxiliary modality and the main modality, we get and Corresponding pancreas segmentation prediction results and is the pancreas segmentation prediction map of the main modality synthetic image, It is the pancreas segmentation prediction map of the auxiliary modality synthetic image.

[0011] against and Design the principal mode curvature consistency loss for and Design auxiliary modal curvature consistency loss to alleviate the loss of modal invariant information during modal conversion;

[0012] against and Design a discriminator for the main modality synthetic image, and Design a discriminator for auxiliary modality synthetic images and obtain the adversarial loss of the two discriminators;

[0013] against and Design the main mode splitter for and An auxiliary modal segmenter is designed to obtain the segmentation loss of the two segmenters;

[0014] against and A dual reconstruction consistency loss is designed;

[0015] In the process of generating pancreatic segmentation prediction results and synthesized images, the MFCE module is used to constrain each other. In the MFCE module, the multi-scale content features of the main modality are , segmentation features and synthetic image generation features Perform fusion to preserve the structural details and spatial information of the target organ;

[0016] The MFCE module contains a lightweight multi-scale feature fusion submodule LMSF. and Simplified to Represents the modality-invariant content feature encoder The intermediate features obtained before the first three downsamplings in the encoding stage are sent to the switchable normalization layer respectively. The switchable normalization layer consists of batch normalization and group normalization, and the weights of the two are 0.5 respectively. The three normalized output results are sent to the depth-wise separable convolution. First, a convolution kernel is used to perform each channel of the input separately. Convolution is used to extract spatial features, and then several The convolution kernel performs independent convolution on each channel. The number of convolution kernels is the number of output channels, and the output features are sent to the channel regularization scaling layer. The feature information of each channel is used by learning the weight of each channel, and then the residual connection is realized with the input of the module to ensure that the feature information at each scale is not lost. The process is described as:

[0017] ;

[0018] in, Expressed as The enhanced output results are: is represented as a channel regularization scaling layer, Represented as a depth-wise separable convolutional layer, Represented as a switchable normalization layer, and They are the enhanced output results, and then the output results are upsampled and spliced ​​in the channel dimension.

[0019] ;

[0020] in, Indicates upsampling to twice the original size. Indicates upsampling to 4 times the original size. Represents a splicing operation on a channel;

[0021] In order to extract global context information from feature maps at different levels, three depth-wise separable convolutions with different dilation rates are used in parallel, and the obtained outputs are concatenated in the channel dimension and then used Convolutional reconstruction to the final multi-scale fusion feature map , The characteristic size and same;

[0022] ;

[0023] in, , and They represent depthwise separable convolutions with dilation rates set to 1, 2, and 4, respectively. Represents a splicing operation on a channel, Indicates that the convolution kernel size is Convolutional layer, BN layer, ReLU layer;

[0024] Output features at the last stage of the modality conversion task Output features of the last stage of the segmentation task A multi-task feature complementary enhancement operation is performed between yes After the features are generated by the LMSF module, the specific operation of MFCE is as follows:

[0025] When the intermediate features of modal conversion are enhanced complementary information, first and Splicing is performed on the channel dimension, the multi-scale semantic information obtained by the encoder is integrated into the segmentation features, and then the convolution operation is performed to restore it to the same level as The same channel gets ,

[0026] ;

[0027] The correlation and dependency between the entire channels are calculated by combining the maximum pooling operation and the average pooling adaptive layer, and a global channel dependency representation vector is generated. ,

[0028] ;

[0029] in, and Represent the adaptive average pooling layer and the maximum pooling layer respectively;

[0030] The MLP layer is used to compress the spatial information into a channel vector. The convolution kernel size used in the MLP layer is Two convolution kernels , and ReLU layers to replace the fully connected operation, perform a sigmoid operation on the channel vector, broadcast the result on each channel and then Perform element-by-element multiplication and perform the result Convolution operation, finally the information is fused into In this way, the segmentation network will focus more on the target pancreas segmentation area, and the formula is expressed as:

[0031] ;

[0032] in, is the pancreas segmentation prediction map of the main modality image, is the ReLU activation function, is the sigmoid function, The operation represents broadcasting the channel vector over the last two dimensions and then summing Perform element-wise multiplication;

[0033] When the intermediate features of the segmentation task are enhanced complementary information, The generation process and Similarly, the formula is expressed as:

[0034] ;

[0035] in, Represented as a fusion of multi-scale semantic information Represented as a global channel dependency .

[0036] As a preferred solution of the unpaired multimodal segmentation method based on curvature consistency described in the present invention, wherein: the main modality curvature consistency loss is expressed as,

[0037] ;

[0038] in, Expressed as the curvature consistency loss for the main mode, express The pixels, express The pixels, For the curvature linear convolution operation, by aligning the segmentation prediction results of the main modality and auxiliary modality synthetic images in the curvature space, the problem of modality invariant information loss during modality conversion is further alleviated to enhance the modality invariant information extraction capability of the shared encoder;

[0039] The auxiliary modal curvature consistency loss is expressed as,

[0040] ;

[0041] in, represents the curvature consistency loss for the auxiliary mode, express The lth pixel in The proposed method is a curvature linear convolution operation. By aligning the segmentation prediction results of the auxiliary modality and the main modality synthetic image in the curvature space, it alleviates the problem of modality invariant information loss during modality conversion and enhances the modality invariant information extraction capability of the shared encoder.

[0042] As a preferred solution of the unpaired multimodal segmentation method based on curvature consistency of the present invention, wherein: the discriminator for the main modality synthetic image includes, in order to generate a realistic main modality synthetic image, adversarial loss as follows:

[0043] ;

[0044] in, is represented as the discriminator for the main modality synthetic image, Expressing hope;

[0045] The discriminator for the auxiliary modality synthetic image includes: in order to generate realistic auxiliary modality synthetic images, a discriminator for the auxiliary modality synthetic image is introduced Conduct adversarial learning and adversarial loss as follows:

[0046] ;

[0047] The dual reconstruction consistency loss includes,

[0048] ;

[0049] The segmentation loss of the segmenter includes,

[0050] ;

[0051] Where N represents the total number of pixels. and They are and The corresponding gold standard, express The pixels, express The pixels, is 0.0001 to prevent the denominator from being 0. Expressed as the binary cross entropy loss for the auxiliary modality, Expressed as the dice similarity loss for the auxiliary modality, Expressed as the binary cross entropy bce loss for the main modality, Expressed as the dice similarity loss for the main modality;

[0052] The loss function used in training the MFCECC pancreas segmentation subnetwork It is expressed as:

[0053] ;

[0054] in, Expressed as the adversarial loss against the main modality, Denoted as the adversarial loss for the auxiliary modality, Expressed as the curvature consistency loss for the main mode, Expressed as the curvature consistency loss for the auxiliary mode, Denoted as the dual reconstruction consistency loss, Denoted as the segmentation loss for the auxiliary modality image, Denote as the segmentation loss for the primary modality image.

[0055] As a preferred solution of the unpaired multimodal segmentation method based on curvature consistency described in the present invention, wherein: the signed distance field constraint and curvature feature fusion pancreatic high-density lesion segmentation subnetwork SDFCCFF includes first freezing No back-propagation is performed to update the parameters, and then and The generated features of different scales are sent to the LMSF module respectively to obtain the corresponding multi-scale fusion features. For the pancreas positioning problem, Added a signed distance constraint branch based on , and then multi-scale fusion features and The generated curvature features of different scales are integrated into Finally, we get an accurate PHDLA segmentation area. The distance from the pixel to the boundary in the signed distance field determines the intensity of the pixel. The decoder's prediction task uses and Co-training implicitly learns the location information of the pancreas,

[0056] ;

[0057] in, Expressed as Signed distance prediction map of pancreatic regions corresponding to decoder branches, Represented as an independent encoder in the second sub-network;

[0058] In the SDFCCFF sub-network framework, the mean square error calculation method is used as The symbol distance loss :

[0059] ;

[0060] in, yes Calculated by formula.

[0061] As a preferred solution of the unpaired multimodal segmentation method based on curvature consistency described in the present invention, wherein: the corresponding multi-scale fusion features are obtained by introducing the MFCECC task on the basis of the dual branch. ,Will and The intermediate output features of the first three stages are fed into In the module, the output features are processed twice The convolution operation with a step size of 2 reduces the size of the feature to 1 / 4 of the original size, and finally obtains two multi-scale fusion feature maps. and , always freezes during training The parameters make the frozen parameters not participate in the update;

[0062] The optimized segmentation result includes integrating the curvature features of the signed distance field fusion into the In the segmentation process, the specific operation is expressed as:

[0063] ;

[0064] in, express No. The output characteristics of the stage, express No. The output features of the stage are the output features of the curvature features fused with the signed distance field. Indicates that embedding is required Decoder The curvature feature map of the signed distance field fusion of the stage, It means upsampling to twice the original size through bilinear interpolation. Represents upsampling by inverse convolution multiple;

[0065] Depend on and After the splicing operation, the curvature convolution is performed to obtain: Depend on and The output features of the second stage of the decoder are obtained by adding pixel by pixel.

[0066] ;

[0067] In the SDFCCFF sub-network framework, the BCE and Dice loss functions are used to calculate the high-density lesion branch loss :

[0068] ;

[0069] in, It is the gold standard for high-density pancreatic lesions;

[0070] In summary, the loss function used in the training of the SDFCCFF high-density lesion segmentation sub-network is as follows:

[0071] .

[0072] Another object of the present invention is to provide an unpaired multimodal segmentation system based on curvature consistency. The data preparation module ensures the quality and diversity of training and verification data, provides a solid foundation for the model, and thus improves the generalization ability and accuracy of the model. The MFCECC construction module significantly improves the accuracy and robustness of pancreatic segmentation by fusing multi-task features and curvature consistency, especially when processing data from different sources. The SDFCCFF construction module uses curvature features to assist in positioning, further improving the segmentation accuracy of high-density lesion areas, and has a significant effect on the identification of complex lesions. Through the MFCECC training module, the network can more effectively learn the common features and unique styles of different modalities, enhancing the adaptability and generalization ability of the model. The multi-task feature complementary enhancement module improves the model's ability to integrate different modal features through mutual constraints between features, thereby enhancing the consistency and accuracy of segmentation. The curvature alignment module improves the model's ability to recognize edges and shapes by applying curvature consistency loss, especially in the accurate segmentation of complex pancreatic structures. The adversarial learning strategy introduced by the adversarial learning module enables the model to generate more realistic synthetic images, further improving the generalization ability of the model when processing unknown data. In the SDFCCFF training module, the focus is on the precise segmentation of high-density lesions and pancreatic localization, which significantly improves the accuracy and efficiency of clinical diagnosis. The feature fusion module and optimization module enhance the model's ability to recognize lesions of different sizes and shapes by fusing multi-scale and curvature features, improving the detailed expression and overall consistency of the segmentation results, and providing strong support for medical image analysis.

[0073] As a preferred solution of the unpaired multimodal segmentation system based on curvature consistency described in the present invention, it includes: a data preparation module, an MFCECC construction module, an SDFCCFF construction module, an MFCECC training module, a multi-task feature complementary enhancement module, a curvature alignment module, an adversarial learning module, an SDFCCFF training module, an optimization module, and a feature fusion module;

[0074] The data preparation module obtains the original training set;

[0075] The MFCECC construction module builds the first-stage multi-task feature complementary enhancement and curvature consistency pancreas segmentation sub-network of the UCPSCC segmentation network;

[0076] SDFCCFF construction module, constructing the second stage of UCPSCC segmentation network, the sub-network for pancreatic high-density lesion segmentation, which integrates the signed distance field constraint and curvature feature;

[0077] MFCECC training module, the first stage of UCPSCC segmentation network is trained in the multi-task feature complementary enhancement and curvature consistency pancreas segmentation sub-network;

[0078] Multi-task feature complementary enhancement module, which uses MFCE module to constrain each other in the process of generating prediction results and synthesized images;

[0079] The curvature alignment module aligns the segmentation prediction results of the real image and the synthetic image according to the curvature consistency loss to obtain accurate pancreas segmentation prediction results;

[0080] The adversarial learning module introduces the discriminator of the auxiliary modality for the auxiliary modality synthetic image and the discriminator of the main modality for the main modality synthetic image to perform adversarial learning, and back-propagates the final loss;

[0081] SDFCCFF training module,The second training stage of the UCPSCC segmentation network is in the,signed distance field constraint and curvature feature fusion pancreatic high-density lesion segmentation sub-network,,which adds a signed distance constraint branch based on the segmentation head of the,pancreatic high-density lesion region;

[0082] The optimization module optimizes the segmentation results, freezes the parameters without back-propagation update, and sends the features of different scales generated by the modality-invariant content feature encoder and the feature encoder to the lightweight multi-scale feature fusion module to obtain the corresponding multi-scale fusion features;

[0083] The feature fusion module integrates the multi-scale fusion features and curvature features of different scales generated by the signed distance constraint branch into the segmentation head of the pancreatic high-density lesion area, and finally obtains accurate segmentation results of the pancreatic high-density lesion area.

[0084] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and wherein when the processor executes the computer program, the processor implements the steps of any one of the methods of unpaired multimodal segmentation based on curvature consistency.

[0085] A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of any one of the methods of unpaired multimodal segmentation based on curvature consistency are implemented.

[0086] Beneficial effects of the present invention: The method of the present invention unifies the modality conversion task and the segmentation task in the same network. Aiming at the problem of the loss of modality-invariant information extracted during the modality conversion task and the target organ information in the segmentation task, a multi-task feature complementary enhancement (MFCE) module and a lightweight multi-scale feature fusion (LMSF) module are designed to better preserve the structural details and spatial information of the target organ. At the same time, the network can use the main modality synthetic image generated by modality conversion during training, so as to expand the training samples to fuse more information of other modalities, and there is no need for multi-modal image pairing;

[0087] A curvature consistency loss is designed to align the same modality invariant information in the modality conversion process with curvature features, further alleviating the loss of modality invariant information in the modality conversion process, thereby enhancing the modality invariant information extraction capability of the shared encoder;

[0088] A signed distance field constraint and curvature feature fusion (SDFCCFF) module is proposed. In the pancreatic high-density lesion segmentation sub-network, a signed distance field constraint and curvature feature fusion method is proposed to integrate the position auxiliary feature information of the pancreas into the lesion area segmentation branch in an explicit way, thereby alleviating the stones and calcification over-segmentation phenomenon in the non-pancreatic area and the missed segmentation phenomenon in the pancreatic area; at the same time, the effectiveness of the proposed LMSF module is once again proved. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0090] Figure 1 A schematic flow chart of an unpaired multimodal segmentation method based on curvature consistency provided by an embodiment of the present invention.

[0091] Figure 2 A schematic diagram of a lightweight multi-scale feature fusion (LMSF) submodule of an unpaired multimodal segmentation method based on curvature consistency provided by an embodiment of the present invention.

[0092] Figure 3 A schematic diagram of a multi-task feature complementary enhancement (MFCE) module of an unpaired multimodal segmentation method based on curvature consistency provided by an embodiment of the present invention.

[0093] Figure 4 A schematic diagram of a sub-network architecture for pancreatic high-density lesion segmentation based on a curvature consistency-based unpaired multimodal segmentation method provided in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0094] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0095] Example 1

[0096] Reference Figure 1-Figure 4 , which is the first embodiment of the present invention, and provides an unpaired multimodal segmentation method based on curvature consistency, comprising:

[0097] S1. Obtaining an original training set, wherein the original training set includes a plurality of CT abdominal organ scanning images of auxiliary modalities and main modalities;

[0098] S2. Construct the first pancreas segmentation subnetwork of the UCPSCC segmentation network Multi-task Feature Complementation Enhancement and Curvature Consistency (MFCECC). MFCECC contains 11 submodules, including 3 encoders, modality-invariant content feature encoder , feature encoder for the main modality style , feature encoder for auxiliary modality style , decoder for the main modality synthetic image , decoder for auxiliary modality synthetic images , segmentation head for auxiliary modality and the segmentation head for the main modality , two discriminators for auxiliary modality and main modality synthetic images, one multi-task feature complementary enhancement module for auxiliary modality and one for main modality, where the subscripts are The representation is for auxiliary mode, the subscript is The representation is for the main mode;

[0099] S3. Construct the second signed distance field constraint and curvature feature fusion (SDFCCFF) high-density lesion segmentation sub-network of the UCPSCC segmentation network. It contains 5 modules, including an independent encoder. and 1 encoder shared with MFCECC , 1 signed distance field fusion curvature feature fusion module, 1 signed distance constraint branch and a segmentation head for pancreatic high-density lesion areas ;

[0100] and It is built based on the residual block of ResNet34. and Adoption and Same architecture; and Using the same architecture as the split head, and The last layer output is 3 channels, and The last layer output is 2 channels; The network with 7 convolutional layers is used. The convolution of the first three layers is followed by a BN normalization layer and a ReLU activation layer. The last four convolutional layers are composed of 4 residual blocks. The discriminator for the auxiliary modality and the main modality synthetic image uses a network with four convolutional layers and a fully connected layer. The convolution in each convolutional layer is followed by an IN normalization layer and a LeakyReLU activation layer, and finally a single-channel feature map is output. The independent encoder in the second sub-network The segmentation head uses the UNet architecture. The number of feature channels in each layer is {64, 128, 256, 512, 1024} respectively.

[0101] S4 and UCPSCC are divided into two sub-networks for training. The multi-task feature complementary enhancement and curvature consistency pancreas segmentation sub-network MFCECC includes the auxiliary modality image and the main modal image Shared Content Encoder Extract content features for auxiliary modality images and content features for the main modality image ; then and Send to two style encoders respectively and In the above example, the style features of the auxiliary modality image are extracted. and style features for the main modality image ; By and Sent to two specific split heads respectively and The pancreas segmentation prediction results of the auxiliary modality image and the main modality image are obtained respectively. and ,Will and Feed generate ,Will and Feed generate , after obtaining the composite image, Feed and In the above example, we obtain the content features of the corresponding main modality synthetic image. and style features ,Will Feed and In the above example, we obtain the content features of the corresponding auxiliary modality synthetic image. and style features ;Will and Feed generate ,Will and Feed generate , considering the synthesized image as a data augmentation sample from the auxiliary modality and the main modality, we get The synthesized primary modality image and Pancreas segmentation prediction results corresponding to the synthesized auxiliary modality image and is the pancreas segmentation prediction result of the main modality synthetic image, is the pancreas segmentation prediction result of the auxiliary modality synthetic image;

[0102] S41, such as Figure 1 As shown, the auxiliary modality image and the main modality image are sent to Obtained from and , and then send it into and Extracted from and ;

[0103] S42, MFCE module contains a lightweight multi-scale feature fusion (LMSF) submodule, such as Figure 2 As shown, and (Simplified to ) represent the modality-invariant content feature encoders The intermediate features obtained before the first three downsamplings in the encoding stage are first sent to the switchable normalization (SN) layer. SN consists of batch normalization (BatchNormalization, BN) and group normalization (Group Normalization, GN), and the weights of the two are 0.5 respectively. Then the three normalized output results are sent to the depthwise separable convolution (DSC), which first uses a convolution kernel to perform a separate convolution on each channel of the input. Convolution is used to extract spatial features, and then several The convolution kernel performs independent convolution on each channel. The number of convolution kernels is the number of output channels. Compared with traditional spatial convolution, depthwise separable convolution can improve feature representation capabilities while reducing computational costs. The output features are then sent to the Channel Scaling (CS) layer, which learns the weights of each channel to better utilize the feature information of each channel. Compared with general fully connected layers or convolutional layers, it requires fewer parameters, which reduces the complexity of the model and reflects the lightweight of the module, while reducing the risk of overfitting. Afterwards, a residual connection is implemented with the input of the module to ensure that feature information at all scales is not lost. The above process can be expressed as:

[0104] ;

[0105] in and They are the enhanced output results. They are then upsampled and concatenated in the channel dimension (for simplicity, the upsampling operation is not shown in Figure 2 (drawn in ).

[0106] ;

[0107] in Indicates upsampling to twice the original size. Indicates upsampling to 4 times the original size. Represents a concatenation operation on a channel.

[0108] Then, in order to extract global context information from feature maps at different levels, three different dilation rates are used in parallel (i.e., The depth-wise separable convolution with the expansion rate set to 1, 2, and 4 in the depth convolution is used to concatenate the outputs in the channel dimension and finally use Convolutional reconstruction to the final multi-scale fusion feature map The characteristic size and same.

[0109] ;

[0110] in, and They represent depthwise separable convolutions with dilation rates set to 1, 2, and 4, respectively. Represents a splicing operation on a channel, Indicates that the convolution kernel size is Convolutional layer, BN layer, ReLU layer;

[0111] S43. Since the feature spaces of different tasks are different, directly combining the features of different tasks may be suboptimal. Therefore, we first let the segmentation decoder generate the rough location of the target pancreas, and then use the MFCE module to incorporate lower-level details and gradually reconstruct the final pancreas segmentation result. Figure 3 As shown in Figure 2, due to the differences in high-level information between different tasks, this method aims to output features in the last stage of the modal conversion task. Output features of the last stage of the segmentation task The multi-task feature complementary enhancement operation is performed between the two tasks, avoiding the fusion of more redundant high-level features from the modality conversion and segmentation tasks. yes After the features are generated by the LMSF module, the specific operation of MFCE is as follows.

[0112] Taking the modal conversion intermediate features as an example to enhance complementary information, first and Splicing is performed on the channel dimension, the multi-scale semantic information obtained by the encoder is integrated into the segmentation feature, and then the convolution operation is performed to restore it to the same level as The same channel gets .

[0113] ;

[0114] The correlation and dependency between the entire channels are calculated by combining the maximum pooling operation and the average pooling adaptive layer, and a global channel dependency representation vector is generated. .

[0115] ;

[0116] in and They represent the adaptive average pooling layer and the maximum pooling layer respectively.

[0117] In order to emphasize the features related to segmentation and suppress irrelevant information, the MLP layer is used to compress the spatial information into a channel vector. The convolution kernel size used in the MLP layer is Two convolution kernels , The full connection operation is replaced by the ReLU layer to reduce the number of parameters. Then the channel vector is sigmoid operated, the result is broadcasted on each channel and then combined with Perform element-by-element multiplication and perform the result Convolution operation, finally this part of information is fused into the In this way, the segmentation network can focus more attention on the target pancreas segmentation area. The specific formula is as follows:

[0118] ;

[0119] in is the pancreas segmentation prediction map of the main modality image, is the ReLU activation function, is the sigmoid function, The operation represents broadcasting the channel vector over the last two dimensions and then summing Perform element-wise multiplication.

[0120] Similarly, when the intermediate features of the segmentation task are enhanced complementary information, The generation process and Similarly, the specific formula process is as follows:

[0121] ;

[0122] In order to ensure that the above encoder and decoder can have strong segmentation and modality conversion capabilities and can make full use of the label information of the two modalities, referring to the idea of ​​CycleGAN, this method can not only generate and , you can also generate Corresponding auxiliary modality image pancreas segmentation prediction map And the main modality composite image In general, the multi-task feature complementary enhancement module takes advantage of the different levels of attention paid to semantic information by the modality conversion task and the segmentation task during training, thereby emphasizing the features related to segmentation while suppressing irrelevant redundant information, ultimately making the network pay more attention to the pancreatic region, thereby improving the accuracy of segmentation.

[0123] S5. Use the MFCE module to constrain each other in the process of generating prediction results and synthesized images;

[0124] According to the proposed curvature consistency loss, the segmentation prediction results of the real image and the synthetic image are aligned respectively, and finally an accurate pancreas segmentation prediction result is obtained.

[0125] ;

[0126] in express The pixels, express The pixels, is the curvature linear convolution operation described above. By aligning the segmentation prediction results of the auxiliary modality and the main modality image in the curvature space, the problem of modality invariant information loss during modality conversion is further alleviated to enhance the modality invariant information extraction capability of the shared encoder. Similarly,

[0127] ;

[0128] in express The pixels, express The pixels, is the curvature linear convolution operation.

[0129] S6. Then, the segmentation prediction results of the real image and the synthetic image are aligned according to the proposed curvature consistency loss, and finally an accurate pancreas segmentation prediction result is obtained;

[0130] S61. In order to generate realistic synthesized main modality images, a main modality discriminator for main modality synthesized images is introduced. Conduct adversarial learning.

[0131] ;

[0132] Similarly, in order to generate realistic synthetic auxiliary modality images, an auxiliary modality discriminator for auxiliary modality synthetic images is introduced Conduct adversarial learning.

[0133] ;

[0134] To ensure content consistency, these reconstructed images should be similar to the original images. Therefore, a dual reconstruction consistency loss is defined.

[0135] ;

[0136] S62. In addition to the above loss functions, the loss functions commonly used in image segmentation tasks are the Binary Cross Entropy loss (BCE loss) and Dice loss. Since the pancreas accounts for a small proportion of CT images, its two-category segmentation task has the characteristics of category imbalance. In this case, the cross entropy loss function easily predicts the entire image as a category with a large number of categories. The Dice loss function can solve this problem. The Dice loss function constrains the network by the overlap between the target area of ​​the segmentation and the real area. Therefore, it is necessary to combine BCE and Diceloss to constrain the segmentation. The design is as follows:

[0137] ;

[0138] Where N represents the total number of pixels. and They are and The corresponding gold standard, express The pixels, express The pixels, The value is 0.0001 to prevent the denominator from being zero.

[0139] In summary, the loss function used in the training of the first MFCECC pancreas segmentation subnetwork is As shown in formula (21):

[0140] ;

[0141] S7. In order to generate realistic synthetic images, an auxiliary modality discriminator for auxiliary modality synthetic images is introduced and a discriminator for the main modality and the synthetic images of the main modality Perform adversarial learning and back-propagate the final loss;

[0142] Use the property in SDF that the distance from a pixel to the boundary determines the intensity of the pixel to set For the prediction task of pancreas, two segmentation heads are trained jointly to implicitly learn the location information of the pancreas.

[0143] ;

[0144] like Figure 4 As shown, in the SDFCCFF sub-network framework, BCE (BCEloss) and Dice (Dice loss) loss functions are used as The loss function is calculated by using the mean squared error (MSE) Loss :

[0145] ;

[0146] in, yes Calculated by formula 25, It is the gold standard for high-density pancreatic lesions.

[0147] S8. The second training stage is in the SDFCCFF high-density lesion segmentation sub-network, targeting the pancreas localization problem. Added a signed distance constraint branch based on ;

[0148] Since the pancreas segmentation task always has problems such as small targets and blurred boundaries that are difficult to solve, the surrounding adhesion organs may be mistakenly classified as the pancreas during the segmentation process. Therefore, in order to further improve the pancreas positioning effect and alleviate the problem of information loss in small targets during training, a method of signed distance field constraint and curvature feature fusion is proposed to integrate the pancreas position auxiliary feature information into the lesion area segmentation branch in an explicit way.

[0149] First, in order to integrate more pancreatic related information, we introduced the MFCECC task based on the dual branch. ,Will and The intermediate output features of the first three stages are fed into In the module, the output features are processed twice The convolution operation with a step size of 2 reduces the size of the feature to 1 / 4 of the original size, and finally obtains two multi-scale fusion feature maps. and , parameters that are always frozen during training so that they do not participate in updates.

[0150] S9. In order to further optimize the segmentation results, first freeze No back-propagation is performed to update the parameters, and then and The generated features of different scales are sent to the LMSF module respectively to obtain the corresponding multi-scale fusion features.

[0151] Preferably, the S9 includes:

[0152] Incorporate curvature features into the The specific operations are as follows:

[0153] ;

[0154] in express No. The output characteristics of the stage, express No. The output features of the stage are combined with the output features after the curvature features. Indicates that embedding is required Decoder The curvature feature map of the stage, is the curvature linear convolution operation. It means upsampling to twice the original size through bilinear interpolation. Represents upsampling by inverse convolution multiples (and transform the feature channels into same size).

[0155] Depend on and After the splicing operation, the curvature convolution is performed (the curvature convolution operation restores the feature channel to the same Same) to get, Depend on and The output features of the second stage of the decoder are summed pixel by pixel.

[0156] ;

[0157] In summary, the loss function used in the second stage SDFCCFF high-density lesion segmentation sub-network training is shown in formula (29):

[0158] ;

[0159] S10, finally, the multi-scale fusion features and The generated curvature features of different scales are integrated into Finally, accurate segmentation results of pancreatic high-density lesion areas are obtained.

[0160] The technical solution of the present invention is further described and explained in conjunction with specific embodiments.

[0161] Reference Figure 1 As shown, it is a flowchart of the overall process of the present invention. The experimental data used were collected by Changhai Hospital, and a total of 127 dual-phase 3D abdominal CT images of patients with chronic pancreatitis were included, namely, the plain scan phase and the venous phase. The axial size of each non-contrast enhanced (plain scan phase) and contrast enhanced (venous phase) CT image was 512×512, and the dual-phase CT images of each patient were taken at different time points using different scanning methods. In order to maintain the consistency of image size in the two data sets, all images were uniformly adjusted to 256×256. Due to the different acquisition methods of NCECT and CECT images and individual differences in patients, the number of cross-sectional images ranged from [35,120]. The sagittal and coronal plane intervals of NCECT images and CECT images were [0.51,0.79] mm, and the axial plane interval was 1 mm. A total of 9865 NCECT and 9978 CECT two-dimensional (2D) axial images were obtained. Since it is difficult to label high-density lesions (stones and calcifications) in the pancreas in CECT images, only the labeling information of the pancreas is included in CECT images, while the pancreas and its high-density lesion areas are labeled in NCECT images. All pancreas and high-density lesion areas are manually labeled by professional doctors, with the pixels in the pancreas area labeled as 1, the pixels in the high-density lesion area labeled as 2, and the other areas labeled as 0.

[0162] In terms of data preprocessing, the original intensity values ​​of CT images were first truncated to [-100, 240] or [900, 1300] to enhance the details of the pancreas and its high-density lesions. Secondly, all data images were normalized to reduce the data differences caused by the medical image acquisition process and obtain a relatively uniform data set; finally, due to the diversity of the locations of the pancreas and its high-density lesions, data enhancement was performed on the images before training, such as random flipping, random noise, and random rotation within [-30, 30] degrees, to avoid model overfitting.

[0163] The present invention uses the pytorch deep learning framework, torch version 1.13.0. All training and validation processes are completed on an NVIDIA GeForce RTX 3090 graphics card equipped with 24G video memory. During the training process, the neural network uses a small batch method to read data, and the batch size is set to 2. The optimization experiments all use the Adaptive Moment Estimation (Adam) optimization algorithm, where , , the initial learning rate is set to , the training cycle epochs is set to 120. For the segmentation network of the second phase of UCPSCC, the experiment uses the Stochastic Gradient Descent (SGD) optimization algorithm, where the initial learning rate is 0.01, the optimizer momentum is 0.9, the weight decay coefficient is 1e-4, and the poly learning strategy is used to adjust the learning rate. The formula is shown in (4.1):

[0164] ;

[0165] in and Respectively represent the current number of iterations and the total number of iterations, the basic learning rate Set to 0.01, decreasing exponentially Set to 0.9. The number of epochs for the second-stage segmentation network of UCPSCC is set to 120. After each iteration, the validation set is used to evaluate the segmentation performance of the network.

[0166] Example 2

[0167] As the second embodiment of the present invention, an unpaired multimodal segmentation method based on curvature consistency is provided. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0168] In order to quantitatively evaluate the performance of the method proposed in the present invention, four evaluation indicators commonly used in the field of image segmentation, namely DSC coefficient (Dice similarity coefficient, DSC), Jac coefficient (Jaccard similarity coefficient), precision and recall, are used to measure the performance of each algorithm for pancreas segmentation. The calculation method of each indicator is as follows:

[0169] ;

[0170] TP (True Positive) represents the number of correctly predicted positive pixels, FP (False Positive) represents the number of incorrectly predicted positive pixels, and FN (False Negative) represents the number of negative pixels that were not correctly predicted. DSC is an important evaluation indicator in image segmentation tasks, which is used to measure the similarity between the predicted results and the standard results. Its value close to 1 indicates a high similarity. The Jac indicator represents the ratio of the intersection and union between the predicted results and the standard results, while the precision measures the proportion of pixels predicted correctly among the pixels predicted by the network as positive, and the recall rate refers to the proportion of positive pixels in the standard that are correctly predicted by the network.

[0171] This part conducts a verification experiment on the proposed UCPSCC segmentation method, and the experiment is mainly divided into five parts. The first part is to conduct an ablation experiment on the MFCECC method in the first stage to verify the effectiveness of the multi-task feature complementary enhancement (MFCE) module, lightweight multi-scale feature fusion (LMSF) module and curvature consistency loss method and their contribution to improving the performance of pancreatic segmentation; the second part is to compare the MFCECC method with the most advanced multimodal unpaired medical image segmentation method to prove the superiority of the algorithm in pancreatic segmentation; the third part is to select the pancreatic PHDLA segmentation backbone network in the second stage, using multiple networks to segment the pancreatic PHDLA in NCECT images, in order to prove the superiority of Unet encoding; the fourth part is to conduct an ablation experiment on the SDFCCFF method in the second stage to verify the effectiveness of the signed distance loss method and the signed distance field constraint and curvature feature fusion module and their contribution to improving the performance of high-density lesion segmentation; the fifth part compares the SDFCCFF method with the most advanced multimodal unpaired medical image segmentation method to prove the superiority of the algorithm in the segmentation of the pancreas and its high-density lesions. The multimodal data of this experiment are CECT images and NCECT images. In this experiment, CECT images are used as auxiliary modalities and NCECT images are used as main modalities, aiming to improve the segmentation performance of the pancreas and its high-density lesions in NCECT.

[0172] As shown in Table 1, Table 1 shows the ablation experiment results of the MFCECC segmentation method.

[0173] Table 1

[0174]

[0175] In order to verify the effectiveness of the MFCECC method proposed in the first stage of UCPSCC, ablation experiments were conducted on the MFCE module, LMSF module and curvature consistency loss method, and the framework without MFCE module, LMSF module and curvature consistency loss method was used as the Baseline. Net2-1 is the result of adding the MFCE module to the Baseline, Net2-2 adds the LMSF module to Net2-1, and Net2-3 adds the multi-stage style inconsistency estimation loss to Net2-2. UNet (NCECT) and UNet (CECT) use Unet to perform single-modal pancreas segmentation on NCECT and CECT images respectively.

[0176] After ablation experiments on the MFCECC method, we obtained Table 1, and the best accuracy results are bolded. From the various indicators, it can be seen that when Unet is used to segment NCECT images separately, the accuracy of Recall, Precision, Jac, and DSC is 76.39%, 78.66%, 64.29%, and 75.53%; when Unet is used to segment CECT images separately, the accuracy of Recall, Precision, Jac, and DSC is 79.40%, 81.94%, 68.21%, and 78.45%, proving that CECT images have a better representation of the pancreas than NCECT images. From the various indicators of the Baseline, it can be seen that the multimodal segmentation basic framework proposed in this section has improved the accuracy of Recall, Jac, and DSC by 2.61%, 0.01%, and 0.53% respectively compared with UNet (NCECT). Therefore, it is also proved that when unpaired multi-modal CT images are used, more pancreas-related feature information can be provided to the network, so that better segmentation results can be obtained than independent single modalities. In addition, the synthesized images after modality conversion can also be viewed as implicit data augmentation for segmentation.

[0177] Net2-1 adds the MFCE method to the Baseline, which better preserves the structural details and spatial information of the target organ through multi-task feature complementarity, and improves Recall, Jac, and DSC by 1.96%, 1.29%, and 1.02% respectively compared to the Baseline. Net2-2 adds the LMSF module to Net2-1, which captures multi-level information from low-level texture features to high-level semantic knowledge features through the encoder, which can help the network better understand and capture features at different scales in the image, and help improve the model's detection and segmentation capabilities for targets of different scales. Therefore, Recall, Precision, Jac, and DSC are improved by 1.82%, 0.82%, 1.83%, and 1.44% respectively compared to the Baseline. Net2-3 adds a curvature consistency loss method to Net2-2, using curvature features to align the segmentation prediction results of the auxiliary modality and the main modality image, further alleviating the problem of modality information loss during modality conversion, while enhancing the shared encoder's ability to extract modality-invariant information, thereby improving Recall, Precision, Jac, and DSC by 1.52%, 3.49%, 3.50%, and 2.72% respectively compared to the Baseline.

[0178] As shown in Table 2, Table 2 shows the comparative experimental results of the MFCECC segmentation method and other unpaired multimodal segmentation algorithms.

[0179] In order to verify the superiority of the MFCECC pancreas segmentation method, this section compares the MFCECC pancreas segmentation method with some advanced multimodal unpaired image segmentation methods, including CPCL, PSTNet, SSUMMLLE, UMSA, MixStyle, CycleGAN, SpBTS, UGAN, and MCTHNet. Among them, UNet (Joint) means training a Unet network with joint NCECT and CECT images.

[0180] Table 2

[0181]

[0182] Table 2 shows the experimental results with CECT as the auxiliary modality and NCECT as the main modality, and the best segmentation performance is bolded. As can be seen from the table, the Recall, Precision, Jac and DSC scores of the full-supervision model of the main modality are 76.39%, 78.66%, 64.29% and 75.53% respectively, while the Recall, Precision, Jac and DSC scores of UNet (Joint) are 77.74%, 76.72%, 62.90% and 74.82% respectively, which are 1.94%, 1.39% and 0.71% lower than that of NCECT full supervision in terms of Precision, Jac and DSC scores respectively. This also verifies that there is indeed a certain modality difference between NCECT and CECT images, so the direct participation of unpaired CECT images in training will lead to a decrease in performance. The MFCECC pancreas segmentation method proposed in Chapter 2 achieved 80.52%, 80.51%, 67.80% and 78.78% in Recall, Precision, Jac and DSC respectively. Especially in DSC, the most important indicator in the segmentation task, the MFCECC method achieved a significant improvement and obtained the best segmentation effect in multiple indicators.

[0183] In order to verify the superiority of Unet for pancreatic PHDLA feature extraction, Unet was compared with other segmentation algorithms. The experiment used NCECT images as training data with a size of 512 × 512. A total of five basic network experiments were conducted, namely FCN, SegNet, CENet, CPFNet, CBAM, ResUNet and Unet, and each comparison basic network used open source code for experiments.

[0184] Table 3 shows the experimental results of pancreatic PHDLA in NCECT images, and the best segmentation performance is bolded. As can be seen from the table, Unet has scores of 88.76%, 79.08%, 72.81% and 79.67% in Recall, Precision, Jac and DSC, respectively, and has achieved the best accuracy in many indicators. It is 2.97%, 1.36%, 0.81% and 0.8% higher than the second-best network CPFNet in Recall, Precision, Jac and DSC, respectively. However, the Precision index is lower than that of CENet, that is, some PHDLA in non-pancreatic areas are mistakenly divided into PHDLA in the pancreas by the network. Therefore, the following experiments will also verify the compensation of the proposed method for this Precision index. In general, the Unet encoder can capture the features of pancreatic PHDLA of different shapes, sizes and locations through its powerful feature extraction ability and effective processing of contextual information, helping the network to segment more accurately.

[0185] As shown in Table 3, Table 3 is the results of the pancreatic PHDLA basic experiment.

[0186] Table 3

[0187]

[0188] In order to verify the effectiveness of the SDFCCFF method proposed in the second stage of UCPSCC, this section conducts ablation experiments on the LMSF module, signed distance loss, and signed distance field constraint and curvature feature fusion module, and compares them with the separate Unet segmentation network to demonstrate the rationality of the method. Net2-4 is the result obtained by adding signed distance loss to Unet, Net2-5 adds signed distance field constraint and curvature feature fusion module without LMSF module on the basis of Net2-4, and Net2-6 adds signed distance field constraint and curvature feature fusion module of LMSF module on the basis of Net2-5.

[0189] The results of the ablation experiment are shown in Table 4, and the best accuracy results are presented in bold. As can be seen from the table, Net2-4 solves the misclassification problem of non-pancreatic organs and tissues by adding the signed distance branch on the basis of Unet, thus improving Recall, Precision, Jac, and DSC by 0.38%, 1.93%, 3.83%, and 2.15% respectively compared with UNet (Single). Net2-5 adds the signed distance field constraint and curvature feature fusion module without the LMSF module, and integrates the auxiliary feature information of the pancreas' position into the lesion area segmentation branch in an explicit way, further improving the pancreas' positioning effect, thus improving Recall, Precision, Jac, and DSC by 5.88%, 0.44%, 4.12%, and 2.88% respectively compared with UNet (Single). Based on Net2-5, Net2-6 connects the LMSF module to the multi-scale features of the encoder, improving Recall, Precision, Jac, and DSC by 4.00%, 3.24%, 5.09%, and 4.07% respectively compared with UNet (Single), further optimizing the boundary segmentation problem of the lesion, and once again proving the fusion capability of the multi-scale features of the LMSF module.

[0190] As shown in Table 4, Table 4 shows the ablation experiment results of the SDFCCFF segmentation method.

[0191] Table 4

[0192]

[0193] In order to verify the superiority of the pancreatic PHDLA segmentation method, the SDFCCFF segmentation method in UCPSCC is compared with some advanced multimodal unpaired image segmentation methods, aiming to compare the methods to improve the segmentation accuracy of pancreatic PHDLA through the localization information of the pancreas. From the comparative experiments and analysis in Table 2, it can be seen that CPCL, PSTNet, SSUMMLLE, UMSA, MixStyle, CycleGAN, SpBTS, UGAN and MCTHNet in the comparison methods have obtained better pancreatic localization information compared with the pancreatic segmentation of a single phase (NCECT). However, due to the lack of the gold standard of high-density lesions in CECT (venous phase) images, it cannot be completed in the same phase as pancreatic segmentation. From the experiments in Table 3, it can be seen that Unet has the best feature extraction ability for the lesion segmentation of small targets. Therefore, in the experiments in this section, Unet is used to train a pancreatic high-density lesion segmentation network. During the test, the pancreatic prediction area generated by the above multimodal comparison method is used as prior information to further optimize the segmentation accuracy of high-density lesions, and the corresponding segmentation accuracy is compared with the proposed method.

[0194] As shown in Table 5, Table 5 shows the comparative experimental results of UCPSCC and other unpaired multimodal segmentation algorithms in pancreatic PHDLA segmentation.

[0195] Table 5

[0196]

[0197] As shown in Table 6, Table 6 shows the overall experimental results of the comparison between UCPSCC and other unpaired multimodal segmentation algorithms in pancreas and PHDLA segmentation.

[0198] Table 6

[0199]

[0200] Table 6 shows the comparative experimental results of UCPSCC and other unpaired multimodal segmentation algorithms on PHDLA segmentation in NCECT images. Table 6 shows the overall experimental results of UCPSCC and other unpaired multimodal segmentation algorithms on pancreas and PHDLA segmentation, and the best segmentation performance is bolded. It can be seen that after introducing the location information of the pancreas as prior information, each method has a certain performance improvement on pancreas PHDLA, and it can be analyzed from the specific data that the more accurate the pancreas segmentation, the more obvious the corresponding pancreas PHDLA performance improvement. At the same time, due to the difficulty of pancreatic small target segmentation, once the pancreas positioning deviation is large, the pancreas PHDLA segmentation performance will be seriously affected. Therefore, efficient use of pancreas positioning prior information is of great significance to PHDLA segmentation. A method of signed distance field constraint and curvature feature fusion is proposed to integrate the position auxiliary feature information of the pancreas into the lesion region segmentation branch in an explicit way. The proposed method improves the Recall, Precision, Jac, and DSC indicators by 7.18%, 0.36%, 2.37%, and 1.60% respectively compared with the suboptimal network SpBTS, thereby alleviating the stones and calcification over-segmentation phenomenon in the non-pancreatic area and the missed segmentation phenomenon in the pancreatic area. From the perspective of the pancreas and its PHDLA as a whole, the proposed UCPSCC method improves the Recall, Precision, Jac, and DSC indicators by 3.57%, 1.24%, 2.31%, and 1.14% respectively compared with the suboptimal network SpBTS. The superiority of the UCPSCC algorithm is demonstrated by these two tables.

[0201] The technical benefits of the present invention are as follows: (1) The present invention proposes a multi-task feature complementary enhancement module method; (2) The present invention proposes a curvature consistency loss function; (3) The present invention proposes a signed distance field constraint and curvature feature fusion method; (5) Compared with common segmentation methods, the method proposed by the present invention can further improve the segmentation performance.

[0202] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0203] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0204] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0205] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0206] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

[0207] Example 3

[0208] The third embodiment of the present invention provides an unpaired multimodal segmentation system based on curvature consistency, including: a data preparation module, an MFCECC construction module, an SDFCCFF construction module, an MFCECC training module, a multi-task feature complementary enhancement module, a curvature alignment module, an adversarial learning module, an SDFCCFF training module, an optimization module, and a feature fusion module;

[0209] The data preparation module obtains the original training set;

[0210] The MFCECC construction module builds the first-stage multi-task feature complementary enhancement and curvature consistency pancreas segmentation sub-network of the UCPSCC segmentation network;

[0211] SDFCCFF construction module, constructing the second stage of UCPSCC segmentation network, signed distance field constraint and curvature feature fusion, signed distance field constraint and curvature feature fusion, pancreatic high-density lesion segmentation sub-network;

[0212] MFCECC training module, the first stage of UCPSCC segmentation network is trained in the multi-task feature complementary enhancement and curvature consistency pancreas segmentation sub-network;

[0213] Multi-task feature complementary enhancement module, which uses MFCE module to constrain each other in the process of generating prediction results and synthesized images;

[0214] The curvature alignment module aligns the segmentation prediction results of the real image and the synthetic image according to the curvature consistency loss to obtain accurate pancreas segmentation prediction results;

[0215] The adversarial learning module introduces the discriminator of the auxiliary modality for the auxiliary modality synthetic image and the discriminator of the main modality for the main modality synthetic image to perform adversarial learning, and back-propagates the final loss;

[0216] SDFCCFF training module,The second training stage of the UCPSCC segmentation network is in the signed distance field constraint and curvature feature fusion,signed distance field constraint and curvature feature fusion pancreatic high-density lesion segmentation subnetwork,,which adds a signed distance constraint branch based on the segmentation head of the pancreatic high-density lesion area;

[0217] The optimization module optimizes the segmentation results. First, the feature extractor is frozen without back-propagation to update the parameters. The features of different scales generated by the modality-invariant content feature encoder and the feature encoder are respectively sent to the lightweight multi-scale feature fusion module to obtain the corresponding multi-scale fusion features.

[0218] The feature fusion module integrates the multi-scale fusion features and curvature features of different scales generated by the signed distance constraint branch into the segmentation head of the pancreatic high-density lesion area, and finally obtains accurate segmentation results of the pancreatic high-density lesion area.

[0219] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. Unpaired multimodal segmentation method based on curvature consistency, characterized by: include, The main modal image x t and the auxiliary modality image x s Input into the multi-task feature complementary enhancement and curvature consistency pancreas segmentation sub-network MFCECC to generate the main modality synthetic image x s→t , auxiliary modality reconstructed image x s→t→s , auxiliary modality synthetic image x t→s , main modality reconstructed image x t→s→t , pancreas segmentation prediction map p of the main modality image t , Main modality synthetic image pancreas segmentation prediction map p s→t , auxiliary modality image pancreas segmentation prediction map p s , auxiliary modality synthetic image pancreas segmentation prediction map p t→s ; Input the main modality image into the pancreatic high-density lesion segmentation sub-network SDFCCFF fused with signed distance field constraint and curvature feature to obtain the signed distance prediction map of the pancreatic area And the pancreatic high-density lesion segmentation prediction map In the first sub-network MFCECC, the pancreas segmentation prediction map p is generated for the main modality image. t And auxiliary modality synthetic image pancreas segmentation prediction map p t→s Design the main modality curvature consistency loss, and predict the pancreas segmentation map p of the auxiliary modality image. s And the main modality composite image pancreas segmentation prediction map p s→t Design auxiliary modal curvature consistency loss to alleviate the loss of modal invariant information during modal conversion; In the second sub-network SDFCCFF, the signed distance loss is designed to solve the over-segmentation phenomenon in the non-pancreatic region and the missed segmentation phenomenon in the pancreatic region; The multi-task feature complementary enhancement and curvature consistency pancreas segmentation sub-network MFCECC contains 11 sub-modules, including 3 encoders, modality invariant content feature encoder E cont , feature encoder for the main modality style Feature encoder for auxiliary modality style Decoder Dec for main modality synthetic image T , decoder Dec for auxiliary modality synthetic images S , segmentation head Seg for auxiliary modality S And the segmentation head Seg for the main mode T , two discriminators for auxiliary modality and main modality synthetic images, one multi-task feature complementary enhancement module MFCE for auxiliary modality and one for main modality, where the subscript S represents the auxiliary modality and the subscript T represents the main modality; The signed distance field constraint and curvature feature fusion pancreatic high-density lesion segmentation sub-network SDFCCFF contains 5 modules, including an independent encoder E and a modality-invariant content feature encoder E shared with MFCECC cont , 1 signed distance field fusion curvature feature fusion module, 1 signed distance constraint branch Seg sdf and a segmentation head Segment of a pancreatic high-density lesion area phdla ; The signed distance field constraint and curvature feature fusion pancreatic high-density lesion segmentation sub-network SDFCCFF includes: first freeze E cont No back-propagation is performed to update the parameters, and then E cont The features of different scales generated by E are sent to the LMSF module respectively to obtain the corresponding multi-scale fusion features. For the pancreas positioning problem, in Seg phdla Added a signed distance constraint branch Seg sdf , and then combine the multi-scale fusion features and Seg sdf The generated curvature features of different scales are integrated into Seg phdla Finally, we get the accurate PHDLA segmentation area; we use the property of the distance from the pixel to the boundary in the signed distance field to determine the intensity of the pixel to set the Seg sdf The decoder's prediction task uses Seg sdf and Seg phdl Co-training implicitly learns the location information of the pancreas, in, Represented as Seg sdf Signed distance prediction map of the pancreatic region corresponding to the decoder branch. E represents the independent encoder in the second sub-network; In the SDFCCFF sub-network framework, the mean square error calculation method is used as Seg sdf The symbol distance loss L sdf : in, Yes t Calculated by formula.

2. The unpaired multimodal segmentation method based on curvature consistency according to claim 1, characterized in that: The multi-task feature complementary enhancement and curvature consistency pancreas segmentation sub-network MFCECC includes x s and x t They are sent to the modality-invariant content feature encoder encoder E cont Extract content features for auxiliary modality images and content features for the main modality image Then x s and x t Send to and In the above example, the style features of the auxiliary modality image are extracted. and style features for the main modality image By adding x s and x t Sent to Seg S and Seg T The pancreas segmentation prediction results p of the auxiliary modality image and the main modality image are obtained respectively. S and p t , x s→t By decoderDec T , and Generate, x t→s By decoderDec S , and After generating and obtaining the synthetic image, x s→t Enter E cont and In the above example, we obtain the content features of the corresponding main modality synthetic image. and style features x t→s Enter E cont and In the above example, we obtain the content features of the corresponding auxiliary modality synthetic image. and style features x s→t→s By Dec S , and Generate, x t→s→t By Dec T , and Generate, treat the synthetic image as a data augmentation sample from the auxiliary modality and the main modality, and get x s→t and x t→s The corresponding p s→t and p t→s , p s→t is the pancreas segmentation prediction map of the main modality synthetic image, p t→s It is the pancreas segmentation prediction map of the auxiliary modality synthetic image; For p t and p t→s Design the principal mode curvature consistency loss for p s and p s→t Design auxiliary modal curvature consistency loss to alleviate the loss of modal invariant information during modal conversion; For x t and x s→t Design a discriminator for the main modality synthetic image, for x s and x t→s Design a discriminator for auxiliary modality synthetic images and obtain the adversarial loss of the two discriminators; For x t and x s→t Design the main mode splitter for x s and x t→s An auxiliary modal segmenter is designed to obtain the segmentation loss of the two segmenters; For x t 、x t→s→t 、x s and x s→t→s Design dual reconstruction consistency loss; In the process of generating pancreas segmentation prediction results and synthetic images, the MFCE module is used to constrain each other. In the MFCE module, the main modality multi-scale content feature F cont , segmentation feature F seg And synthesize the image to generate features F trs Perform fusion to preserve the structural details and spatial information of the target organ; The MFCE module contains a lightweight multi-scale feature fusion submodule LMSF. and Simplified to Represents the modality-invariant content feature encoder E cont The intermediate features obtained before the first three downsamplings in the encoding stage are sent to the switchable normalization layer respectively. The switchable normalization layer consists of batch normalization and group normalization, and the weights of the two are 0.5 respectively. The three normalized output results are sent to the depth-separable convolution. First, a convolution kernel is used to perform 3×3 convolution on each input channel separately to extract spatial features, and then several 1×1 convolution kernels are used to perform independent convolution on each channel. The number of 1×1 convolution kernels is the number of output channels. Then the output features are sent to the channel regularization scaling layer. The feature information of each channel is used by learning the weight of each channel, and then the residual connection is realized with the input of the module to ensure that the feature information at each scale is not lost. The process is described as follows: in, Expressed as The enhanced output results, CS represents the channel regularization scaling layer, DSC represents the depth-separable convolutional layer, and SN represents the switchable normalization layer. and They are the enhanced output results, and then the output results are upsampled and spliced ​​in the channel dimension. Among them, Up2 means upsampling to twice the original size, Up4 means upsampling to four times the original size, and Concat means the concatenation operation on the channel; In order to extract global context information from feature maps at different levels, three depth-wise separable convolutions with different dilation rates are used in parallel, the outputs are concatenated in the channel dimension, and then reconstructed into the final multi-scale fused feature map using 1×1 convolution. The characteristic size and same; Among them, DSC D1 ,DSC D2 and DSC D4 They represent depth-wise separable convolutions with expansion rates set to 1, 2, and 4, respectively. Concat represents the concatenation operation on the channel. Conv 1×1 Represents a convolutional layer with a convolution kernel size of 1×1, a BN layer, and a ReLU layer; Output features at the last stage of the modality conversion task Output features of the last stage of the segmentation task Multi-task feature complementary enhancement operations are performed between Yes F cont After the features are generated by the LMSF module, the specific operation of MFCE is as follows: When the intermediate features of modal conversion are enhanced complementary information, first trs and Splicing is performed on the channel dimension to integrate the multi-scale semantic information obtained by the encoder into the segmentation features, and then the convolution operation is performed to restore it to the same level as F. seg The same channel gets Among them, Conv 3×3 Represents a convolutional layer with a convolution kernel size of 3×3, a BN layer, and a ReLU layer; The correlation and dependency between the entire channels are calculated by combining the maximum pooling operation and the average pooling adaptive layer, and a global channel dependency representation vector is generated. Among them, Avgpool and Maxpool represent the adaptive average pooling layer and the maximum pooling layer respectively; The MLP layer is used to compress the spatial information into a channel vector. In the MLP layer, two convolution kernels W1 and W2 with a convolution kernel size of 1×1 and a ReLU layer are used to replace the full connection operation. The sigmoid operation is performed on the channel vector, and the result is broadcast on each channel and then combined with Perform element-by-element multiplication, perform a 3×3 convolution operation on the result, and finally merge the information into F by element-by-element addition. seg In this way, the segmentation network will focus more on the target pancreas segmentation area, and the formula is expressed as: Among them, p t is the pancreas segmentation prediction map of the main modality image, δ is the ReLU activation function, σ is the sigmoid function, The operation represents broadcasting the channel vector over the last two dimensions and then summing Perform element-wise multiplication; When the intermediate features of the segmentation task are enhanced complementary information, x t→s The generation process and p t Similarly, the formula is expressed as: in, It is represented as F which integrates multi-scale semantic information. seg , z seg Represented as a global channel dependency 3. The unpaired multimodal segmentation method based on curvature consistency according to claim 2, characterized in that: The principal modal curvature consistency loss is expressed as, in, Expressed as the curvature consistency loss for the main mode, Indicates p t The lth pixel in Indicates p t→s The lth pixel in D CConv For the curvature linear convolution operation, by aligning the segmentation prediction results of the main modality and auxiliary modality synthetic images in the curvature space, the problem of modality invariant information loss during modality conversion is further alleviated to enhance the modality invariant information extraction capability of the shared encoder; The auxiliary modal curvature consistency loss is expressed as, in, Expressed as the curvature consistency loss for the auxiliary mode, Indicates p s→t The lth pixel in D CConv The proposed method is a curvature linear convolution operation. By aligning the segmentation prediction results of the auxiliary modality and the main modality synthetic image in the curvature space, it alleviates the problem of modality invariant information loss during modality conversion and enhances the modality invariant information extraction capability of the shared encoder.

4. The unpaired multimodal segmentation method based on curvature consistency according to claim 3, characterized in that: The discriminator for the main modality synthetic image includes an adversarial loss in order to generate a realistic main modality synthetic image. as follows: Among them, D T It is represented as the discriminator for the main modality synthetic image, and Ev(·) represents the expectation; The discriminator for the auxiliary modality synthetic image includes: In order to generate a realistic auxiliary modality synthetic image, a discriminator D for the auxiliary modality synthetic image is introduced. S Conduct adversarial learning and adversarial loss as follows: The dual reconstruction consistency loss includes, The segmentation loss of the segmenter includes, Among them, N represents the total number of pixels, y s and t x s and x t The corresponding gold standard, Represents y s The lth pixel in Represents y t For the l-th pixel in , ε is 0.0001 to prevent the denominator from being 0. Expressed as the binary cross entropy loss for the auxiliary modality, Expressed as the dice similarity loss for the auxiliary modality, Expressed as the binary cross entropy bce loss for the main modality, Expressed as the dice similarity loss for the main modality; The loss function L used in the training of the MFCECC pancreas segmentation subnetwork MFCECC It is expressed as: in, Expressed as the adversarial loss against the main modality, Denoted as the adversarial loss for the auxiliary modality, Expressed as the curvature consistency loss for the main mode, Denoted as the curvature consistency loss for the auxiliary mode, L Cyc Denoted as the dual reconstruction consistency loss, Denoted as the segmentation loss for the auxiliary modality image, Denote as the segmentation loss for the primary modality image.

5. The unpaired multimodal segmentation method based on curvature consistency according to claim 4, characterized in that: The corresponding multi-scale fusion features include: introducing E in the MFCECC task on the basis of the dual branch cont , E cont The intermediate output features of the first three stages of E are sent to the LMSF module respectively, and then the output features are subjected to two 3×3 convolution operations with a step size of 2 to reduce the size of the features to 1 / 4 of the original size, and finally two multi-scale fusion feature maps are obtained. and Always freeze E during training cont The parameters make the frozen parameters not participate in the update; The optimized segmentation result includes integrating the curvature features of the signed distance field fusion into the Segment in a hierarchical manner. phdla In the segmentation process, the specific operation is expressed as: in, Seg sdf The output features of the k+1th stage, Seg phdla The output features of the k+1th stage are the output features of the curvature features fused with the signed distance field. Indicates that Seg needs to be embedded phdla Curvature feature map of the signed distance field fusion at the decoder stage k+1, It means upsampling to twice the original size through bilinear interpolation. Indicates upsampling by deconvolution 2 k-2 multiple; Depend on and After the splicing operation, the curvature convolution is performed to obtain: Depend on and Seg phdla The output features of the second stage of the decoder are obtained by adding pixel by pixel. In the SDFCCFF sub-network framework, the BCE and Dice loss functions are used to calculate the high-density lesion branch loss in, It is the gold standard for high-density pancreatic lesions; In summary, the loss function L used in the training of the SDFCCFF high-density lesion segmentation sub-network is SDFCCFF as follows:

6. A system based on the unpaired multimodal segmentation method based on curvature consistency according to any one of claims 1 to 5, characterized in that: Including data preparation module, MFCECC construction module, SDFCCFF construction module, MFCECC training module, multi-task feature complementary enhancement module, curvature alignment module, adversarial learning module, SDFCCFF training module, optimization module, feature fusion module; The data preparation module obtains the original training set; The MFCECC construction module constructs the first-stage multi-task feature complementary enhancement and curvature consistency pancreas segmentation sub-network of the UCPSCC segmentation network; SDFCCFF construction module, constructing the second stage of UCPSCC segmentation network, the sub-network for pancreatic high-density lesion segmentation, which integrates the signed distance field constraint and curvature feature; MFCECC training module, the first stage of UCPSCC segmentation network is trained in the multi-task feature complementary enhancement and curvature consistency pancreas segmentation sub-network; Multi-task feature complementary enhancement module, which uses MFCE module to constrain each other in the process of generating prediction results and synthesized images; The curvature alignment module aligns the segmentation prediction results of the real image and the synthetic image according to the curvature consistency loss to obtain accurate pancreas segmentation prediction results; The adversarial learning module introduces the discriminator of the auxiliary modality for the auxiliary modality synthetic image and the discriminator of the main modality for the main modality synthetic image to perform adversarial learning, and back-propagates the final loss; SDFCCFF training module,The second training stage of the UCPSCC segmentation network is in the,signed distance field constraint and curvature feature fusion pancreatic high-density lesion segmentation sub-network,,which adds a signed distance constraint branch based on the segmentation head of the,pancreatic high-density lesion region; Optimize the module, optimize the segmentation results, and freeze E first cont Without back-propagation to update parameters, the features of different scales generated by the modality-invariant content feature encoder and the feature encoder are respectively sent to the lightweight multi-scale feature fusion module to obtain the corresponding multi-scale fusion features; The feature fusion module integrates the multi-scale fusion features and curvature features of different scales generated by the signed distance constraint branch into the segmentation head of the pancreatic high-density lesion area, and finally obtains accurate segmentation results of the pancreatic high-density lesion area.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.