A joint segmentation method for pancreas, tumor and duodenum based on dual-domain fusion network

Abstract

The present invention discloses a method for joint segmentation of pancreas, tumor and duodenum based on dual-domain fusion network, which is specifically as follows: using clinical multi-center abdominal CT images as data sets, and preprocessing the abdominal CT using prior information; adopting coarse-fine two-stage segmentation for the processed images, firstly inputting the images into a coarse segmentation network to obtain the approximate location information of the pancreas, pancreatic tumor and duodenum; in the fine segmentation stage, firstly performing dual-domain construction, and secondly, inputting the dual-domain images into a dual-domain fusion segmentation network at the same time, and using a ring-shaped multi-level attention module in the dual-domain fusion segmentation network to obtain global continuous attention in the columnar space domain; respectively encoding the physical space domain and the columnar space domain, and using a cross-domain information fusion module to realize the process of domain fusion; and obtaining the final multi-organ segmentation result in the dual-domain fusion network.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and relates to the segmentation of the pancreas, pancreatic tumors and duodenum, and in particular to a dual-domain spatial fusion network for auxiliary diagnosis of pancreatic ductal carcinoma and its infiltration relationship with the duodenum. Background Art

[0002] Pancreatic ductal adenocarcinoma is one of the most deadly diseases in the digestive system, mainly due to its occult onset, rapid progression and extremely fatal prognosis. In the process of using abdominal CT to treat and diagnose pancreatic ductal carcinoma, the characteristics of the pancreas and pancreatic tumors bring great difficulties. At the same time, the spatial relationship between pancreatic tumors and duodenum is also disturbed by the complex anatomical environment and is difficult to determine. Therefore, the spatial characteristics of the pancreas, pancreatic tumors and duodenum have brought great difficulties to the manual segmentation of pancreatic cancer and the development of automatic segmentation technology. At present, some automatic segmentation algorithms have been applied to the segmentation of pancreatic ductal carcinoma, but the infiltration relationship between the duodenum and pancreatic tumors has rarely been noticed.

[0003] The complex spatial relationship between pancreatic ductal carcinoma and the duodenum poses a huge challenge to the adjuvant treatment of pancreatic ductal carcinoma. First, pancreatic ductal carcinoma and the duodenum have a variety of infiltration relationships, which are divided into whether there is an infiltration relationship. In cases with an infiltration relationship, the degree of infiltration is divided into surface contact only and deep embedding, while there is also a distance difference without an infiltration relationship; secondly, the pancreas, pancreatic tumors and duodenum are in a complex spatial environment. The three target organs are in the middle of the abdomen and are surrounded by multiple other organs, such as the kidneys, liver, aorta, venous artery, stomach, etc., and these organs are similar in nature and have relatively close voxels, so they are difficult to distinguish; finally, there are huge individual differences between different individuals, and the size, shape, position and other characteristics of the three target organs are very different. Therefore, the above characteristics bring great difficulties to the automatic segmentation algorithm.

[0004] In order to solve the challenges brought by the characteristics of pancreatic cancer, automatic abdominal CT segmentation technology is constantly developing. The purpose of automatic pancreatic cancer segmentation is to obtain the pathological characteristics of pancreatic cancer in patients with higher efficiency and higher accuracy for clinical diagnosis. However, the existing algorithms ignore the spatial characteristics of the pancreas, pancreatic tumors and duodenum, which limits the further improvement of the segmentation effect. Summary of the invention

[0005] Purpose of the invention: The present invention aims to provide a method for joint segmentation of the pancreas, tumor, and duodenum based on a dual-domain fusion network. A coarse segmentation network is used to obtain the approximate location information of the pancreatic tumor; on this basis, a columnar spiral decomposition is performed with the tumor as the center, and the physical space domain is cropped to establish a dual domain. In addition, a circular multi-level attention mechanism is used to obtain global continuous attention in the columnar space domain. The physical space domain and the columnar space domain are then encoded separately, and a cross-domain information fusion module is used to realize the process of inter-domain fusion. In order to retain more spatial details, local dense connections and multi-scale feature extraction mechanisms are used in the dual-domain fusion network to obtain the final segmentation results of the pancreas, pancreatic tumors, and duodenum.

[0006] To achieve the above object, the present invention provides a method for joint segmentation of pancreas, pancreatic tumor, and duodenum based on a dual-domain spatial fusion network, comprising the following steps:

[0007] S1: Preprocessing: Unified processing of multi-center abdominal CT images: window width and window position setting, pixel normalization, and image rotation;

[0008] S2: Inputting the preprocessed CT image slices in step S1 into the coarse segmentation network model to obtain the location information of the pancreatic tumor;

[0009] S3: According to the rough segmentation result in step S2, dual domain construction is performed, which includes: by establishing a cylindrical polar coordinate system, the physical domain space voxels are mapped to the cylindrical space, thereby constructing a cylindrical expansion map, and obtaining a physical CT image by cutting out redundant voxels around the pancreatic tumor;

[0010] S4: according to the spatial continuity of the columnar expansion image in step S3, the spatial correlation between pixels is obtained by using the circular multi-level attention, which includes: continuous image cropping, intra-block local correlation module and inter-block context continuity module;

[0011] S5: Input the physical CT image in step S3 and the columnar spatial features in step S4 into the encoder for feature extraction;

[0012] S6: Fusing the columnar-physical dual-domain features in step S5, using a cross-domain feature fusion module to perform dual-domain fusion from the channel and spatial levels, respectively, including two parts: a domain-specific module and a cross-domain compatibility module at each level;

[0013] S7: Decode the dual-domain fusion features in step S6, wherein the entire encoding-decoding structure includes two structures: local dense connection and multi-scale feature extraction mechanism, to obtain the final segmentation results of the pancreas, pancreatic tumor and duodenum.

[0014] The abdominal CT images of multiple centers were uniformly processed, including: (1) setting the window width of the abdominal CT images to 200HU and the window level to 100HU based on the clinical prior information of the pancreas, pancreatic tumors, and duodenum; (2) normalizing the images based on the range of voxel values ​​in the CT images to constrain the range of voxel values ​​to ,(3) The CT images are rotated at 90°, 180° and 270° respectively to achieve data enhancement.

[0015] The rough segmentation process in step S2 includes the following steps: the location information of pancreatic tumor in clinical priors, cropping of CT slices, the corresponding coordinates of the cropped area are length: 100~400, width: 120~480 (both in pixels), and the image is segmented using an encoder-decoder network. The encoder and decoder of the network contain four convolutional layers, which are composed of a 5X5 2D convolutional layer, a batch normalization layer, and an activation function of The encoder adds a maximum pooling layer at the end of each layer, and the decoder adds an upsampling layer at the end of each layer to obtain a coarse segmentation result. .

[0016] The dual domain construction mainly includes the construction of the columnar space domain and the construction of the physical space domain:

[0017] Constructing a cylindrical spatial domain: First, establish a cylindrical coordinate system based on the above rough segmentation results The center of the pancreatic tumor is located in the middle, and a three-dimensional coordinate space is established, in which the center point of the tumor is the center of the coordinate system ,Sure , , The direction of the axis is determined, and the corresponding polar angle, azimuth and radius are initialized using the columnar polar coordinate equation. Then, the voxel values ​​in the columnar space are obtained in a spiral manner by simultaneously transforming the polar angle and azimuth, and mapped to the columnar plane to obtain a columnar spiral line. Multiple columnar spiral lines at different radii form a columnar expansion diagram. , thus realizing the construction of the columnar space domain,

[0018] Constructing the physical space domain: Based on the above rough segmentation results The location of the pancreatic tumor in the middle, cropping the CT image, with the tumor center as the cropping center, cropping range: length: -72~72, width: -56~56, height: -88~88, cropping to obtain the image blocks of the pancreas, pancreatic tumor, duodenum and surrounding tissues .

[0019] The acquisition of the circular multi-level attention mechanism includes: continuous image cropping, local correlation module within the block and continuity module between contexts between blocks. The columnar transformation makes the spatial voxels mapped to the columnar space domain. First, the columnar expansion map is Perform vertical cropping to preserve the original continuity of the image and obtain multiple slices , the continuous voxel correlation is obtained within and between blocks, and the correlation is obtained by the following formula:

[0020]

[0021] Among them, is a linear layer, Represents a slice, is the number of consecutive slices, represents the position code, Represents the slice after position encoding, represent After linear transformation and feature information obtained by attention mechanism, , , , They represent the feature vectors of the input slices respectively. The intra-block feature is implemented based on each cut block, while the inter-block feature regards each cut block as a pixel, obtains the correlation between each cut block, and finally adds the intra-block local features and the inter-block context features together to obtain the annular multi-level spatial features. .

[0022] To extract dual-domain features, the dual-domain features are input into the encoder at the same time. The encoder consists of four layers of 5X5 2D convolutional layers, a normalization layer of Batch Normalization, an activation layer with Relu activation function, and a maximum pooling layer Maxpooling layer, so the dual-domain features are obtained. and , .

[0023] The dual-domain features are fused simultaneously from the channel level and the spatial level. Each level consists of a domain-specific module and a cross-domain compatibility module. The specific implementation formula of the domain-specific module in the columnar space domain is as follows:

[0024]

[0025]

[0026] in, Represents the feature block in the columnar space domain, The size is equal to the columnar feature block, is the number of multi-head attention in the columnar space domain, which is a hyperparameter and is set to 8. , , are three different space vectors of the feature blocks in the columnar space domain. By calculating the internal correlation of the three space vectors, the final features of the domain-specific module in the columnar space are obtained. express;

[0027] The specific implementation formula of the domain-specific module in the physical space domain is as follows:

[0028]

[0029]

[0030] in, Represents the feature block in the physical space domain, is equal to the size of the physical feature block, is the number of multi-head attention in the physical space domain, which is a hyperparameter and is set to 8. , , are three different space vectors of the feature block in the physical space domain. By calculating the internal correlation of the three space vectors, the final features of the domain-specific module in the physical space are obtained. express;

[0031] The cross-domain compatible module is specifically expressed as follows:

[0032]

[0033] in It is the result of the fusion of columnar space domain and physical space domain. is equal to the size of the feature block, is the number of multi-head attention, set to 4. Finally, the fused features, columnar space domain features, and physical space domain features are residually connected to obtain the final result. .

[0034] After obtaining the features of each layer, in order to retain more spatial details, we designed two structures in the entire encoding-decoding structure: local dense connection and multi-scale feature extraction mechanism:

[0035] Local dense connection: In order to retain more image detail features, we pass the local channel features of each layer backward for further fusion and extraction. The specific description is shown in the following formula:

[0036]

[0037] in represents a composite operation function, replacing it with group normalization, rectified linear units, or convolution, express Operations, is the local channel quantity of the feature map representing these convolutional layers, for The characteristics of the layer,

[0038] Multi-scale feature extraction mechanism: A multi-scale feature extraction mechanism is added to the underlying segmentation network. The specific description is shown in the following formula:

[0039]

[0040] in represents the sigmoid function, is the dilated convolution, represents the input features, It means that after the expansion rate The features after the dilated convolution and sigmoid are as follows: is the dilation rate in the multi-scale feature extraction mechanism, =1,2,3,5, Represents the concat operation, It is the final output result of the multi-scale feature extraction mechanism. Through the above mechanism, the final segmentation results of the pancreas, pancreatic tumor and duodenum are obtained.

[0041] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0042] 1. The present invention makes full use of the spatial relationship between the pancreas, pancreatic tumors and duodenum, and proposes a joint segmentation method for the pancreas, pancreatic tumors and duodenum based on a dual-domain spatial fusion network;

[0043] 2. The present invention uses cylindrical spiral decomposition to map the physical domain spatial features into the cylindrical spatial domain, which not only highlights the spatial relationship between pancreatic tumor and duodenum infiltration, but also retains the correlation between voxels in the physical spatial domain;

[0044] 3. The present invention adopts a ring-shaped multi-level attention mechanism for images in the columnar space domain, so that the pixels in the columnar space domain have the relevance and continuity of the abdominal physical space;

[0045] 4. The present invention adopts a dual-domain feature fusion module to simultaneously fuse at the channel level and the spatial level. Each level consists of a domain-specific module and a cross-domain integration module, which further improves the segmentation accuracy.

[0046] 5. The present invention designs a local dense connection and multi-scale feature extraction mechanism in the entire encoding-decoding structure to fully utilize the detail information in the image features. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic diagram of the topological structure of the pancreas, pancreatic tumor, and duodenum joint segmentation method based on a dual-domain spatial fusion network provided by the present invention.

[0048] Figure 2 A schematic diagram of a topological structure of a columnar spiral decomposition provided by the present invention,

[0049] Figure 3 A schematic diagram of the topological structure of a ring-shaped multi-level attention mechanism provided by the present invention,

[0050] Figure 4 A schematic diagram of the topological structure of a cross-domain information fusion module provided by the present invention,

[0051] Figure 5 A schematic diagram of the topological structure of a local dense connection and multi-scale feature extraction mechanism provided by the present invention,

[0052] Figure 6 A schematic diagram of a process flow of a method for joint segmentation of pancreas, pancreatic tumor, and duodenum based on a dual-domain spatial fusion network provided by the present invention.

[0053] Figure 7 Schematic diagram of the segmentation results of the pancreas, pancreatic tumor and duodenum according to the present invention and other segmentation methods,

[0054] Figure 8 The figure is a schematic diagram of the segmentation results of the pancreas, pancreatic tumor and duodenum according to the ablation experiment results of the present invention.

[0055] Fig. 9 This is a schematic diagram of the infiltration relationship between pancreatic tumor and duodenum and other segmentation results of the present invention.

[0056] Fig.10 It is a clustering diagram of the segmentation results of the present invention based on the classification method. DETAILED DESCRIPTION

[0057] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0058] Embodiment: In the process of segmenting pancreatic ductal carcinoma and its infiltration relationship with the duodenum, the spatial dependency between the three target organs helps to solve the challenges brought by the specificity of the target organs, but the existing methods ignore the above spatial characteristics. Therefore, this method uses a dual-domain fusion network to establish a columnar-physical dual-domain space through columnar spiral decomposition and physical space clipping; at the same time, in order to retain the spatial continuity in the columnar space, a circular multi-level attention mechanism is designed to explore the spatial correlation under different continuous slices; in order to fully realize the fusion of the dual domains and realize the full utilization of spatial information, the present invention designs a cross-domain information fusion module, which not only retains the integrity of the space but also highlights the infiltration spatial relationship between pancreatic tumors and duodenum; in order to fully retain the detailed features in the network model, the present invention also designs a local dense connection and multi-scale feature extraction mechanism in the segmentation network to improve the final segmentation accuracy of the pancreas, pancreatic tumors and duodenum and the accuracy of the infiltration relationship between pancreatic tumors and duodenum.

[0059] like Figure 5 FIG. 1 is a flow chart of the present invention, a method for joint segmentation of pancreas, pancreatic tumor, and duodenum based on a dual-domain spatial fusion network, comprising the following steps:

[0060] S1: Based on the characteristics of abdominal CT images from multiple centers, image preprocessing was performed, including: (1) Clinical prior information about the pancreas, pancreatic tumors, and duodenum, setting the window width of the abdominal CT image to 200HU and the window level to 100HU. (2) The range of voxel values ​​in the CT image was normalized, and the range of voxel values ​​was constrained to [0, 1]. (3) The CT image was rotated at 90°, 180°, and 270° to achieve data enhancement.

[0061] S2: According to the location information of pancreatic tumors in clinical priors, the CT slices are cropped. The corresponding coordinates of the cropped area are length: 100~400, width: 120~480 (both in pixels). The image is segmented using an encoder-decoder network. The encoder and decoder of the network contain four convolutional layers, which are 5X5 2D convolutional layers, Batch Normalization normalization layers, and activation layers with Relu activation function. The encoder adds a maximum pooling layer at the end of each layer, and the decoder adds an upsampling layer at the end of each layer to obtain a rough segmentation result. .

[0062] S3: Based on the above rough segmentation results, dual domains are constructed, which mainly includes the construction of the columnar space domain and the construction of the physical space domain:

[0063] Constructing a cylindrical space domain: Figure 2 As shown, first, a cylindrical coordinate system is established, and the above rough segmentation results The center of the pancreatic tumor is located in the middle, and a three-dimensional coordinate space is established, in which the center point of the tumor is the center of the coordinate system ,Sure , , The direction of the axis is used to establish the columnar polar coordinate equation and initialize the corresponding polar angle, azimuth angle and radius. Then, the voxel value in the columnar space is obtained in a spiral manner by simultaneously transforming the polar angle and azimuth angle, and mapped to the columnar plane to obtain a columnar spiral line. Multiple columnar spiral lines at different radii form a columnar expansion diagram. , thereby realizing the construction of the columnar space domain.

[0064] Constructing the physical space domain: Based on the above rough segmentation results The location of the pancreatic tumor in the middle, cropping the CT image, with the tumor center as the cropping center, cropping range: length: -72~72, width: -56~56, height: -88~88, cropping to obtain the image blocks of the pancreas, pancreatic tumor, duodenum and surrounding tissues .

[0065] S4: Based on the spatial correlation in the columnar spiral decomposition, a circular multi-level attention mechanism is obtained, which includes: continuous image cropping, intra-block local correlation module and inter-block context continuity module, as shown in the following figure. Figure 3 The cylindrical transformation maps the spatial voxels into the cylindrical spatial domain, retaining the relative spatial positions of some voxels. However, due to the stretching of the circular spiral, some spatial relationships are destroyed. Therefore, we first expand the cylindrical image. Perform vertical cropping to preserve the original continuity of the image and obtain multiple slices We obtain the correlation of continuous voxels within and between blocks. The correlation is specifically expressed by the following formula:

[0066]

[0067] Among them, is a linear layer, Represents a slice, is the number of consecutive slices, represents the position code, Represents the slice after position encoding, represent After linear transformation and feature information obtained by attention mechanism, , , , They represent the feature vectors of the input slices respectively. The intra-block feature is implemented based on each cut block, while the inter-block feature regards each cut block as a pixel, obtains the correlation between each cut block, and finally adds the intra-block local features and the inter-block context features together to obtain the annular multi-level spatial features. .

[0068] S5: Extract dual-domain features. Mainly input the dual-domain features into the encoder at the same time. The encoder consists of four layers of 5X5 2D convolutional layers, a normalization layer of Batch Normalization, an activation layer with Relu activation function, and a maximum pooling layer Maxpooling layer, so as to obtain dual-domain features. and , .

[0069] S6: Specific Figure 4 As shown in the figure, the dual-domain features are fused at the channel level and the spatial level at the same time. Each level is composed of a domain-specific module and a cross-domain compatibility module. The specific implementation formula of the domain-specific module in the columnar space domain is as follows:

[0070]

[0071]

[0072] in, Represents the feature block in the columnar space domain, The size is equal to the columnar feature block, is the number of multi-head attention in the columnar space domain, which is a hyperparameter and is set to 8. , , are three different space vectors of the feature blocks in the columnar space domain. By calculating the internal correlation of the three space vectors, the final features of the domain-specific module in the columnar space are obtained. express;

[0073] The specific implementation formula of the domain-specific module in the physical space domain is as follows:

[0074]

[0075]

[0076] in, Represents the feature block in the physical space domain, is equal to the size of the physical feature block, is the number of multi-head attention in the physical space domain, which is a hyperparameter and is set to 8. , , are three different space vectors of the feature block in the physical space domain. By calculating the internal correlation of the three space vectors, the final features of the domain-specific module in the physical space are obtained. Indicates; the cross-domain compatible module is specifically expressed as follows:

[0077]

[0078] in It is the result of the fusion of columnar space domain and physical space domain. is equal to the size of the feature block, is the number of multi-head attention, set to 4. Finally, the fused features, columnar space domain features, and physical space domain features are residually connected to obtain the final result. .

[0079] S7: In order to preserve more spatial details, we designed two structures in the entire encoding-decoding structure: local dense connection and multi-scale feature extraction mechanism. Figure 5 As shown:

[0080] Local dense connection: In order to retain more image detail features, we pass the local channel features of each layer backward for further fusion and extraction. The specific description is shown in the following formula:

[0081]

[0082] in represents a composite operation function, replacing it with group normalization, rectified linear units, or convolution, express Operation, is the local channel quantity of the feature map representing these convolutional layers, for Layer features, multi-scale feature extraction mechanism: A multi-scale feature extraction mechanism is added to the bottom layer of the segmentation network. The specific description is shown in the following formula:

[0083]

[0084] in represents the sigmoid function, is the dilated convolution, represents the input features, It means that after the expansion rate The features after the dilated convolution and sigmoid are as follows: is the dilation rate in the multi-scale feature extraction mechanism, =1,2,3,5, Represents the concat operation, It is the final output result of the multi-scale feature extraction mechanism. Through the above mechanism, the final segmentation results of the pancreas, pancreatic tumor and duodenum are obtained.

[0085] In order to prove the effectiveness of the present invention, the present invention also provides comparative experiments and ablation experiments:

[0086] Specifically, the present invention selects CT images from 6 centers, wherein the detailed information of each center image is shown in Table 1:

[0087] Table 1:

[0088]

[0089] In the ablation experiment, the experimental results of this method are compared with those of columnar spiral decomposition, ring multi-level attention mechanism, cross-domain information fusion module and local dense connection. The comparison results are shown in Table 2 and the visualization results are shown in Figure 8 As shown,

[0090] Table 2:

[0091]

[0092] The indicator corresponds to Dice, in %, where CST stands for columnar-physical dual domains, ICCB stands for the intra-block local correlation module in the circular multi-level attention mechanism, IPCB stands for the inter-block context continuity module in the circular multi-level attention mechanism, CPIM stands for the cross-domain information fusion module, and LCDC stands for local dense connection.

[0093] In the comparative experiment, the experimental results of this method are compared with those of V-mesh, CAS-Net, ARU-Net, CAN, RunCAN, and CPFNet. The comparison results are shown in Tables 3 to 8 and Figure 7 As shown,

[0094] Table 3: Comparative experimental results under the center 1 dataset,

[0095]

[0096] Table 4: Comparative experimental results under the center 2 dataset,

[0097]

[0098] Table 5: Comparative experimental results under the center 3 dataset,

[0099]

[0100] Table 6: Comparative experimental results under the center 4 dataset,

[0101]

[0102] Table 7: Comparative experimental results under the center 5 dataset,

[0103]

[0104] Table 8: Comparative experimental results under the center 6 dataset,

[0105]

[0106] In order to ensure the fairness of the experimental results, the above experiments all performed fine segmentation based on coarse segmentation, where the unit of Dice is %, and the unit of ASD is mm. The present invention can better utilize the spatial correlation of organs to improve the segmentation accuracy of target organs.

[0107] In order to show the accuracy of the above segmentation method in terms of infiltration relationship, Fig. 9 The infiltration accuracy of this method under different infiltration degrees is demonstrated. At the same time, the experimental results of this method are input into the support vector machine, random forest, ResNet, LA-Net, and DLGNet classification networks. The classification results are shown in Table 9.

[0108] Table 9:

[0109]

[0110] Fig.10 The clustering effect of the present invention is demonstrated, wherein our segmentation method has certain advantages in the classification accuracy of infiltration relationships, and the labels of infiltration classification are derived from the clinical pathology gold standard.

[0111] It should be noted that the above embodiments are not intended to limit the protection scope of the present invention, and equivalent changes or substitutions made on the basis of the above technical solutions all fall within the protection scope of the claims of the present invention.

Claims

1. A method for joint segmentation of pancreas, tumor and duodenum based on dual-domain fusion network, characterized in that: The method comprises the following steps: S1: Preprocessing: Unify the multi-center abdominal CT images: window width and window position setting, pixel normalization, image rotation, S2: Input the pre-processed CT image slices in step S1 into the coarse segmentation network model to obtain the location information of the pancreatic tumor. S3: Based on the rough segmentation result in step S2, dual domain construction is performed, which includes: mapping the physical domain space voxels to the columnar space by establishing a columnar polar coordinate system, thereby constructing a columnar expansion map, and obtaining a physical CT image by cutting out redundant voxels around the pancreatic tumor. S4: According to the spatial continuity of the columnar expansion image in step S3, the spatial correlation between pixels is obtained by using the circular multi-level attention, which includes: continuous image cropping, intra-block local correlation module and inter-block context continuity module. S5: Input the physical CT image in step S3 and the columnar space features in step S4 into the encoder for feature extraction. S6: Fusing the columnar-physical dual-domain features in step S5, using a cross-domain feature fusion module to perform dual-domain fusion from the channel and spatial levels, respectively. Each level includes two parts: a domain-specific module and a cross-domain compatibility module. S7: Decoding the dual-domain fusion features in step S6, wherein the entire encoding-decoding structure includes two structures: local dense connection and multi-scale feature extraction mechanism, to obtain the final segmentation results of the pancreas, pancreatic tumor and duodenum; The dual domain construction includes two parts: the construction of the columnar space domain and the construction of the physical space domain: Constructing the cylindrical space domain: First, establish a cylindrical coordinate system based on the above rough segmentation result I slice The center of the pancreatic tumor is located in the middle, and a three-dimensional coordinate space is established, in which the center point of the tumor is the center of the coordinate system. sd , determine the directions of the x, y, and z axes, and use the columnar polar coordinate equation to initialize the corresponding polar angle, azimuth, and radius. Then, by simultaneously transforming the polar angle and azimuth, the voxel value in the columnar space is spirally obtained and mapped to the columnar plane to obtain a columnar spiral line. Multiple columnar spiral lines at different radii constitute a columnar expansion diagram I Cylin d er , thus realizing the construction of the columnar space domain, Construct the physical space domain: Based on the above rough segmentation result I slice The pancreatic tumor is located in the middle, and the CT image is cropped, with the tumor center as the cropping center. The cropping range is: length: -72~72, width: -56~56, height: -88~88, and the image block I of the pancreas, pancreatic tumor, duodenum and surrounding tissues is obtained by cropping. Physics ; The acquisition of the circular multi-level attention mechanism includes: continuous image cropping, local correlation module within the block and continuity module between contexts between blocks. The columnar transformation makes the spatial voxels mapped to the columnar space domain. First, the columnar expansion graph I Cylinder Perform vertical cropping to preserve the original continuity of the image and obtain multiple slices I tokens , the continuous voxel correlation is obtained within and between blocks, and the correlation is obtained by the following formula: Q n =K n =V n =LN(H n-1 ) H n =LN(H′ n )+H′ n Where LN is the linear layer, i represents the slice, n is the number of consecutive slices, and e i represents position coding, H0 represents the slice after position coding, H n Represents H n-1 After linear transformation and feature information obtained by attention mechanism, n≠0, Q n 'K n , V n They represent the feature vectors of the input slices respectively. The intra-block is realized based on each cut block, while the inter-block is to regard each cut block as a pixel, obtain the correlation between each cut block and the cut block, and finally add the local features within the block and the inter-block context features together to obtain the annular multi-level spatial feature I Clinder-CMAM .

2. The method for joint segmentation of pancreas, tumor and duodenum based on dual-domain fusion network according to claim 1, characterized in that: The abdominal CT images of multiple centers were uniformly processed, including: (1) setting the window width of the abdominal CT images to 200HU and the window position to 100HU based on the clinical prior information of the pancreas, pancreatic tumors, and duodenum; (2) normalizing the images based on the range of voxel values ​​in the CT images, and constraining the range of voxel values ​​to [0, 1]; (3) rotating the CT images at 90°, 180°, and 270° to achieve data enhancement.

3. The method for joint segmentation of pancreas, tumor and duodenum based on dual-domain fusion network according to claim 1, characterized in that: The rough segmentation process in step S2 includes the following steps: the location information of pancreatic tumor in clinical prior, the CT slice is cropped, the corresponding coordinates of the cropped area are length: 100-400, width: 120-480, and the image is segmented using an encoder-decoder network. The encoder and decoder of the network contain four convolutional layers, which are 5X5 2D convolutional layers, BatchNormalization normalization layers, and activation layers with activation functions as Relu. The encoder adds a maximum pooling layer at the end of each layer, and the decoder adds an upsampling layer at the end of each layer to obtain a rough segmentation result I slice .

4. The method for joint segmentation of pancreas, tumor and duodenum based on dual-domain fusion network according to claim 1, characterized in that: The dual-domain features are extracted by inputting them into the encoder at the same time. The encoder consists of four 5X5 2D convolutional layers, a Batch Normalization layer, an activation layer with Relu as the activation function, and a Maxpooling layer to obtain the dual-domain features F Cylinder-i and F Physics-i , i=0,l,2,3.

5. The method for joint segmentation of pancreas, tumor and duodenum based on dual-domain fusion network according to claim 1, characterized in that: The dual-domain features are fused simultaneously from the channel level and the spatial level. Each level consists of a domain-specific module and a cross-domain compatibility module. The specific implementation formula of the domain-specific module in the columnar space domain is as follows: Among them, F sd-i Represents the feature block in the columnar space domain, Head sd ×hidden sd The size of the Head is equal to that of the columnar feature block. sd is the number of multi-head attention in the columnar space domain, which is a hyperparameter and is set to 8. sd , K sd , V sd are three different space vectors of the feature blocks in the columnar space domain. By calculating the internal correlation of the three space vectors, the final features of the domain-specific module in the columnar space are obtained. self-sd express; The specific implementation formula of the domain-specific module in the physical space domain is as follows: Among them, F pd-i Represents the feature block in the physical space domain, Head pd ×hidden pd The size of the Head is equal to the physical feature block. pd is the number of multi-head attention in the physical space domain, which is a hyperparameter and is set to 8. pd , K pd , V pd are three different space vectors of the feature block in the physical space domain. By calculating the internal correlation of the three space vectors, the final features of the domain-specific module in the physical space are obtained. self-pd express; The cross-domain compatible module is specifically expressed as follows: Among them, F cross It is the result of the fusion of columnar space domain and physical space domain. Head′ × The size of hidden′ is equal to that of feature block, Head′ is the number of multi-head attention, which is set to 4. Finally, the fused features are residually connected with the columnar space domain features and the physical space domain features to obtain the final result F sd-pd .

6. The method for joint segmentation of pancreas, tumor and duodenum based on dual-domain fusion network according to claim 1, characterized in that: Local dense connection: The local channel features of each layer are passed backward for further fusion and extraction. The specific description is shown in the following formula: F n =O(C(F n-1 ,F n-2 [0;k n-2 ],...,F0[0:k0])) Where O represents a composite operation function, which is replaced by group normalization, rectified linear unit, or convolution, C represents a concat operation, k0, ...k n-2 is the local channel quantity of the feature map of these convolutional layers, F n is the feature of layer n, Multi-scale feature extraction mechanism: A multi-scale feature extraction mechanism is added to the underlying segmentation network. The specific description is shown in the following formula: Where S represents the sigmoid function, DilConv is the dilated convolution, and f input represents the input feature, f′ k It represents the features after dilated convolution with a dilation rate of k and sigmoid, where k is the dilation rate in the multi-scale feature extraction mechanism, k = 1, 2, 3, 5, C represents the concat operation, and F is the final output result of the multi-scale feature extraction mechanism. Through the above mechanism, the final segmentation results of the pancreas, pancreatic tumor and duodenum are obtained.

Images