Prognostic survival analysis method for pancreatic ductal carcinoma based on two-branch adaptive network

Abstract

The present invention discloses a pancreatic ductal carcinoma prognosis survival analysis method based on a dual-branch adaptive network, comprising: first, using multi-center clinical abdominal CT images as a dataset, preprocessing the abdominal CT and performing coarse segmentation to achieve the localization of the pancreas and pancreatic tumors; second, using the segmentation network to construct a pancreas-pancreatic tumor parallel branch to obtain the anatomical characteristics of the pancreas and pancreatic tumors; then, using encoders to extract features from the dual branches respectively, and using a dual-stage pre-training-fine-tuning adaptive training method in the feature extraction stage to deeply extract the anatomical features of the pancreas and pancreatic tumors; finally, the spatial position relationship of the pancreas and pancreatic tumors plays a crucial role in prognostic analysis. A branch-guided fusion module is used to comprehensively capture the complex and diverse spatial position relative relationships of the pancreas and pancreatic tumors, and the output is input into the prognostic network to obtain the final pancreatic ductal carcinoma prognosis survival analysis results.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, relates to prognostic survival analysis of pancreatic ductal carcinoma, and in particular to a prognostic survival analysis based on a double-branch adaptive network for auxiliary diagnosis of pancreatic ductal carcinoma patients. Background Art

[0002] The pancreas is one of the most important organs in the digestive system. It secretes hormones and digestive enzymes commonly used in the digestive process and regulates the body's blood sugar levels, thereby assisting the entire digestive process. Pancreatic ductal carcinoma is a malignant tumor that originates from the pancreatic ductal epithelial cells. Clinically, it is considered the most malignant type of pancreatic cancer. It has the characteristics of insidious onset, rapid progression, and an extremely fatal prognosis. It is known as the "King of Cancer" in the field of oncology. In order to better assist medical personnel in dynamically understanding the course of pancreatic ductal carcinoma patients and to minimize the pain caused by pancreatic ductal carcinoma to patients, prognostic survival analysis of pancreatic ductal carcinoma plays an important role.

[0003] Abdominal CT is a commonly used diagnostic tool for pancreatic ductal carcinoma. As a non-invasive, rapid, and widely available imaging modality, it plays a crucial role in the diagnosis of pancreatic ductal carcinoma. CT images contain multiple anatomical characteristics of the pancreas and pancreatic tumors, such as grayscale, texture, and shape. These characteristics give CT images great potential for prognostic analysis of pancreatic ductal carcinoma. Prognostic analysis based on the clinical features of the pancreas and pancreatic tumors in CT images can stratify patients into high-risk and low-risk groups. This allows medical professionals to tailor treatment strategies to each risk group, alleviating the pain of pancreatic ductal carcinoma to patients. Currently, manual extraction of clinical anatomical features of pancreatic ductal carcinoma from CT images is the primary prognostic method. However, this method relies heavily on clinician experience and subjective judgment, resulting in low efficiency. Furthermore, some features of the pancreas and pancreatic tumors are relatively subtle, and the spatial relationships between them are complex, posing challenges to the clinical diagnosis of pancreatic ductal carcinoma.

[0004] To address the challenges posed by the inherent anatomical characteristics of the pancreas and pancreatic tumors, deep learning-based prognostic survival analysis algorithms for pancreatic ductal carcinoma are under development. Prior art publications have described models that analyze preoperative imaging data of PDAC patients and establish predictive models for PDAC survival and tumor subtypes.

[0005] Parameters such as the maximum standardized uptake value, mean standardized uptake value, metabolic tumor volume, and lesion glycolysis in PDAC patients were studied to achieve prognostic analysis. However, the growth direction, growth rate, and internal texture of pancreatic tumors in CT images are relatively hidden. The anatomical characteristics that affect the prognostic analysis of pancreatic ductal carcinoma in different patients are complex and diverse. The relative spatial position relationship between the pancreas and pancreatic tumors is complex and variable, which limits the performance of the prognostic effect. Therefore, the prognostic survival analysis of pancreatic ductal carcinoma still needs further improvement to meet the clinical needs for the prognosis of pancreatic ductal carcinoma. Summary of the Invention

[0006] Purpose of the invention: The present invention aims to provide a prognostic survival analysis method for pancreatic ductal carcinoma based on a dual-branch adaptive network. First, coarse segmentation is used to achieve the positioning of the pancreas and pancreatic tumors. On this basis, the segmentation network is used to construct a pancreas-pancreatic tumor parallel branch. Based on the segmentation results and segmentation features, the target area and its neighborhood information are fully utilized to obtain the anatomical characteristics of the pancreas and pancreatic tumors. Then, the encoder is used to extract features from the two branches respectively. In the feature extraction stage, a two-stage pre-training-fine-tuning adaptive training method is used to enhance the prognostic model's learning ability for complex anatomical characteristics. Finally, a branch-guided fusion module is used to comprehensively capture the complex and diverse spatial positional relative relationships of the pancreas and pancreatic tumors, and the output is input into the prognostic network to obtain the final pancreatic ductal carcinoma prognostic survival analysis results.

[0007] To achieve the above objectives, the present invention provides a pancreatic ductal carcinoma prognosis survival analysis method based on a two-branch adaptive network, comprising the following steps:

[0008] S1: Preprocessing: A unified preprocessing of multi-center clinical abdominal CT datasets was performed, including: window width and window position adjustment, histogram equalization, Sobel operator, and data enhancement;

[0009] S2: performing coarse segmentation on the pre-processed CT image slices in step S1 to obtain approximate location information of the pancreas and pancreatic tumor;

[0010] S3: Based on the coarse segmentation results in step S2, the CT image is cropped and a pancreas-pancreatic tumor parallel branch structure is constructed using a segmentation network, which includes: pancreas region images, pancreas region features and pancreatic tumor region images, pancreatic tumor region features, so as to fully utilize the information of the pancreas and pancreatic tumor target regions and their neighborhoods;

[0011] S4: performing feature extraction based on the dual-branch input in step S3, including: a pancreas encoder and a pancreatic tumor encoder;

[0012] S5: A two-stage pre-training-fine-tuning adaptive training method is used for the dual-branch features in step S4. The first stage is the pre-training stage, and the second stage is the fine-tuning adaptive stage. The two stages are trained serially to deeply capture the anatomical features of the pancreas and pancreatic tumors, and enhance the prognostic model's ability to learn complex anatomical characteristics.

[0013] S6: Fusing the dual-branch features in step S5, using a branch-guided fusion module to comprehensively capture the complex and diverse spatial relative relationships between the pancreas and pancreatic tumors from the multi-scale feature extraction module and the multi-view information fusion module;

[0014] S7: Input the dual-branch fusion features in step S6 into the prognostic network to obtain the final pancreatic ductal carcinoma prognostic survival analysis results.

[0015] As an improvement of the present invention, a unified preprocessing of clinical abdominal CT datasets from multiple centers is performed, including the following: (1) Window width and window position adjustment: Based on the clinical prior information of the pancreas and pancreatic tumors, the window width of the abdominal CT image is set to 200HU and the window position is set to 100HU, and the image is normalized to constrain the range of voxel values ​​to [0, 1]. (2) Histogram equalization: By changing the histogram of the image, the grayscale of each pixel in the image is changed to improve the contrast of pancreatic tumors in the CT image. (3) Sobel operator: Edge detection is performed on the pancreas and pancreatic tumors to make the contrast between the edge and the fat tissue more obvious. (4) Data enhancement: The CT image is rotated at 90°, 180°, and 270°, and the image is scaled to achieve data enhancement.

[0016] As an improvement of the present invention, the coarse segmentation process in step S2 includes the following steps: the location information of the pancreatic tumor in the clinical prior is used to crop the CT slice, and the corresponding coordinates of the cropped area are length: 100~400, width: 120~480 (both in pixels), and the image is segmented using a segmentation network. The encoder and decoder of the network include four convolutional layers, which are composed of a 3X3 2D convolutional layer, a Batch Normalization normalization layer, and an activation layer with a Relu activation function. The encoder adds a maximum pooling layer at the end of each layer, and the decoder adds an upsampling layer at the end of each layer to obtain a coarse segmentation result. By performing coarse segmentation on CT images, the impact of redundant spatial environment on prognostic analysis can be effectively alleviated.

[0017] As an improvement of the present invention, a pancreas-pancreatic tumor parallel branch structure is constructed based on the segmentation network. First, the CT image is cropped with the tumor center area according to the coarse segmentation result in step S2. The cropping range is: length: -72~72, width: -56~56, height: -88~88 (unit: voxel), and the pancreas and its surrounding tissue images are cropped. Then we use the segmentation network to segment the image. The network's encoder and decoder contain four convolutional layers, which are composed of a 5X5 3D convolutional layer, a batch normalization layer, and an activation layer with a Relu activation function. The encoder adds a maximum pooling layer at the end of each layer, and the decoder adds an upsampling layer at the end of each layer to obtain the segmentation results of the pancreas and pancreatic tumors. and Based on this, we constructed the target area of ​​the pancreas and pancreatic tumors, and we used the segmentation results to obtain a CT area image containing only the pancreas and pancreatic tumors. and To obtain the neighborhood information containing the pancreas and pancreatic tumors, we use the features of the third layer decoder in the segmentation network as another input and Therefore, in the pancreatic branches, we constructed a pancreatic CT regional image and pancreatic field characteristics As input, in the pancreatic tumor branch, we constructed the pancreatic tumor CT region image and pancreatic field characteristics This step can effectively integrate the information of the spatial microenvironment of the pancreas and pancreatic tumors, thereby improving the performance of the prognostic network to a certain extent.

[0018] As an improvement to the present invention, feature extraction is performed on the pancreas-pancreatic tumor parallel branch. First, the dual inputs of pancreas and pancreatic tumor are combined. The specific implementation formula is as follows:

[0019]

[0020] in and is the final input of the CT image and feature combination of pancreas and pancreatic tumor, and C represents the serial operation. and Perform feature extraction. The feature extractor consists of an encoder-decoder structure, which includes four convolutional layers, a 5X5 3D convolutional layer, a batch normalization layer, and an activation layer with a Relu activation function. The encoding layer is added to the end of each layer, and the decoding layer is added to the end of each layer. Finally, the dual-branch feature is obtained. and . Provide rich semantic information for subsequent dual-branch information fusion.

[0021] As an improvement of the present invention, we adopt a pre-training-fine-tuning adaptive training method for the dual-branch feature extraction stage, in which the first stage is the pre-training stage and the second stage is the fine-tuning adaptive stage. In the first stage, we pre-train the pancreas and pancreatic tumor branches respectively, and use the model parameters obtained by pre-training and To initialize the model parameters for the dual-branch feature extraction phase. In the second phase, the model parameters for feature extraction will be updated based on the pre-trained weights as the model is trained. The specific weight change formula is as follows:

[0022]

[0023] in The weight matrix for fine-tuning the pre-trained parameters is added to the pre-trained weight matrix to obtain the final weight matrix and To enable the prognostic network to better handle the complex and diverse characteristics of the pancreas and pancreatic tumors, we adaptively decomposed the fine-tuning weight matrix into two low-rank matrices A and B, thereby simplifying the parameter fine-tuning process. The specific implementation formula is as follows:

[0024]

[0025] in , , , d is Update the rank of the parameter matrix, A and B are the low-rank matrices after decomposition, n and m are the number of rows and columns of the parameter matrix, d is the rank of the parameter matrix, and R represents the set of real numbers. and Represent the weight matrices of pancreas and pancreatic tumor after pre-training update. Decompose into and The process of two matrices makes the matrix Initialization using a random Gaussian distribution provides a more balanced starting point for network training. This step more accurately extracts the anatomical features of the pancreas and pancreatic tumors from CT images and enhances the prognostic model's ability to learn complex anatomical characteristics. Pre-training enables the model to initially locate the target region within CT images and features. Fine-tuning the adaptive phase further refines the detailed features of the pancreas and pancreatic tumors.

[0026] As an improvement of the present invention, the dual-branch features are fused, wherein the fusion module under the branch guidance includes two parts: a multi-scale feature extraction module and a multi-view information fusion module; the dual-branch features are fused, wherein the fusion module under the branch guidance includes two parts: a multi-scale feature extraction module and a multi-view information fusion module, and the multi-scale feature extraction module: the module extracts the features of the dual branches respectively. and For multi-scale extraction, dilated convolution with different expansion rates is used to construct four parallel feature groups to obtain feature information under different receptive fields. First, the four parallel feature groups are constructed. The specific implementation formula is as follows:

[0027]

[0028] in and Indicates that the pancreas and tumor have an expansion rate of The convolutional features under and Represents the features of the dilated convolution features of the pancreas and tumor after being weighted by the Sigmoid function, where The expansion rate can take values ​​of 1, 2, 3 and 5, so we get Four parallel feature groups representing the pancreas, Four parallel feature groups representing tumors, Represents the Sigmoid function. Next, the multi-scale features will be aggregated. The specific implementation formula is as follows:

[0029]

[0030] in Indicates a concatenation operation, and is the output result of the multi-scale feature extraction module.

[0031] Multi-view information fusion module: Through the above multi-scale feature extraction module, the multi-scale features of the pancreas and pancreatic tumors are obtained respectively and Based on this, multi-view information fusion is performed. First, the multi-scale features from the same layer of pancreas and pancreatic tumors are aggregated together, and the dual-branch features are fused based on this. Then, through maximum pooling and average pooling, the key information in the feature channel can be captured from multiple perspectives. Specifically, the multi-layer perception mechanism is used with average pooling and maximum pooling, and the Sigmoid function is applied to calculate the channel attention coefficient K. Then, the attention coefficient is multiplied by the feature to obtain a reweighted feature vector. Finally, these reweighted features are residually connected with the multi-scale pancreatic and tumor features to obtain the final fusion feature. , the specific implementation formula is as follows:

[0032]

[0033]

[0034]

[0035] in Represents a multi-layer perception mechanism, represents the maximum pooling layer, represents the average pooling layer, represents the Sigmoid function, Indicates a concatenation operation, The multi-scale features representing the pancreas and pancreatic tumors are concatenated together. The multi-view information fusion module further fuses the multi-scale features containing spatial information of the pancreas and pancreatic tumors, further characterizing their relative spatial position.

[0036] As an improvement of the present invention, the features after the dual-branch fusion are input into the prognostic network to perform prognostic survival analysis. The prognostic network contains three fully connected layers, and DropOut is added after the first two layers to obtain the final prognostic results.

[0037] Through the above mechanism, we obtained the prognostic survival analysis results of pancreatic ductal carcinoma.

[0038] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0039] 1. This paper fully utilizes the anatomical characteristics and spatial relationship of the pancreas and pancreatic tumors to propose a prognostic survival analysis method for pancreatic ductal carcinoma based on a two-branch adaptive network;

[0040] 2. This paper fully captures the anatomical characteristics and neighborhood information of the pancreas and pancreatic tumors, reduces the interference of redundant information, and proposes a pancreas-pancreatic tumor parallel branching structure based on the segmentation network;

[0041] 3. This invention uses a pre-training-fine-tuning adaptive training approach, where the first stage is the pre-training stage and the second stage is the fine-tuning adaptive stage, allowing the prognostic network to better capture the complex and diverse anatomical characteristics of the pancreas and pancreatic tumors in different patients;

[0042] 4. Aiming at the complex relative spatial relationship between the pancreas and pancreatic tumors, the present invention adopts a branch-guided fusion module and constructs a multi-scale feature extraction module and a multi-view information fusion module to effectively enhance the topological space representation capability of the prognostic model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Schematic diagram of the topological structure of the pancreatic ductal carcinoma prognosis survival analysis method based on a dual-branch adaptive network provided by the present invention,

[0044] Figure 2 Schematic diagram of the pre-training-fine-tuning adaptive training method provided by the present invention,

[0045] Figure 3 A schematic diagram of the topological structure of a branch-guided fusion module provided by the present invention is shown.

[0046] Figure 4 A schematic diagram of a process for prognostic survival analysis of pancreatic ductal carcinoma based on a dual-branch adaptive network provided by the present invention.

[0047] Figure 5 Schematic diagram of Kaplan-Meier survival curve of the present invention in this method and other prognostic survival analysis methods,

[0048] Figure 6 This is a schematic diagram of the present invention's method and other prognostic survival analysis methods on the Grad-CAM image.

[0049] Figure 7 Schematic diagram of the Kaplan-Meier survival curve of the prognostic survival analysis method of the ablation experiment of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0051] Example: In the prognostic survival analysis of pancreatic ductal carcinoma, the anatomical characteristics and complex spatial relationships of the pancreas and pancreatic tumors have a significant impact on prognosis. Therefore, a prognostic survival analysis method for pancreatic ductal carcinoma based on a two-branch adaptive network was designed. This method constructs a parallel pancreas-pancreatic tumor branching structure based on a segmentation network. By introducing target region and neighborhood information, it reduces the interference of redundant information on prognostic performance and provides a solid foundation for subsequent information extraction. Secondly, to address the impact of the complex and diverse anatomical characteristics of the pancreas and pancreatic tumors on the prognostic network, we designed a pre-training-fine-tuning adaptive training method. This method consists of two stages. The first stage is the pre-training stage, in which we initialize the model feature extractor using pre-trained parameters to achieve preliminary localization of the target region in CT images and features. The second stage is the fine-tuning adaptive stage, in which parameters are adaptively adjusted to further refine the detailed features of the pancreas and pancreatic tumors. Finally, to comprehensively capture the spatial relationships between the pancreas and pancreatic tumors, we designed a branch-guided fusion module. The branch-guided fusion module consists of two parts: one is the multi-scale feature extraction module, which performs multi-scale feature fusion on the semantic features of the two branches; the other is the multi-view information fusion module, which fuses the two-branch features from multiple different attention angles.

[0052] like Figure 4 FIG. 1 is a flow chart of the present invention, which shows a method for prognostic survival analysis of pancreatic ductal carcinoma based on a dual-branch adaptive network, comprising the following steps:

[0053] S1: A unified preprocessing of the multi-center clinical abdominal CT dataset was performed, including: (1) Window width and window position adjustment: Based on the clinical prior information of the pancreas and pancreatic tumors, the window width of the abdominal CT images was set to 200HU and the window position was set to 100HU. The images were normalized to constrain the voxel value range to [0, 1]. (2) Histogram equalization: By changing the image histogram, the grayscale of each pixel in the image was changed to improve the contrast of pancreatic tumors in the CT images. (3) Sobel operator: The edge detection of the pancreas and pancreatic tumors was performed to make the contrast between the edge and the fat tissue more obvious. (4) Data enhancement: The CT images were rotated at 90°, 180°, and 270°, and the images were scaled to achieve data enhancement.

[0054] S2: The coarse segmentation process includes the following steps: the location information of pancreatic tumors in clinical priors, the CT slices are cropped, and the corresponding coordinates of the cropped area are length: 100~400, width: 120~480 (both in pixels), and the image is segmented using a segmentation network. The encoder and decoder of the network contain four convolutional layers, which are composed of a 3X3 2D convolutional layer, a BatchNormalization normalization layer, and an activation layer with a Relu activation function. The encoder adds a maximum pooling layer at the end of each layer, and the decoder adds an upsampling layer at the end of each layer to obtain the coarse segmentation result. .

[0055] S3: Construct a pancreas-pancreatic tumor parallel branch structure based on the segmentation network. First, based on the coarse segmentation result in step S2, the CT image is cropped with the tumor center area. The cropping range is: length: -72~72, width: -56~56, height: -88~88 (unit: voxel). The cropped image of the pancreas and its surrounding tissues is obtained. Then we use the segmentation network to segment the image. The network encoder and decoder contain four convolutional layers, which are composed of 5X5 3D convolutional layers, BatchNormalization normalization layers, and activation layers with Relu activation function. The encoder adds a maximum pooling layer at the end of each layer, and the decoder adds an upsampling layer at the end of each layer to obtain the segmentation results of the pancreas and pancreatic tumors. and Based on this, the target area of ​​the pancreas and pancreatic tumors is constructed, and the segmentation results are used to obtain a CT area image containing only the pancreas and pancreatic tumors. and To obtain the neighborhood information containing the pancreas and pancreatic tumors, we use the features of the third layer decoder in the segmentation network as another input and Therefore, in the pancreatic branches, we constructed a pancreatic CT regional image and pancreatic field characteristics As input, in the pancreatic tumor branch, a pancreatic tumor CT region image is constructed and pancreatic field characteristics as input.

[0056] S4: Perform feature extraction on the pancreas-pancreatic tumor parallel branch. First, combine the dual inputs of pancreas and pancreatic tumor. The specific implementation formula is as follows:

[0057]

[0058] in and is the final input of the CT image and feature combination of pancreas and pancreatic tumor, and C represents the serial operation. and Perform feature extraction. The feature extractor consists of an encoder-decoder structure, which includes four convolutional layers, a 5X5 3D convolutional layer, a batch normalization layer, and an activation layer with a Relu activation function. The encoding layer is added to the end of each layer, and the decoding layer is added to the end of each layer. Finally, the dual-branch feature is obtained. and .

[0059] S5: Adopt the pre-training-fine-tuning adaptive training method for the dual-branch feature extraction stage, such as Figure 2 As shown in Figure 1, the first stage is the pre-training stage, and the second stage is the fine-tuning and adaptive stage. In the first stage, the two branches of pancreas and pancreatic tumor are pre-trained respectively, and the model parameters obtained by pre-training are used. and To initialize the model parameters for the dual-branch feature extraction phase. In the second phase, the model parameters for feature extraction will be updated based on the pre-trained weights as the model is trained. The specific weight change formula is as follows:

[0060]

[0061] in The weight matrix for fine-tuning the pre-trained parameters is added to the pre-trained weight matrix to obtain the final weight matrix and To enable the prognostic network to better handle the complex and diverse characteristics of the pancreas and pancreatic tumors, we adaptively decomposed the fine-tuning weight matrix into two low-rank matrices A and B, thereby simplifying the parameter fine-tuning process. The specific implementation formula is as follows:

[0062]

[0063] in , , , d is Update the rank of the parameter matrix, and Represent the weight matrices of pancreas and pancreatic tumor after pre-training update. Decompose into and The process of two matrices makes the matrix Using random Gaussian distribution for initialization provides a more balanced starting point for network training.

[0064] S6: Fuse the dual-branch features, such as Figure 3As shown in the figure, the branch-guided fusion module includes two parts: a multi-scale feature extraction module and a multi-view information fusion module.

[0065] Multi-scale feature extraction structure: This module extracts features from both branches separately. and Perform multi-scale extraction and use dilated convolution with different expansion rates to construct four parallel feature groups to obtain multiple receptive field features. First, we construct four parallel feature groups. The specific implementation formula is as follows:

[0066]

[0067] in and The expansion rate is The convolution features of Represents the Sigmoid function. Next, we will aggregate the multi-scale features. The specific implementation formula is as follows:

[0068]

[0069] in Indicates a concatenation operation, and is the output result of the multi-scale feature extraction module.

[0070] Multi-view information fusion module: Through the above multi-scale feature extraction module, we obtain the multi-scale features of pancreas and pancreatic tumors respectively and , based on this we perform multi-perspective information fusion. First, we aggregate the multi-scale features from the same layer of pancreas and pancreatic tumors, and use this as the basis for dual-branch feature fusion. Then, through maximum pooling and average pooling, we are able to capture key information in feature channels from multiple perspectives. Specifically, we use a multi-layer perception mechanism with average pooling and maximum pooling, and apply the Sigmoid function to calculate the channel attention coefficient K. We then multiply the attention coefficient with the feature to obtain a reweighted feature vector. Finally, we perform a residual connection between these reweighted features and the multi-scale pancreatic and tumor features to obtain the final fusion feature The specific implementation formula is as follows:

[0071]

[0072]

[0073]

[0074] in Represents a multi-layer perception mechanism, represents the maximum pooling layer, represents the average pooling layer, represents the Sigmoid function, Indicates a concatenation operation, Features representing the pancreas and pancreatic tumors after multi-scale features are concatenated together.

[0075] S7: The dual-branch fusion features are input into the prognostic network for prognostic survival analysis. The prognostic network consists of three fully connected layers, with DropOut added after the first two layers to obtain the final prognostic results.

[0076] Through the above mechanism, we obtained the final prognostic survival analysis results of pancreatic ductal carcinoma.

[0077] In order to prove the effectiveness of the present invention, the present invention also provides comparative experiments and ablation experiments:

[0078] Specifically, the present invention selects CT images from 6 centers, wherein the detailed information of each center is shown in Table 1:

[0079] Table 1

[0080]

[0081] In the ablation experiment, the experimental results of this method were compared with those of the pancreatic-tumor parallel branch (PTPB), the pre-training-fine-tuning adaptive training method (PFAT), and the branch-guided fusion module (FGFM). The ablation results of the pancreatic-tumor parallel branch are shown in Table 2, the ablation results of the pre-training-fine-tuning adaptive training method and the branch-guided fusion module are shown in Table 3, and the Kaplan-Meier survival curve results are shown in Figure 7 As shown,

[0082] Table 2

[0083]

[0084] Table 3

[0085]

[0086] The indicator corresponds to C-index, which indicates the probability that the predicted result is consistent with the actual result. Both the table and the visualization results demonstrate the effectiveness of the three modules of this method.

[0087] In the comparative experiment, the experimental results of this method are compared with 2-Net, DeepSurv, DLSP, GuanRank, AdvMIL, pix2surv, and CancerRiskNet methods are compared. The comparison results are shown in Tables 4, 5, 6, 7, and 8. The Kaplan-Meier survival curve results are shown in Figure 5 As shown, the visualization results of Grad-CAM are as follows Figure 6 As shown,

[0088] Table 4

[0089]

[0090] Table 5

[0091]

[0092] Table 6

[0093]

[0094] Table 7

[0095]

[0096] Table 8

[0097]

[0098] To ensure fairness in the experimental results, the above experiments all performed fine segmentation based on coarse segmentation. Table 4 shows the prognostic comparison results for the internal test set, while Tables 5, 6, 7, and 8 show the prognostic comparison results for the external test set. The present invention can better capture the anatomical characteristics and spatial positional relationships of the pancreas and pancreatic tumors, improving the prognostic performance of this method.

[0099] It should be noted that the above embodiments are not intended to limit the scope of protection of the present invention, and equivalent changes or substitutions made on the basis of the above technical solutions fall within the scope of protection of the claims of the present invention.

Claims

1. A pancreatic ductal carcinoma prognosis survival analysis method based on a dual-branch adaptive network, characterized by: The following steps are involved: S1： Preprocessing: A unified preprocessing of multi-center clinical abdominal CT datasets was performed, including window width and window position adjustment, histogram equalization, Sobel operator, and data enhancement. S2: performing coarse segmentation on the pre-processed CT image slices in step S1 to obtain preliminary location information of the pancreas and pancreatic tumor; S3: Based on the coarse segmentation results in step S2, the CT image is cropped and a pancreas-pancreatic tumor parallel branch structure is constructed using a segmentation network, which includes: pancreas region images, pancreas region features, pancreatic tumor region images, pancreatic tumor region features, and fully utilizes information about the pancreas and pancreatic tumor target regions and their neighborhoods; S4: performing feature extraction based on the dual-branch input in step S3, including: a pancreas encoder and a pancreatic tumor encoder; S5: A two-stage pre-training-fine-tuning adaptive training method is used for the dual-branch features in step S4. The first stage is the pre-training stage, and the second stage is the fine-tuning adaptive stage. The two stages are trained serially to deeply capture the anatomical features of the pancreas and pancreatic tumors, and enhance the prognostic model's ability to learn complex anatomical characteristics. S6: Fusing the dual-branch features in step S5, using a branch-guided fusion module to comprehensively capture the complex and diverse spatial relative relationships between the pancreas and pancreatic tumors from the multi-scale feature extraction module and the multi-view information fusion module; S7: Inputting the dual-branch fusion features in step S6 into the prognostic network to obtain the final pancreatic ductal carcinoma prognostic survival analysis results; Among them, the dual-branch features are fused, and the branch-guided fusion module includes two parts: a multi-scale feature extraction module and a multi-view information fusion module. Multi-scale feature extraction module: the features F of the two branches are extracted separately pancreas and F tumor For multi-scale extraction, dilated convolution with different expansion rates is used to construct four parallel feature groups to obtain feature information under different receptive fields. First, the four parallel feature groups are constructed. The specific implementation formula is as follows: where f pancreas-i and f tumor-i represents the convolution features of pancreas and tumor under dilation rate i, f' pancreas-i and f' tumor-i The expanded convolution features of the pancreas and tumor are weighted by the Sigmoid function, where i is the expansion rate with values ​​of 1, 2, 3, and 5, and f′ is obtained. pancreas-1 ,f′ pancreas-2 ,f′ pancreas-3 ,f′ pancreas-5 Four parallel feature groups representing the pancreas, f′ tumor-1 ,f′ tumor-2 ,f′ tumor-3 ,f′ tumor-5 Representing four parallel feature groups of tumors, S represents the Sigmoid function. Secondly, the multi-scale features are aggregated. The specific implementation formula is as follows: F pancreas-fd =C(f′ pancreas-1 ,f′ pancreas-2 ,f′ pancreas-3 ,f′ pancreas-5 ) F tumor-fd =C(f′ tumor-1 ,f′ tumor-2 ,f′ tumor-3 ,f′ tumor-5 ) Where C represents the serial operation, F pancreas-fd and F tumor-fd is the output result of the multi-scale feature extraction module; Multi-view information fusion module: Through the above multi-scale feature extraction module, the multi-scale features F of pancreas and pancreatic tumor are obtained respectively. pancreas-fd and F tumor-fd Based on this, multi-view information fusion is performed. First, the multi-scale features from the same layer of pancreas and pancreatic tumors are aggregated together, and the dual-branch features are fused based on this. Then, through maximum pooling and average pooling, the key information in the feature channel can be captured from multiple perspectives. Specifically, the multi-layer perception mechanism is used with average pooling and maximum pooling, and the Sigmoid function is applied to calculate the channel attention coefficient K. Then, the attention coefficient is multiplied by the feature to obtain a reweighted feature vector. Finally, these reweighted features are residually connected with the multi-scale pancreatic and tumor features to obtain the final fusion feature F fusion , the specific implementation formula is as follows: Fc=C(F pancreas-fd ,F tumor-fd ) K=S{MLP(MP(F c ))+MLP(AP(F c ))} Among them, MLP stands for multi-layer perception mechanism, MP stands for maximum pooling layer, AP stands for average pooling layer, S stands for Sigmoid function, C stands for concatenation operation, and F c Features representing the pancreas and pancreatic tumors after multi-scale features are concatenated together.

2. The pancreatic ductal carcinoma prognosis survival analysis method based on a dual-branch adaptive network according to claim 1, characterized in that: A unified preprocessing of multi-center clinical abdominal CT datasets was performed, including: (1) Window width and window position adjustment: Based on the clinical prior information of the pancreas and pancreatic tumors, the window width of the abdominal CT image is set to 200HU and the window position is set to 100HU, and the image is normalized to constrain the range of voxel values ​​to [0, 1]; (2) Histogram equalization: By changing the histogram of the image, the grayscale of each pixel in the image is changed to improve the contrast of pancreatic tumors in the CT image; (3) Sobel operator: Edge detection is performed on the pancreas and pancreatic tumors to make the contrast between the edge and the fat tissue more obvious; (4) Data enhancement: The CT image is rotated at 90°, 180° and 270° respectively, and the image is scaled to achieve data enhancement.

3. The pancreatic ductal carcinoma prognosis survival analysis method based on a dual-branch adaptive network according to claim 2, characterized in that: The coarse segmentation process in step S2 includes the following steps: the location information of pancreatic tumors in clinical priors, the CT slices are cropped, the corresponding coordinates of the cropped area are 100-400 in length and 120-480 in width, both in pixels, and the image is segmented using a segmentation network. The encoder and decoder of the network contain four convolutional layers, which are composed of a 3X3 2D convolutional layer, a BatchNormalization normalization layer, and an activation layer with a Relu activation function. The encoder adds a maximum pooling layer at the end of each layer, and the decoder adds an upsampling layer at the end of each layer to obtain the coarse segmentation result I slice .

4. The pancreatic ductal carcinoma prognosis survival analysis method based on a dual-branch adaptive network according to claim 3, characterized in that: Based on the segmentation network, a parallel branch structure of pancreas-pancreatic tumor is constructed. First, the CT image is cropped with the tumor center area according to the coarse segmentation result in step S2. The cropping range is: length: -72~72, width: -56~56, height: -88~88, and the unit is voxel. The cropped image I of the pancreas and its surrounding tissues is obtained. local , and then use the segmentation network to segment the image. The network encoder and decoder contain four convolution layers, which are composed of 5X5 3D convolution layers, Batch Normalization normalization layers, and activation layers with Relu activation function. The encoder adds a maximum pooling layer at the end of each layer, and the decoder adds an upsampling layer at the end of each layer to obtain the segmentation results of the pancreas and pancreatic tumors. pancreas and L tumor , construct the target area of ​​pancreas and pancreatic tumor, and use the segmentation result to obtain the CT area image I containing only pancreas and pancreatic tumor pancreas-L and I tumor-L In order to obtain the neighborhood information containing the pancreas and pancreatic tumors, the features of the third layer decoder in the segmentation network are used as another input I pancreas-F and I tumor-F , in the pancreatic branches, a pancreatic CT regional image I was constructed pancreas-L and pancreatic domain features I pancreas-F As input, in the pancreatic tumor branch, a pancreatic tumor CT region image I is constructed tumor-L and pancreatic domain features I tumor-F as input.

5. The pancreatic ductal carcinoma prognosis survival analysis method based on a dual-branch adaptive network according to claim 4, characterized in that: To extract features from the pancreas-pancreatic tumor parallel branch, first combine the dual inputs of pancreas and pancreatic tumor. The specific implementation formula is as follows: I pancreas =C(I pancreas-L ,I pancreas-F ) I tumor =C(I tumor-L ,I tumor-F ) Among them I pancreas and I tumor The final input of the CT image and feature combination of pancreas and pancreatic tumor, C represents the serial operation, then, I pancreas and I tumor For feature extraction, the feature extractor consists of an encoding-decoding structure, which includes four convolutional layers, a 5X5 3D convolutional layer, a Batch Normalization normalization layer, and an activation layer with Relu activation function. The encoding layer is added to the end of each layer, and the decoding layer is added to the end of each layer. The upsampling layer is finally added to obtain the dual-branch feature F pancreas and F tumor .

6. The pancreatic ductal carcinoma prognosis survival analysis method based on a dual-branch adaptive network according to claim 5, characterized in that: The pre-training-fine-tuning adaptive training method is used for the dual-branch feature extraction stage. The first stage is the pre-training stage, and the second stage is the fine-tuning adaptive stage. In the first stage, the pancreas and pancreatic tumor branches are pre-trained respectively, and the model parameters W obtained by pre-training are used. pancreas and W tumor To initialize the model parameters of the dual-branch feature extraction stage; in the second stage, the model parameters of the feature extraction will be updated as the model training progresses based on the pre-trained weights. The specific weight change formula is as follows: Wn pancreas =W pancreas +ΔW IN ntumor =In tumor +ΔW Among them, ΔW is the weight matrix for fine-tuning the pre-training parameters. By adding it to the pre-training weight matrix, the final weight matrix W is obtained. npancreas and W ntumor , the fine-tuning weight matrix is ​​adaptively decomposed into two low-rank matrices A and B. The specific implementation formula is as follows: <h2 style=";text-align:left;direction:ltr">W<h2 style=";text-align:left;direction:ltr"> npancreas <h2 style=";text-align:left;direction:ltr"> =W<h2 style=";text-align:left;direction:ltr"> pancreas <h2 style=";text-align:left;direction:ltr"> +AB <h2 style=";text-align:left;direction:ltr">W<h2 style=";text-align:left;direction:ltr"> ntumor <h2 style=";text-align:left;direction:ltr"> =W<h2 style=";text-align:left;direction:ltr"> tumor <h2 style=";text-align:left;direction:ltr"> +AB where ΔW∈R n×m , A∈R n×d , B∈R d×m , n and m are the number of rows and columns of the parameter matrix, d is the rank of the parameter matrix, R represents the set of real numbers, W npancreas and W ntumor They represent the pancreatic weight matrix and pancreatic tumor weight matrix after pre-training update, respectively. In the process of decomposing the fine-tuned weight matrix ΔW into two matrices A and B, the matrix AB is initialized with a random Gaussian distribution, providing a more balanced starting point for network training.

7. The pancreatic ductal carcinoma prognosis survival analysis method based on a dual-branch adaptive network according to claim 1, characterized in that: The dual-branch fused features were input into the prognostic network for prognostic survival analysis. The prognostic network contained three fully connected layers, and DropOut was added after the first two layers to obtain the final prognostic results. The final prognostic results were analyzed to divide pancreatic ductal carcinoma patients into high-risk and low-risk groups based on CT images.

Images