A system and method for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery

Abstract

The present invention provides a system and method for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery, belonging to the technical field of machine learning-based prediction of postoperative cognitive dysfunction in patients. The system includes a data receiving and acquisition module, an abdominal CT feature extraction module, a patient clinical and information data encoding module, and a gastrointestinal tumor patient postoperative cognitive dysfunction prediction module. The data receiving and acquisition module extracts three-dimensional features of abdominal imaging data to obtain an abdominal CT image fusion feature map. The patient clinical and information data encoding module performs feature encoding on the patient clinical data and information data to obtain a clinical feature vector. The gastrointestinal tumor patient postoperative cognitive dysfunction prediction module uses the abdominal CT image fusion feature map and clinical feature vector as input and outputs a quantitative evaluation result of the patient's postoperative cognitive dysfunction. The present invention achieves effective classification of cognitive dysfunction, providing strong auxiliary support for clinical intervention and management of postoperative patients.

Description

Technical Field

[0001] The present invention belongs to the technical field of prediction of postoperative cognitive dysfunction in patients based on machine learning, and in particular relates to a system and method for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery. Background Art

[0002] Postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery has attracted considerable attention due to its high incidence and clinical harm. Its occurrence is closely related to gut-brain axis disruption caused by surgical stress, neurotoxicity of chemotherapeutic drugs (such as oxaliplatin), and metabolic abnormalities. Cognitive dysfunction is a common neurological complication in patients undergoing gastrointestinal tumor surgery, especially in the elderly, and is primarily manifested by memory loss, decreased executive function, attention deficits, and impaired social skills. Studies have shown that patients undergoing non-cardiac surgery have a high incidence of cognitive dysfunction within one week after surgery, with some patients still experiencing cognitive impairment three months after surgery. This cognitive decline not only prolongs hospital stays and increases medical expenses, but may also accelerate the progression of postoperative dementia and even increase long-term mortality, posing significant challenges to patient quality of life, the burden of family care, and the social medical system.

[0003] Currently, strategies for preventing and treating postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery are primarily categorized into pharmacological and non-pharmacological interventions. Non-pharmacological measures include optimizing surgical techniques (such as minimally invasive techniques), maintaining intraoperative organ perfusion, meticulously managing the depth of anesthesia, enhancing postoperative analgesia, and conducting early rehabilitation training. Pharmacological interventions involve the use of brain-protective drugs such as dexmedetomidine, anti-inflammatory drugs, and neurometabolic regulators. Although these measures can reduce the risk of cognitive dysfunction to a certain extent, their effectiveness is limited by individual variability and the complexity of pathological mechanisms. Therefore, establishing an accurate risk prediction system to enable early identification of high-risk patients has become a core goal in optimizing the management of cognitive dysfunction.

[0004] Methods for predicting postoperative cognitive impairment in patients undergoing gastrointestinal tumor surgery fall into four main categories: clinical assessment tools, biomarker testing, imaging techniques, and artificial intelligence models. Clinical assessment tools include neuropsychological test batteries (such as the HVLT, TMT, and MoCA) and risk prediction models (such as the ISPOCD model). While these tools are authoritative, they are time-consuming and significantly influenced by the physician's clinical experience. Biomarker testing, including blood (IL-6, NfL, etc.), cerebrospinal fluid (Aβ42 / tau), and gene (APOE ε4) analysis, can objectively reflect the risk of neurological injury, but some tests are invasive and costly. Imaging techniques such as MRI and functional MRI reveal structural and functional abnormalities in the brain, and the emerging near-infrared spectroscopy (NIRS) can also monitor brain oxygen status in real time during surgery. However, these methods are limited by the high subjectivity of clinical assessments and the lack of specificity and timeliness of biomarker analysis.

[0005] Artificial intelligence models integrate multimodal data (clinical indicators + MRI + EEG) and utilize algorithms such as random forests and deep learning to improve predictive effectiveness. They have been widely used to predict the prognosis of cognitive dysfunction and have achieved results consistent with clinical expert evaluations. However, existing technologies have the following problems:

[0006] Excessive reliance on brain imaging data: Existing AI models primarily integrate neuroimaging data such as brain MRI and EEG, ignoring the correlation between abdominal organ structural changes and cognitive impairment. This results in an inability to utilize the non-invasive data source of routine preoperative abdominal CT scans of gastrointestinal tumor patients for prediction.

[0007] Inefficient multimodal fusion mechanisms: Existing methods for fusing imaging and clinical data often use simple splicing or shallow feature interactions, making it difficult to capture high-order cross-modal correlations (such as the synergistic effect between intraoperative drug dosage and tumor morphology), limiting the model's expressive power.

[0008] Insufficient generalization ability: The model relies on brain-specific detection (such as EEG), which is limited by the popularity of equipment and patient cooperation, making it difficult to promote in grassroots hospitals; it does not introduce prior knowledge of organ anatomy and is sensitive to image noise. Summary of the Invention

[0009] In response to the above problems, the first aspect of the present invention provides a system for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery, comprising a data receiving and acquisition module, an abdominal CT feature extraction module, a patient clinical and information data encoding module, and a gastrointestinal tumor patient postoperative cognitive dysfunction prediction module;

[0010] The data receiving and acquiring module is used to receive and acquire the patient's abdominal imaging data, the patient's clinical data and the patient's information data;

[0011] The abdominal CT feature extraction module includes a tumor region feature extraction network, an organ-level local feature extraction network, and a multi-level feature fusion network; the tumor region feature extraction network performs three-dimensional convolution feature extraction on the patient's preoperative abdominal images, captures deep information about the tumor's spatial distribution, density, and morphology, and obtains a tumor morphology feature map; the organ-level local feature extraction network uses the patient's preoperative and postoperative abdominal images as input, segments the stomach, colon, and rectum, and obtains a key organ spatial position feature map; the multi-level feature fusion network is used to fuse the tumor morphology feature map with the key organ spatial position feature map to obtain an abdominal CT image fusion feature map;

[0012] The patient clinical and information data encoding module performs feature encoding on the patient clinical data and information data to obtain a clinical feature vector;

[0013] The gastrointestinal tumor patient postoperative cognitive dysfunction prediction module takes the abdominal CT image fusion feature map and the clinical feature vector as input, and outputs a quantitative evaluation result of the patient's postoperative cognitive dysfunction.

[0014] Preferably, the abdominal imaging data includes the patient's preoperative abdominal imaging data. and postoperative abdominal imaging of the patient The patient clinical data include perioperative data of patients with gastric, colon and rectal tumors, specifically the results of preoperative neuropsychological assessment of patients , Postoperative neuropsychological evaluation results of patients , anesthesia method , surgical method , intraoperative drug dosage , time parameters and brain function monitoring indicators ; The patient information data includes the patient's past medical history information , patient attribute indicators , gender, height and weight, education level, and description of the patient's current gastrointestinal tumor condition .

[0015] Preferably, the tumor region feature extraction network is an improved architecture based on 3D ResNet-50, including a pre-trained 3D Mask R-CNN layer, a dynamic density histogram equalization layer, a residual attention backbone network and a fully connected layer; Preoperative abdominal imaging of patients Input pre-trained 3D Mask R-CNN layer positioning The tumor area in The image is cropped into a tumor region cube; the tumor region cube is input into the dynamic density histogram equalization layer to enhance the contrast of the tumor boundary in the tumor region cube to obtain an enhanced tumor region cube; the enhanced tumor region cube is input into the residual attention backbone network to obtain the deep features of the tumor morphology. The residual attention backbone network contains four residual blocks, a 3D attention layer and a global adaptive average pooling layer connected in sequence. The number of channels of the four residual blocks connected in sequence are 64, 128, 256 and 512 respectively. Each residual block contains three 3×3×3 convolutional layers for extracting tumor region features layer by layer. Each residual block is followed by a 3D attention layer. The 3D attention layer is used to weightedly fuse the input features and output features of each residual block. The global adaptive average pooling layer is used to compress the feature dimension of the last residual block output by the 3D attention layer to obtain a reduced dimensionality feature; the reduced dimensionality feature is input into the fully connected layer for random inactivation operation to obtain the final tumor morphology feature map.

[0016] Preferably, the organ-level local feature extraction network is an improved architecture based on 3D nnU-Net, including an image registration layer, a pre-trained 3D-Unet model and a transformer encoder; the specific data processing process is:

[0017] The first Postoperative abdominal imaging of patients Input image registration layer and patient preoperative abdominal image Perform image registration, and the image registration layer uses the scale-invariant feature conversion algorithm to extract The SIFT feature points and The key matching points of SIFT feature points are used to calculate the affine transformation matrix, and the affine transformation matrix is ​​used to transform The pixels are mapped to Pixel coordinate system, to achieve and Cross-modal spatial registration to obtain the Postoperative abdominal images of patients ;

[0018] Use the pre-trained 3D-Unet model to Postoperative abdominal images of patients Perform semantic segmentation to obtain Gastric, colonic, and rectal characteristics of each patient;

[0019] The first The stomach feature map, colon feature map and rectal feature map of each patient are input into the transformer encoder to obtain the first The transformer encoder is used to model the spatial contextual relationship within the organs to obtain more localized spatial feature information that is more relevant to the risk of cognitive impairment.

[0020] Preferably, the multi-level feature fusion network The tumor morphology feature map, gastric organ spatial position feature map, colon organ spatial position feature map and rectal organ spatial feature map of each patient are fused; the multi-level feature fusion network includes a feature alignment layer, a channel attention weighted layer and a first feature splicing layer;

[0021] The feature alignment layer reduces the dimension of the tumor morphology feature map; the feature alignment layer includes a fully connected layer and a 3D transposed convolution;

[0022] The channel attention weighted layer takes the spatially aligned tumor morphology feature map and organ feature map as input; the channel attention weighted layer includes a global average pooling layer, two fully connected layers and a sigmoid activation function layer;

[0023] The first feature splicing layer takes the channel-weighted tumor morphology feature map, the gastric organ spatial position feature map, the colon organ spatial position feature map and the rectal organ spatial feature map as input, and performs horizontal feature splicing to obtain an abdominal CT image fusion feature map with a channel number of 1024.

[0024] Preferably, the patient clinical and information data encoding module includes a CLIP model, a numerical encoding model and a second feature splicing layer;

[0025] Input the text data from the patient's clinical and patient information data into the CLIP model, and project different types of structured text into the same latent semantic space to obtain text semantic feature vectors;

[0026] The numerical data in the patient's clinical and patient information data are first input into the numerical encoding model to convert the numerical data into a numerical feature vector; then the numerical feature vector is input into the encoder of the CLIP model to convert the numerical feature vector into a numerical semantic feature vector whose feature dimension is consistent with the dimension of the text semantic feature vector;

[0027] The second feature splicing layer performs horizontal feature splicing on the numerical semantic feature vector and the text semantic feature vector to obtain a clinical feature vector.

[0028] Preferably, the gastrointestinal tumor patient postoperative cognitive dysfunction prediction module includes a global average pooling layer, a dual-headed self-attention mechanism network and a cognitive dysfunction prediction network;

[0029] The global average pooling layer performs dimensionality reduction on the abdominal CT image fusion feature map, and calculates the average value of all spatial positions of each channel in the abdominal CT image fusion feature map to obtain the abdominal CT image fusion feature vector;

[0030] The dual-headed self-attention mechanism network takes the abdominal CT image fusion feature vector and the clinical feature vector as input; the dual-headed self-attention mechanism network includes a linear mapping layer, a residual feature splicing layer and a dual-headed self-attention mechanism layer; the abdominal CT image fusion feature vector is input into the linear mapping layer and mapped into a query vector and key vector , the clinical feature vector is input into the linear mapping layer and mapped into a value vector ; Secondly, the clinical feature vector is input into the linear mapping layer and mapped into the query vector and key vector , the abdominal CT image fusion feature vector is input into the linear mapping layer and mapped into a value vector ; again [ ; ; ]and[ ; ; ] Simultaneously input the dual-head self-attention mechanism layer to obtain the weight of the abdominal CT image fusion feature vector relative to the clinical feature vector and the weight of the clinical feature vector relative to the abdominal CT image fusion feature vector ; Again, the abdominal CT image is fused with the feature vector and weight Perform point multiplication to obtain weighted abdominal CT image fusion feature vector, and combine the clinical feature vector with the weight Perform point multiplication to obtain a weighted clinical feature vector; finally, add the weighted abdominal CT image fusion feature vector and the weighted clinical feature vector, and then input them into the residual feature splicing layer together with the abdominal CT image fusion feature vector and the clinical feature vector to perform horizontal feature splicing to obtain the fusion feature;

[0031] The cognitive dysfunction prediction network includes three fully connected layers, three random dropout layers, and a softmax activation function layer; each fully connected layer is connected to a random dropout layer, the fully connected layer is used to reduce the dimension of the fusion feature, and the random dropout layer is used to freeze the parameters of some neurons in the fully connected layer. The fusion feature processed by the last fully connected layer is input into the RELU activation function layer to output the patient's postoperative cognitive function score.

[0032] Preferably, the 3D-Unet model in the abdominal CT feature extraction module is trained using an organ prior constraint training strategy. The specific process is as follows:

[0033] Relying on the prior knowledge of stomach, colon and rectum images provided by the public dataset, the average shape feature map of the stomach organ, the average shape feature map of the colon organ and the average shape feature map of the rectum organ are obtained; first, for an image data sample in the public dataset, the three-dimensional mask of the stomach, colon and rectum in the image is extracted, and Procrustes analysis is used to align all organ instances to the same coordinate system to eliminate image rotation and translation differences; secondly, vertex-based non-rigid registration is used for each organ, and the aligned shapes of the stomach, colon and rectum organs are vectorized to obtain the shape feature map of the stomach organ, the shape feature map of the colon organ and the shape feature map of the rectum organ; again, the main The component analysis method calculates the shape principal components of the stomach organ shape feature map, colon organ shape feature map, and rectal organ shape feature map respectively, and retains the top k principal components that explain 95% of the shape variance to obtain the reduced-dimensionality stomach organ shape feature map, colon organ shape feature map, and rectal organ shape feature map. Finally, the above steps are performed on all stomach organs, colon organs, and rectal organs in the public dataset, and all the reduced-dimensionality stomach organ shape feature maps, reduced-dimensionality colon organ shape feature maps, and reduced-dimensionality rectal organ shape feature maps are input into the average pooling layer to obtain the average shape feature map of the stomach organ, the average shape feature map of the colon organ, and the average shape feature map of the rectum organ.

[0034] During the training process, the 3D-Unet model in the abdominal CT feature extraction module performs semantic segmentation on the patient's postoperative abdominal registration images to obtain the stomach feature map, colon feature map, and rectum feature map, and calculates the Frobenius norm between the stomach feature map and the average shape feature map of the stomach organ. , the Frobenius norm between the colon feature map and the average shape feature map of the colon organ and the Frobenius norm between the rectal feature map and the average shape feature map of the rectal organ , and adopt the loss function Fine-tune the parameters of the 3D-Unet model:

[0035] ;

[0036] in is the training set size.

[0037] Preferably, a binary cross entropy loss function is used to perform overall training on the abdominal CT feature extraction module, the patient clinical and information data encoding module, and the gastrointestinal tumor patient postoperative cognitive dysfunction prediction module. During the training process, the 3D-Unet model and the CLIP model are frozen, and only other neural network structures other than the 3D-Unet model and the CLIP model are trained. The loss function is the average root mean square error between the patient cognitive dysfunction prediction result output by the gastrointestinal tumor patient postoperative cognitive dysfunction prediction module and the actual result.

[0038] A second aspect of the present invention provides a method for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery, wherein the system for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery as described in the first aspect is deployed on a detection terminal; and the method includes the following steps:

[0039] Real-time acquisition of standard input data of patient abdominal images, patient clinical and patient information;

[0040] The above standard input data were input into the deployed postoperative cognitive dysfunction prediction system for gastrointestinal tumor surgery patients;

[0041] The prediction score is output, and the doctor combines the prediction results of postoperative cognitive dysfunction in gastrointestinal tumor patients with clinical experience to diagnose whether the patient will have postoperative cognitive dysfunction.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] 1. Deeply integrate preoperative abdominal CT imaging features (such as pancreatic fat infiltration and intestinal wall thickness variation) with perioperative clinical data (anesthesia method, drug dosage, brain oxygen monitoring, etc.), breaking through the limitations of a single data source. Dynamically weight different modal features through a cross-modal attention mechanism to improve feature expression capabilities;

[0044] 2. An improved 3D ResNet-50 network captures the spatial distribution and morphological characteristics of tumors, combined with nnU-Net to segment key organs, and improves anatomical structure recognition accuracy through organ prior constraint training;

[0045] 3. The feature interaction network realizes deep coupling of imaging and clinical data to avoid fusion noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of the overall technical route of the present invention.

[0047] Figure 2 This is the structural diagram of the abdominal CT feature extraction module of the present invention.

[0048] Figure 3 This is a structural diagram of the patient clinical and information data encoding module of the present invention.

[0049] Figure 4 This is a structural diagram of the module for predicting cognitive dysfunction after surgery for gastrointestinal tumor patients according to the present invention.

[0050] Figure 5 This is a diagram showing the tumor region segmentation effect in an embodiment of the present invention.

[0051] Figure 6 This is a confusion matrix result diagram in an embodiment of the present invention.

[0052] Figure 7 Graph showing the ROC curve results in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] This paper uses deep learning technology to fully utilize the patient's perioperative abdominal organ image information, physiological data and cognitive status, and proposes a method for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery. The overall process is as follows: Figure 1 As shown:

[0054] First, perioperative abdominal imaging data, clinical data, and information data of patients with gastric cancer, colon cancer, and rectal cancer were collected. The patients' cognitive behavior was evaluated using the preoperative neuropsychological assessment results and postoperative neuropsychological assessment results in the clinical data to obtain the patients' postoperative cognitive function scores as the labels of the dataset. The abdominal imaging data, the remaining clinical data, and information data were used as data samples to construct the training set and test set. Secondly, an abdominal CT feature extraction module was constructed to extract the three-dimensional features of the abdominal imaging data and obtain the abdominal CT image fusion feature map, which covers the global image features, tumor morphological features, and spatial location features of key organs. Thirdly, a patient clinical and information data encoding module was constructed to feature encode the patient clinical data and information data to obtain a clinical feature vector. Thirdly, a gastrointestinal tumor patient postoperative cognitive dysfunction prediction module was constructed, which took the abdominal CT image fusion feature map and clinical feature vector as input to achieve quantitative evaluation results of the patient's postoperative cognitive dysfunction. Finally, a multimodal model training strategy was designed to train the abdominal CT feature extraction module, the patient clinical and information data encoding module, and the gastrointestinal tumor patient postoperative cognitive dysfunction prediction module.

[0055] The invention will be further described below with reference to specific embodiments.

[0056] 1. Dataset Construction

[0057] First, preoperative and postoperative clinical information, abdominal CT images, intraoperative data, and neuropsychological scale results from patients with gastric, colon, and rectal cancer at different stages (early, mid-, and late stages) were collected and preprocessed to protect privacy. Next, experts labeled the patients' cognitive behaviors based on their neuropsychological scale results, including normal cognition and cognitive impairment. Using the patients' preoperative and postoperative clinical information, abdominal CT images, and intraoperative data as samples, and the patients' postoperative cognitive function scores as labels, a fully labeled dataset was obtained.

[0058] Construct a data system of cognitive dysfunction after gastrointestinal tumor surgery that covers multi-source heterogeneous information, including three types of patient abdominal imaging data, patient clinical data, and patient information data. The patient abdominal imaging data is the patient's preoperative abdominal imaging data. and postoperative abdominal imaging of the patient ; Patient clinical data include perioperative data of patients with gastric, colon and rectal tumors, specifically the results of preoperative neuropsychological assessments (MMSE score), postoperative neuropsychological evaluation results of patients (MMSE score), anesthesia method (general anesthesia or combined epidural block), surgical method (open or laparoscopic surgery), intraoperative drug dosage (dexmedetomidine dosage, remifentanil dosage, propofol dosage), time parameters (operation time and anesthesia time) and brain function monitoring indicators (The duration of intraoperative cerebral oxygen saturation (rSO2) below 35 and the duration of its relative change ΔrSO2 greater than 13%); patient information data includes the patient's medical history information (whether suffering from hypertension and diabetes), patient attribute indicators (age (0 represents male, 1 represents female), gender, height and weight, education level (0 represents less than bachelor's degree, 1 represents bachelor's degree, 2 represents postgraduate degree)) and description of the patient's current gastrointestinal tumor condition Through the systematic collection, organization, and privacy-massaging of the aforementioned multi-source heterogeneous data, a complete and high-quality dataset can be constructed, providing a solid data foundation for feature extraction of subsequent models and prediction of the risk of postoperative cognitive impairment.

[0059] Experts based on the patient's preoperative neuropsychological assessment results and postoperative neuropsychological evaluation results of patients The patients' cognitive behavior was assessed to obtain the postoperative cognitive function score, which ranged from 1 to 10. The higher the score, the higher the degree of cognitive dysfunction.

[0060] Preoperative abdominal imaging of the patient , Postoperative abdominal imaging of the patient , anesthesia method , surgical method , intraoperative drug dosage , time parameters (operation time and anesthesia time), brain function monitoring indicators , patient's medical history information , patient attribute indicators and description of the patient's current gastrointestinal tumor condition As data samples, the patient's cognitive behavioral assessment category is used as a label to construct the training set and test set; in order to improve the generalization ability of the model, the patient data collected by the present invention covers the perioperative data of three types of patients: gastric cancer patients, colon cancer patients, and rectal cancer patients at different stages (early, middle, and late).

[0061] 2. Constructing the Abdominal CT Feature Extraction Module

[0062] The abdominal CT feature extraction module includes a tumor region feature extraction network, an organ-level local feature extraction network, and a multi-level feature fusion network. Figure 2 As shown in the figure; the tumor region feature extraction network is used to extract the patient's preoperative abdominal images Perform three-dimensional convolution feature extraction to capture deep information about the tumor's spatial distribution, density, and morphology, and obtain a tumor morphological feature map; and postoperative abdominal imaging of the patient The input organ-level local feature extraction network is used to segment the stomach, colon and rectum to obtain the key organ spatial position feature map; the multi-level feature fusion network is used to fuse the tumor morphology feature map and the key organ spatial position feature map to obtain the abdominal CT image fusion feature map; the abdominal CT feature extraction module is used to extract the first The specific steps for determining the three-dimensional features of a patient's abdominal CT are as follows:

[0063] 1. The tumor region feature extraction network is an improved architecture based on 3D ResNet-50, including pre-trained 3DMask R-CNN layer, dynamic density histogram equalization layer, residual attention backbone network and fully connected layer; Preoperative abdominal imaging of patients Input pre-trained 3D Mask R-CNN layer positioning The tumor area in The tumor region cube is cropped to 64×64×32; the tumor region cube is input into the dynamic density histogram equalization layer to enhance the contrast of the tumor boundary in the tumor region cube to obtain the enhanced tumor region cube; the enhanced tumor region cube is input into the residual attention backbone network to obtain the deep features of the tumor morphology. The residual attention backbone network contains four residual blocks, a 3D attention layer and a global adaptive average pooling layer connected in sequence. The number of channels of the four residual blocks connected in sequence are 64, 128, 256 and 512 respectively. Each residual block contains three 3×3×3 convolutional layers for extracting tumor region features layer by layer. Each residual block is connected to a 3D attention layer, which is used to average the channels of each residual block. A weighted fusion of input and output features prevents gradient explosion during model training. A global adaptive average pooling layer compresses the feature dimensions of the last residual block output by the 3D attention layer to obtain reduced-dimensional features. The reduced-dimensional features are input into a fully connected layer for random dropout to obtain the final tumor morphology feature map (H×W×D×512), where H, W, and D represent the length, width, and depth of the tumor morphology feature map, respectively. This feature map not only captures three types of static structural information: tumor volume, edge morphology, and texture distribution, but also effectively preserves the spatial hierarchy of the organ and the relationship between tissues. It can describe the overall abdominal structure and regional tumor distribution patterns, providing a global information foundation for subsequent cognitive impairment risk prediction.

[0064] 2. The organ-level local feature extraction network is an improved architecture based on 3D nnU-Net, including an image registration layer, a pre-trained 3D-Unet model, and a transformer encoder, as follows:

[0065] 1) Postoperative abdominal imaging of patients Input image registration layer and patient preoperative abdominal image Perform image registration, and the image registration layer uses the scale-invariant feature conversion algorithm to extract The SIFT feature points and The key matching points of SIFT feature points are used to calculate the affine transformation matrix, and the affine transformation matrix is ​​used to transform The pixels are mapped to Pixel coordinate system, to achieve and Cross-modal spatial registration to obtain the Postoperative abdominal images of patients ;

[0066] 2) Use the 3D-Unet model pre-trained on the TotalSegmentator public dataset to Postoperative abdominal images of patients Perform semantic segmentation to obtain The stomach, colon, and rectum feature maps of each patient were generated. The 3D-Unet model pre-trained on the TotalSegmentator public dataset has strong transfer learning and adaptive modeling capabilities, enabling high-precision segmentation across different anatomical structures.

[0067] 3) Use transformer encoder to Organ-level local feature extraction is performed on the stomach feature map, colon feature map, and rectum feature map of each patient. The Transformer encoder is used to model the spatial contextual relationship within the organ to obtain more localized spatial feature information that is more relevant to the risk of cognitive impairment. The stomach feature map, colon feature map and rectal feature map of each patient are input into the transformer encoder to obtain the first The spatial position feature map of the stomach organ (H1×W1×D1×256), the spatial position feature map of the colon organ (H1×W1×D1×256), and the spatial position feature map of the rectum organ (H1×W1×D1×256) of each patient; H1, W1, and D1 are the length, width, and depth of the Transformer encoder output feature map respectively;

[0068] 3. Construct a multi-level feature fusion network to The tumor morphology feature map, gastric organ spatial position feature map, colon organ spatial position feature map and rectal organ spatial feature map of each patient are fused; the multi-level feature fusion network includes a feature alignment layer, a channel attention weighted layer and a feature horizontal splicing layer. The specific steps are as follows:

[0069] 1) Since the dimension of the tumor morphology feature map is H×W×D×256, while the dimension of the organ feature map is H1×W1×D1×256, in order to avoid the introduction of noise due to dimensionality mismatch when fusing the tumor morphology feature map and the organ feature map, the dimensions of the tumor morphology feature map and the organ feature map need to be aligned. Therefore, a feature alignment layer is constructed to reduce the dimension of the tumor morphology feature map. The feature alignment layer includes a fully connected layer and a 3D transposed convolution. First, the tumor morphology feature map is input into the fully connected layer to reduce the dimension of the tumor morphology feature map with 256 channels (H×W×D×256). Secondly, the 3D transposed convolution (kernel size 3×3×3, stride 1) is used to convert the 256-channel tumor morphology feature map into the same spatial size as the organ feature map, obtaining the spatially aligned tumor morphology feature map (H1×W1×D1×256).

[0070] 2) Input the spatially aligned tumor morphology feature map and organ feature map into the channel attention weighted layer; the channel attention weighted layer includes a global average pooling layer, two fully connected layers and a sigmoid activation function layer; firstly, input the spatially aligned tumor morphology feature map, gastric organ spatial position feature map, colon organ spatial position feature map and rectal organ spatial feature map into the global average pooling layer respectively, and perform global average pooling on each channel in the feature map to obtain the tumor morphology feature vector, gastric organ spatial position feature vector, colon organ spatial position feature vector and rectal organ spatial feature vector; secondly, input the tumor morphology feature vector, gastric organ spatial position feature vector, colon organ spatial position feature vector and rectal organ spatial position feature vector into the global average pooling layer. The intermediate feature vector is input into two fully connected layers and then into the sigmoid activation function layer to obtain the channel weights of the tumor morphology feature map, the channel weights of the gastric organ spatial position feature map, the channel weights of the colon organ spatial position feature map, and the channel weights of the rectal organ spatial feature map; the sigmoid activation function layer limits the weights to [0,1] to avoid gradient explosion; the spatially aligned tumor morphology feature map, gastric organ spatial position feature map, colon organ spatial position feature map, and rectal organ spatial feature map are multiplied by their corresponding channel attention weights to obtain the channel-weighted tumor morphology feature map, channel-weighted gastric organ spatial position feature map, channel-weighted colon organ spatial position feature map, and channel-weighted rectal organ spatial feature map;

[0071] 3) The channel-weighted tumor morphological feature map, gastric organ spatial position feature map, colon organ spatial position feature map, and rectal organ spatial feature map are input into the feature splicing layer for horizontal feature splicing to obtain an abdominal CT image fusion feature map with 1024 channels. This feature map covers the global image features, tumor morphological features, and key organ spatial position features.

[0072] 3. Constructing a patient clinical and information data coding module

[0073] Construct a patient clinical and information data encoding module to perform feature encoding on patient clinical data and patient information data, such as Figure 3 As shown in the figure, the patient clinical and information data encoding module includes the CLIP model, the numerical encoding model and the feature splicing layer; the CLIP model is a multimodal pre-trained neural network that can effectively capture the potential correlation and semantic deviation in language information; the patient clinical and information data encoding module is used to encode the first The specific steps for feature encoding of individual patient clinical data and information data are as follows:

[0074] 1. Use CLIP model to Anesthesia method for each patient , surgical method , past medical history information and current description of gastrointestinal tumors After feature encoding, word segmentation and embedding, the data is input into the CLIP model. The CLIP model projects different types of structured text into the same latent semantic space to obtain text semantic feature vectors.

[0075] 2. Since the traditional CLIP model cannot directly process numerical data, the present invention designs a numerical encoding model to convert numerical data into numerical feature vectors; the numerical encoding model includes a normalization layer and an encoder of a pre-trained BERT model; the intraoperative drug dosage is converted into a numerical feature vector. , time parameters , brain function monitoring indicators and patient attribute indicators After the data is normalized in the input normalization layer, it is input into the encoder of the pre-trained BERT model to obtain a numerical feature vector;

[0076] 3. Input the numerical feature vector into the encoder of the CLIP model to convert the numerical feature vector into a numerical semantic feature vector with the same feature dimension as the text semantic feature vector, so that it can be fused with the text semantic feature vector in the same feature space;

[0077] 4. Concatenate the numerical semantic feature vector and the text semantic feature vector input features to perform horizontal feature concatenation to obtain the clinical feature vector.

[0078] 4. Constructing a prediction module for postoperative cognitive dysfunction in patients with gastrointestinal tumors

[0079] Construct a prediction module for postoperative cognitive dysfunction in patients with gastrointestinal tumors to achieve quantitative evaluation results of patients' postoperative cognitive dysfunction. The model structure is as follows: Figure 4 As shown in the figure, the prediction module for postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery includes an interactive feature fusion network and a cognitive dysfunction prediction network. The abdominal CT image fusion feature map and clinical feature vector of each patient are input into the prediction module of postoperative cognitive dysfunction in gastrointestinal tumor patients. The specific steps are as follows:

[0080] To more fully explore the potential connection between abdominal CT image fusion features and clinical features, this paper designs an interactive feature fusion network. Its core lies in the dual-headed self-attention structure, which can reflect the relationship between any two positions in the input feature sequence in a weighted form, thereby capturing the deep coupling effect between different features. The interactive feature fusion network includes a global average pooling layer and a dual-headed self-attention mechanism network.

[0081] Since the abdominal CT image fusion feature is a 3D feature map and the clinical feature vector is a vector, in order to prevent the introduction of additional noise during the fusion process, the present invention uses a global average pooling layer to reduce the dimension of the abdominal CT image fusion feature map, and calculates the average value of all spatial positions of each channel in the abdominal CT image fusion feature map to obtain the abdominal CT image fusion feature vector;

[0082] The abdominal CT image fusion feature vector and the clinical feature vector are input into the dual-headed self-attention mechanism network; the dual-headed self-attention mechanism network includes a linear mapping layer, a residual feature splicing layer and a dual-headed self-attention mechanism layer; first, the abdominal CT image fusion feature vector is input into the linear mapping layer and mapped into a query vector and key vector , the clinical feature vector is input into the linear mapping layer and mapped into a value vector ; Secondly, the clinical feature vector is input into the linear mapping layer and mapped into the query vector and key vector , the abdominal CT image fusion feature vector is input into the linear mapping layer and mapped into a value vector ; again [ ; ; ]and[ ; ; ] Simultaneously input the dual-head self-attention mechanism layer to obtain the weight of the abdominal CT image fusion feature vector relative to the clinical feature vector and the weight of the clinical feature vector relative to the abdominal CT image fusion feature vector ; Again, the abdominal CT image is fused with the feature vector and weight Perform point multiplication to obtain weighted abdominal CT image fusion feature vector, and combine the clinical feature vector with the weight Perform point multiplication to obtain a weighted clinical feature vector. Finally, add the weighted abdominal CT image fusion feature vector and the weighted clinical feature vector, and then input them into the residual feature splicing layer together with the abdominal CT image fusion feature vector and the clinical feature vector to obtain fusion features for horizontal feature splicing, ensuring that the model strikes a balance between deep interaction and information fidelity.

[0083] The fusion features were input into the cognitive dysfunction prediction network to obtain the The prediction results of postoperative cognitive dysfunction for 30 patients were obtained; the cognitive dysfunction prediction network consists of three fully connected layers, three random dropout layers and one softmax activation function layer; each fully connected layer is connected to a random dropout layer, the fully connected layer is used to reduce the dimension of the fusion features, and the random dropout layer is used to freeze the parameters of some neurons in the fully connected layer to prevent the model from exploding with gradients during training. The fusion features processed by the last fully connected layer are input into the RELU activation function layer to output the patient's postoperative cognitive function score.

[0084] 5. Training Strategy

[0085] A multimodal model training strategy was designed to train the abdominal CT feature extraction module, the patient clinical and information data encoding module, and the gastrointestinal tumor patient postoperative cognitive dysfunction prediction module. The multimodal model training strategy included an organ prior constraint training strategy and a cross-entropy training strategy, as follows:

[0086] Design an organ prior constraint training strategy to train the 3D-Unet model in the abdominal CT feature extraction module. The specific process is as follows:

[0087] Relying on the prior knowledge of stomach, colon and rectum images provided by the public dataset AMOS, the average shape feature map of the stomach organ, the average shape feature map of the colon organ and the average shape feature map of the rectum organ are obtained; first, for an image data sample in the public dataset, the three-dimensional mask of the stomach, colon and rectum in the image is extracted, and Procrustes analysis is used to align all organ instances to the same coordinate system to eliminate image rotation and translation differences; secondly, vertex-based non-rigid registration is used for each organ, and the aligned shapes of the stomach, colon and rectum organs are vectorized to obtain the shape feature map of the stomach organ, the shape feature map of the colon organ and the shape feature map of the rectum organ; and then the main The component analysis method calculates the shape principal components of the stomach, colon, and rectum shape feature maps, respectively, and retains the top k principal components that explain 95% of the shape variance to obtain the reduced-dimensionality stomach, colon, and rectum shape feature maps. Finally, the above steps are performed on all stomach, colon, and rectum organs in the public dataset AMOS, and all the reduced-dimensionality stomach, colon, and rectum shape feature maps are input into the average pooling layer to obtain the average shape feature map of the stomach, colon, and rectum.

[0088] During the training process, the 3D-Unet model in the abdominal CT feature extraction module performs semantic segmentation on the patient's postoperative abdominal registration images to obtain the stomach feature map, colon feature map, and rectum feature map, and calculates the Frobenius norm between the stomach feature map and the average shape feature map of the stomach organ. , the Frobenius norm between the colon feature map and the average shape feature map of the colon organ and the Frobenius norm between the rectal feature map and the average shape feature map of the rectal organ , and adopt the loss function Fine-tune the parameters of the 3D-Unet model:

[0089] ;

[0090] in is the training set size;

[0091] The abdominal CT feature extraction module, the patient clinical and information data encoding module, and the gastrointestinal cancer postoperative cognitive dysfunction prediction module were trained using a cross-entropy loss function. During training, the 3D-Unet and CLIP models were frozen, and only the neural network structures other than the 3D-Unet and CLIP models were trained. The loss function was the average root mean square error (RMSE) between the predicted patient cognitive dysfunction output by the gastrointestinal cancer postoperative cognitive dysfunction prediction module and the actual patient cognitive dysfunction output. Physicians combined the predicted patient cognitive dysfunction output with clinical experience to diagnose whether a patient had postoperative cognitive dysfunction.

[0092] 6. Experimental Results

[0093] The present invention cooperated with a number of medical centers to collect abdominal imaging data, clinical data and information data of gastrointestinal tumor patients during the perioperative period, and carried out unified preprocessing and privacy protection on them; among them, there were 398 samples of patients with gastric organ tumors, 486 samples of patients with colon organ tumors and 438 samples of patients with rectal organ tumors, with a total of 1,322 samples in the data set. Subsequently, the patients' postoperative cognitive function scores were calculated based on the neuropsychological scale or expert diagnostic criteria to form a well-labeled data set to facilitate subsequent feature extraction and model training. The 1,322 samples were divided into a training set of 1,058 cases and a test set of 264 cases. Among them, 601 / 457 cases had no cognitive dysfunction / cognitive dysfunction in the training set. And 133 / 131 cases had no cognitive dysfunction / cognitive dysfunction in the test set.

[0094] Figure 5 The effect of using the segmentation model to segment intestinal tumors is demonstrated. The fine-tuned TotalSegmentator can accurately segment gastrointestinal tumors.

[0095] In order to better evaluate and optimize the overall performance of the model, the present invention selects a series of commonly used classification and prediction indicators for quantitative evaluation. First, based on the matching of the true label and the predicted result, the model can measure the accuracy of its classification of the sample as a whole by the accuracy rate, while using the recall rate to evaluate the completeness of identifying patients undergoing gastrointestinal tumor surgery, and combining the precision rate or F1 score to balance the recall rate and precision rate. In addition, constructing the ROC curve and calculating the area under it helps to demonstrate the classification performance of the model under different risk thresholds, and provide an objective basis for the selection of thresholds. Through the comprehensive analysis of these evaluation indicators, the present invention can continuously adjust the model structure and optimize the training process, so that it can accurately identify the risk of postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery while minimizing false positives and missed reports, providing more reliable decision support for clinical applications.

[0096] Table 1 Model prediction results

[0097]

[0098] This study evaluated a model predicting postoperative cognitive dysfunction using multimodal data from 264 postoperative patients, including 133 patients without cognitive dysfunction and 131 patients with cognitive dysfunction, demonstrating a relatively balanced distribution of labels. Overall, the model performed well, demonstrating the feasibility and effectiveness of the model-assisted prediction of postoperative cognitive dysfunction in physicians.

[0099] like Figure 6 As shown in the confusion matrix, among 133 patients without cognitive impairment, doctors with limited clinical experience, assisted by the model, successfully identified 111 cases and misclassified 22 cases as having cognitive impairment. Meanwhile, among 131 patients with cognitive impairment, 108 cases were correctly identified as having cognitive impairment and 23 cases were misclassified as not having cognitive impairment. This result indicates that, with the assistance of the model, clinicians' ability to identify the two categories was relatively balanced, with no significant bias.

[0100] Further analysis of performance metrics revealed that the precision for the non-cognitive impairment category was 0.828, recall was 0.835, and F1 score was 0.831; while the precision for the cognitive impairment category was 0.831, recall was 0.824, and F1 score was 0.828. The macro-average F1 score was 0.829, indicating that the model-assisted clinicians maintained relatively consistent performance across the two categories. The weighted average F1 score was also 0.829, indicating that performance was robust across the sample and not significantly affected by class imbalance.

[0101] like Figure 7As shown in the figure, for probability prediction, the area under the receiver operating characteristic (ROC) curve is 0.90, indicating that the model has strong discriminatory power. The overall trend of the ROC curve is relatively ideal, suggesting that the model maintains a good balance between sensitivity and specificity at different probability thresholds.

[0102] Based on the above indicators, the constructed prediction model for postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery demonstrated high accuracy and robustness on this experimental dataset, showing promising potential for early identification of patients with cognitive dysfunction. This model not only enabled clinicians to effectively classify cognitive dysfunction but also maintained a balanced recognition of the two types of samples, providing strong support for clinical intervention and management of postoperative patients. Future research could further expand the sample size, improve the model's generalization capabilities, and further optimize and interpret it in conjunction with clinical characteristics.

[0103] This method not only shows higher accuracy and stability in experimental evaluation, but also has good clinical promotion value. It can provide important reference for doctors in preoperative stratification management and postoperative intervention decision-making, and promote the prediction of cognitive dysfunction in patients after gastrointestinal tumor surgery towards practicality and intelligence.

[0104] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

[0105] Although the above describes the specific implementation methods of the present invention, it does not limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A system for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery, characterized in that: It includes a data receiving and acquisition module, an abdominal CT feature extraction module, a patient clinical and information data encoding module, and a module for predicting postoperative cognitive dysfunction in gastrointestinal tumor patients; The data receiving and acquiring module is used to receive and acquire the patient's abdominal imaging data, the patient's clinical data and the patient's information data; The abdominal CT feature extraction module includes a tumor region feature extraction network, an organ-level local feature extraction network, and a multi-level feature fusion network; the tumor region feature extraction network performs three-dimensional convolution feature extraction on the patient's preoperative abdominal images, captures deep information about the tumor's spatial distribution, density, and morphology, and obtains a tumor morphology feature map; the organ-level local feature extraction network uses the patient's preoperative and postoperative abdominal images as input, segments the stomach, colon, and rectum, and obtains a key organ spatial position feature map; the multi-level feature fusion network is used to fuse the tumor morphology feature map with the key organ spatial position feature map to obtain an abdominal CT image fusion feature map; The patient clinical and information data encoding module performs feature encoding on the patient clinical data and information data to obtain a clinical feature vector; The gastrointestinal tumor patient postoperative cognitive dysfunction prediction module takes the abdominal CT image fusion feature map and the clinical feature vector as input, and outputs a quantitative evaluation result of the patient's postoperative cognitive dysfunction.

2. The system for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery according to claim 1, characterized in that: The abdominal image data includes the patient's preoperative abdominal image and postoperative abdominal imaging of the patient The patient clinical data include perioperative data of patients with gastric, colon and rectal tumors, specifically the results of preoperative neuropsychological assessment of patients , Postoperative neuropsychological evaluation results of patients , anesthesia method , surgical method , intraoperative drug dosage , time parameters and brain function monitoring indicators ; The patient information data includes the patient's past medical history information , patient attribute indicators , gender, height and weight, education level, and description of the patient's current gastrointestinal tumor condition .

3. The system for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery according to claim 1, wherein: The tumor region feature extraction network is an improved architecture based on 3D ResNet-50, including a pre-trained 3D MaskR-CNN layer, a dynamic density histogram equalization layer, a residual attention backbone network and a fully connected layer; Preoperative abdominal imaging of patients Input pre-trained 3D Mask R-CNN layer positioning The tumor area in The image is cropped into a tumor region cube; the tumor region cube is input into the dynamic density histogram equalization layer to enhance the contrast of the tumor boundary in the tumor region cube to obtain an enhanced tumor region cube; the enhanced tumor region cube is input into the residual attention backbone network to obtain the deep features of the tumor morphology. The residual attention backbone network contains four residual blocks, a 3D attention layer and a global adaptive average pooling layer connected in sequence. The number of channels of the four residual blocks connected in sequence are 64, 128, 256 and 512 respectively. Each residual block contains three 3×3×3 convolutional layers for extracting tumor region features layer by layer. Each residual block is followed by a 3D attention layer. The 3D attention layer is used to weightedly fuse the input features and output features of each residual block. The global adaptive average pooling layer is used to compress the feature dimension of the last residual block output by the 3D attention layer to obtain a reduced dimensionality feature; the reduced dimensionality feature is input into the fully connected layer for random inactivation operation to obtain the final tumor morphology feature map.

4. The system for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery according to claim 3, wherein: The organ-level local feature extraction network is an improved architecture based on 3D nnU-Net, including an image registration layer, a pre-trained 3D-Unet model, and a transformer encoder. The specific data processing process is as follows: The first Postoperative abdominal imaging of patients Input image registration layer and patient preoperative abdominal image Perform image registration, and the image registration layer uses the scale-invariant feature conversion algorithm to extract The SIFT feature points and The key matching points of SIFT feature points are used to calculate the affine transformation matrix, and the affine transformation matrix is ​​used to transform The pixels are mapped to Pixel coordinate system, to achieve and Cross-modal spatial registration to obtain the Postoperative abdominal images of patients ; Use the pre-trained 3D-Unet model to Postoperative abdominal images of patients Perform semantic segmentation to obtain Gastric, colonic, and rectal characteristics of each patient; The first The stomach feature map, colon feature map and rectal feature map of each patient are input into the transformer encoder to obtain the first The transformer encoder is used to model the spatial contextual relationship within the organs to obtain more localized spatial feature information that is more relevant to the risk of cognitive impairment.

5. The system for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery according to claim 4, characterized in that: The multi-level feature fusion network The tumor morphology feature map, gastric organ spatial location feature map, colon organ spatial location feature map and rectal organ spatial feature map of each patient are fused; multi-level feature fusion network Includes feature alignment layer, channel attention weighted layer and first feature splicing layer; The feature alignment layer reduces the dimension of the tumor morphology feature map; the feature alignment layer includes a fully connected layer and a 3D transposed convolution; The channel attention weighted layer takes the spatially aligned tumor morphology feature map and organ feature map as input; the channel attention weighted layer includes a global average pooling layer, two fully connected layers and a sigmoid activation function layer; The first feature splicing layer takes the channel-weighted tumor morphology feature map, the gastric organ spatial position feature map, the colon organ spatial position feature map and the rectal organ spatial feature map as input, and performs horizontal feature splicing to obtain an abdominal CT image fusion feature map with a channel number of 1024.

6. The system for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery according to claim 1, characterized in that: The patient clinical and information data encoding module includes a CLIP model, a numerical encoding model and a second feature splicing layer; Input the text data from the patient's clinical and patient information data into the CLIP model, and project different types of structured text into the same latent semantic space to obtain text semantic feature vectors; The numerical data in the patient's clinical and patient information data are first input into the numerical encoding model to convert the numerical data into a numerical feature vector; then the numerical feature vector is input into the encoder of the CLIP model to convert the numerical feature vector into a numerical semantic feature vector whose feature dimension is consistent with the dimension of the text semantic feature vector; The second feature splicing layer performs horizontal feature splicing on the numerical semantic feature vector and the text semantic feature vector to obtain a clinical feature vector.

7. The system for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery according to claim 1, wherein: The postoperative cognitive dysfunction prediction module for gastrointestinal tumor patients includes a global average pooling layer, a dual-head self-attention mechanism network, and a cognitive dysfunction prediction network; The global average pooling layer performs dimensionality reduction on the abdominal CT image fusion feature map, and calculates the average value of all spatial positions of each channel in the abdominal CT image fusion feature map to obtain the abdominal CT image fusion feature vector; The dual-headed self-attention mechanism network takes the abdominal CT image fusion feature vector and the clinical feature vector as input; the dual-headed self-attention mechanism network includes a linear mapping layer, a residual feature splicing layer and a dual-headed self-attention mechanism layer; the abdominal CT image fusion feature vector is input into the linear mapping layer and mapped into a query vector and key vector , the clinical feature vector is input into the linear mapping layer and mapped into a value vector ; Secondly, the clinical feature vector is input into the linear mapping layer and mapped into the query vector and key vector , the abdominal CT image fusion feature vector is input into the linear mapping layer and mapped into a value vector ; again [ ; ; ]and[ ; ; ] Simultaneously input the dual-head self-attention mechanism layer to obtain the weight of the abdominal CT image fusion feature vector relative to the clinical feature vector and the weight of the clinical feature vector relative to the abdominal CT image fusion feature vector ; Again, the abdominal CT image is fused with the feature vector and weight Perform point multiplication to obtain weighted abdominal CT image fusion feature vector, and combine the clinical feature vector with the weight Perform point multiplication to obtain a weighted clinical feature vector; finally, add the weighted abdominal CT image fusion feature vector and the weighted clinical feature vector, and then input them into the residual feature splicing layer together with the abdominal CT image fusion feature vector and the clinical feature vector to perform horizontal feature splicing to obtain the fusion feature; The cognitive dysfunction prediction network includes three fully connected layers, three random dropout layers, and a softmax activation function layer; each fully connected layer is connected to a random dropout layer, the fully connected layer is used to reduce the dimension of the fusion feature, and the random dropout layer is used to freeze the parameters of some neurons in the fully connected layer. The fusion feature processed by the last fully connected layer is input into the RELU activation function layer to output the patient's postoperative cognitive function score.

8. The system for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery according to claim 1, characterized in that: The 3D-Unet model in the abdominal CT feature extraction module is trained using the organ prior constraint training strategy. The specific process is as follows: Relying on the prior knowledge of stomach, colon and rectum images provided by the public dataset, the average shape feature map of the stomach organ, the average shape feature map of the colon organ and the average shape feature map of the rectum organ are obtained; first, for an image data sample in the public dataset, the three-dimensional mask of the stomach, colon and rectum in the image is extracted, and Procrustes analysis is used to align all organ instances to the same coordinate system to eliminate image rotation and translation differences; secondly, vertex-based non-rigid registration is used for each organ, and the aligned shapes of the stomach, colon and rectum organs are vectorized to obtain the shape feature map of the stomach organ, the shape feature map of the colon organ and the shape feature map of the rectum organ; again, the main The component analysis method calculates the shape principal components of the stomach organ shape feature map, colon organ shape feature map, and rectal organ shape feature map respectively, and retains the top k principal components that explain 95% of the shape variance to obtain the reduced-dimensionality stomach organ shape feature map, colon organ shape feature map, and rectal organ shape feature map. Finally, the above steps are performed on all stomach organs, colon organs, and rectal organs in the public dataset, and all the reduced-dimensionality stomach organ shape feature maps, reduced-dimensionality colon organ shape feature maps, and reduced-dimensionality rectal organ shape feature maps are input into the average pooling layer to obtain the average shape feature map of the stomach organ, the average shape feature map of the colon organ, and the average shape feature map of the rectum organ. During the training process, the 3D-Unet model in the abdominal CT feature extraction module performs semantic segmentation on the patient's postoperative abdominal registration images to obtain the stomach feature map, colon feature map, and rectum feature map, and calculates the Frobenius norm between the stomach feature map and the average shape feature map of the stomach organ. , the Frobenius norm between the colon feature map and the average shape feature map of the colon organ and the Frobenius norm between the rectal feature map and the average shape feature map of the rectal organ , and adopt the loss function Fine-tune the parameters of the 3D-Unet model: ； in is the training set size.

9. The system for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery according to claim 1, characterized in that: The cross-entropy loss function was used to train the abdominal CT feature extraction module, the patient clinical and information data encoding module, and the gastrointestinal tumor postoperative cognitive dysfunction prediction module as a whole. The 3D-Unet model and the CLIP model were frozen during the training process, and only the other neural network structures except the 3D-Unet model and the CLIP model were trained. The loss function was the average root mean square error between the patient cognitive dysfunction prediction results output by the gastrointestinal tumor postoperative cognitive dysfunction prediction module and the actual results.

10. A method for predicting postoperative cognitive dysfunction in patients undergoing gastrointestinal tumor surgery, characterized by: Deploying the postoperative cognitive dysfunction prediction system for gastrointestinal tumor surgery patients according to any one of claims 1 to 9 on a detection terminal; and The following processes are included: Real-time acquisition of standard input data of patient abdominal images, patient clinical and patient information; The above standard input data were input into the deployed postoperative cognitive dysfunction prediction system for gastrointestinal tumor surgery patients; The prediction score is output, and the doctor combines the prediction results of postoperative cognitive dysfunction in gastrointestinal tumor patients with clinical experience to diagnose whether the patient will have postoperative cognitive dysfunction.

Images