Whole-process gastric cancer intelligent diagnosis and treatment recommendation method, device, equipment and medium

Abstract

The invention provides a full-process gastric cancer intelligent diagnosis and treatment recommendation method and device, equipment and a medium, and effectively solves the problems that an existing gastric cancer diagnosis and treatment system can only realize partial functions, is single in data and lacks interpretability. The method comprises the following steps: respectively acquiring medical data of a gastric cancer patient from multiple data dimensions, and determining a preprocessing mode of the medical data; preprocessing the corresponding medical data based on the preprocessing mode, and encoding the preprocessed medical data based on the corresponding encoding mode to obtain a corresponding feature vector; fusing the corresponding feature vectors through various fusion mechanisms to obtain fused feature vectors, and inputting the fused feature vectors into a trained diagnosis and treatment recommendation model; and controlling the diagnosis and treatment recommendation model to process the fusion feature vector to obtain a full-process diagnosis and treatment recommendation scheme with explanation information, so as to perform full-process treatment on the gastric cancer patient based on the full-process diagnosis and treatment recommendation scheme.

Description

Technical Field

[0001] The present application relates to the field of medical artificial intelligence technology, and specifically to a full-process gastric cancer intelligent diagnosis and treatment recommendation method, device, equipment and medium. Background Art

[0002] Traditional gastric cancer treatment decisions rely on the experience and judgment of the attending physician, but in actual operation they are limited by experience, incomplete information, data heterogeneity, and individual differences. In particular, there is a great deal of subjectivity and lag in the selection of preoperative / postoperative treatment strategies, efficacy estimation, adverse reaction assessment, and recurrence intervention. In particular, there is a lack of quantitative indicators for recurrence prediction, and a passive situation of "recurrence means late stage" often occurs.

[0003] Despite recent breakthroughs in artificial intelligence (AI) in the medical field, such as imaging diagnostic assistance and gene mutation detection, few have achieved a comprehensive, multimodal, and highly interpretable gastric cancer diagnosis and treatment system. Existing gastric cancer diagnosis and treatment systems that implement some of these functions generally have the following limitations: Single data: Only imaging or clinical information is used, and there is a lack of multimodal integration of imaging, pathology, molecules, nutrition, etc., and molecular markers or imaging, pathology and other information cannot be integrated and utilized at the same time. AI-based processing usually relies on structured medical records and cannot process high-dimensional imaging / pathology data.

[0004] Partial tasks: Most of them only provide diagnosis or prediction at a certain stage, and cannot cover the entire diagnosis and treatment path. They lack multi-task linkage and "modeling of the entire diagnosis and treatment process" and can only assist non-core decision points.

[0005] Lack of interpretability: The model is like a "black box" and lacks the ability to restore clinical pathways and visualize reasoning graphs. Summary of the Invention

[0006] In view of this, the purpose of this application is to provide a full-process intelligent diagnosis and treatment recommendation method, device, equipment and medium for gastric cancer. This full-process intelligent diagnosis and treatment recommendation method, device, equipment and medium for gastric cancer effectively solves the problem that the existing gastric cancer diagnosis and treatment system can only realize partial functions and the data is single and lacks interpretability.

[0007] In a first aspect, embodiments of the present application provide a full-process gastric cancer intelligent diagnosis and treatment recommendation method, which is applied to a full-process gastric cancer intelligent diagnosis and treatment recommendation system. The method includes: Acquire medical data of gastric cancer patients from multiple data dimensions, determine the data type corresponding to the medical data, and determine a preprocessing method for the medical data based on the data type; different data types correspond to different preprocessing methods and encoding methods; preprocess the corresponding medical data based on the preprocessing mode, and encode the preprocessed medical data based on a corresponding encoding mode to obtain a corresponding feature vector; fuse the corresponding feature vectors through a plurality of fusion mechanisms to obtain a fused feature vector, and input the fused feature vector into the trained diagnosis and treatment recommendation model; control the diagnosis and treatment recommendation model to process the fused feature vector to obtain a full-process diagnosis and treatment recommendation scheme with explanation information, so as to perform full-process treatment on the gastric cancer patient based on the full-process diagnosis and treatment recommendation scheme.

[0008] With reference to the first aspect, in a first possible implementation manner of the first aspect, the control of the diagnosis and treatment recommendation model to process the fused feature vector to obtain the full-process diagnosis and treatment recommendation scheme with explanation information comprises: calling a plurality of sub-modules of the diagnosis and treatment recommendation model to process the fused feature vector respectively to obtain corresponding sub-processing results; integrating the sub-processing results corresponding to the plurality of sub-modules to obtain the full-process diagnosis and treatment recommendation scheme.

[0009] With reference to the first aspect, in a second possible implementation manner of the first aspect, the integration based on the sub-processing results corresponding to the plurality of sub-modules to obtain the full-process diagnosis and treatment recommendation scheme comprises: generating a plurality of candidate treatment schemes based on the plurality of sub-processing results, and evaluating the plurality of candidate treatment schemes to obtain corresponding evaluation results; comparing the evaluation results corresponding to the plurality of candidate treatment schemes to select the full-process diagnosis and treatment recommendation scheme.

[0010] With reference to the first aspect, in a third possible implementation manner of the first aspect, the calling of the plurality of sub-modules of the diagnosis and treatment recommendation model to process the fused feature vector respectively to obtain the corresponding sub-processing results comprises: designing a corresponding construction mechanism for each of a plurality of key clinical tasks of the full-process diagnosis and treatment recommendation scheme; constructing and training a plurality of sub-modules through the construction mechanisms corresponding to the plurality of sub-modules to process the fused feature vector respectively.

[0011] With reference to the first aspect, in a fourth possible implementation manner of the first aspect, the integration of the sub-processing results corresponding to the plurality of sub-modules to obtain the full-process diagnosis and treatment recommendation scheme further comprises: calculating and sorting the contribution degrees of a plurality of medical data in the full-process diagnosis and treatment recommendation scheme to select a plurality of key medical data; Based on the plurality of key medical data, a plurality of interpretation information is generated to visualize the interpretation of the whole-process diagnosis and treatment recommendation scheme.

[0012] With reference to the first aspect, in a fifth possible implementation manner of the first aspect, the fusing the corresponding feature vectors through a plurality of fusion mechanisms to obtain a fused feature vector comprises: Based on the corresponding feature vectors, a patient feature map is constructed and a plurality of target feature vectors are screened out; the target feature vectors have dynamic weights; The plurality of target feature vectors, the patient feature map and the remaining feature vectors are fused through a neighborhood weighting mechanism to obtain a fused feature vector.

[0013] With reference to the first aspect, in a sixth possible implementation manner of the first aspect, the preprocessing the corresponding medical data based on the preprocessing manner comprises: Matching the mapping relationship between the data type and the preprocessing manner and data type to obtain a matching result; Based on the matching result, the preprocessing manner of the medical data is determined, and the preprocessing manner is executed.

[0014] In a second aspect, an embodiment of the present application provides a whole-process intelligent diagnosis and treatment recommendation device for gastric cancer, which is applied to a whole-process intelligent diagnosis and treatment recommendation system for gastric cancer, and the device comprises: An acquisition module is configured to acquire medical data of a gastric cancer patient in multiple dimensions from multiple data dimensions, determine a data type corresponding to the medical data, and determine a preprocessing manner of the medical data based on the data type; different data types correspond to different preprocessing manners and encoding manners; An encoding module is configured to preprocess corresponding medical data based on the preprocessing manner, and encode the preprocessed medical data based on a corresponding encoding manner to obtain a corresponding feature vector; An input module is configured to fuse the corresponding feature vectors through a plurality of fusion mechanisms to obtain a fused feature vector, and input the fused feature vector into a trained diagnosis and treatment recommendation model; A treatment module is configured to control the diagnosis and treatment recommendation model to process the fused feature vector to obtain a whole-process diagnosis and treatment recommendation scheme with interpretation information, and perform whole-process treatment on the gastric cancer patient based on the whole-process diagnosis and treatment recommendation scheme.

[0015] In a third aspect, an electronic device is provided, comprising a processor, a memory, and a bus, the memory storing machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to perform the steps of any one of the full-process intelligent diagnosis and treatment recommendation methods for gastric cancer.

[0016] In a fourth aspect, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, when the computer program is run by a processor, the steps of any one of the full-process intelligent diagnosis and treatment recommendation methods for gastric cancer are performed.

[0017] The embodiments of the present application provide a full-process intelligent diagnosis and treatment recommendation method for gastric cancer, which is applied to a full-process intelligent diagnosis and treatment recommendation system for gastric cancer. The method first acquires medical data of a gastric cancer patient from multiple data dimensions respectively, determines data types corresponding to the medical data, and determines a preprocessing manner of the medical data based on the data types. Different data types correspond to different preprocessing manners and encoding manners. Then, the corresponding medical data is preprocessed based on the preprocessing manner, and the preprocessed medical data is encoded based on the corresponding encoding manner to obtain a corresponding feature vector. The corresponding feature vectors are fused through multiple fusion mechanisms to obtain a fused feature vector, and the fused feature vector is input into a trained diagnosis and treatment recommendation model. Finally, the diagnosis and treatment recommendation model processes the fused feature vector to obtain a full-process diagnosis and treatment recommendation scheme with explanation information, and the gastric cancer patient is treated based on the full-process diagnosis and treatment recommendation scheme. The medical data of the gastric cancer patient acquired from multiple data dimensions is processed to generate a full-process diagnosis and treatment recommendation scheme with explanation information, so as to realize the generation and recommendation of a full-process diagnosis and treatment scheme for the gastric cancer patient, and form an explanation and description of the full-process diagnosis and treatment recommendation scheme based on the explainable content, which facilitates the understanding and use of the scheme by the attending physician and the gastric cancer patient, enhances the convenience of the work of the attending physician, and avoids the problems of single data, local task, and lack of explainability in the existing gastric cancer diagnosis and treatment systems that realize part of the functions. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0019] Figure 1A schematic diagram of a full-process gastric cancer intelligent diagnosis and treatment recommendation method provided in an embodiment of the present application is shown; Figure 2 A schematic diagram of the process of obtaining a fused feature vector according to an embodiment of the present application is shown; Figure 3 A schematic diagram of a process for obtaining a full-process diagnosis and treatment recommendation plan provided in an embodiment of the present application is shown; Figure 4 A schematic diagram of the structure of a full-process intelligent diagnosis and treatment recommendation device for gastric cancer provided in an embodiment of the present application is shown; Figure 5 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0021] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0022] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0023] Existing gastric cancer diagnosis and treatment systems that implement partial functions generally have the following limitations: single data, only using imaging or clinical information, lacking the integration of imaging, pathology, molecular, nutritional and other multimodalities, and molecular markers or imaging, pathology and other information cannot be integrated and utilized simultaneously. When processing based on AI, it usually relies on structured medical records and cannot process high-dimensional imaging / pathology data; local tasks: most of them only perform diagnosis or prediction at a certain stage, and cannot cover the entire diagnosis and treatment pathway. There is a lack of multi-task linkage and "full diagnosis and treatment process modeling", and can only assist non-core decision points; lack of interpretability, the model is like a "black box", and does not have the ability to restore clinical pathways and visualize reasoning graphs.

[0024] Based on this, the embodiments of the present application provide a full-process gastric cancer intelligent diagnosis and treatment recommendation method, device, equipment and medium, which are described below through examples.

[0025] Example 1 To facilitate understanding of this embodiment, a full-process gastric cancer intelligent diagnosis and treatment recommendation method disclosed in this embodiment is first described in detail. Figure 1 The flowchart of a full-process gastric cancer intelligent diagnosis and treatment recommendation method shown in the figure is provided. The present application provides a full-process gastric cancer intelligent diagnosis and treatment recommendation method, which is applied to a full-process gastric cancer intelligent diagnosis and treatment recommendation system. The method includes: S101. Obtaining medical data of a gastric cancer patient from multiple data dimensions, determining a data type corresponding to the medical data, and determining a preprocessing method for the medical data based on the data type; different data types correspond to different preprocessing methods and encoding methods; S102, preprocessing the corresponding medical data based on the preprocessing method, and encoding the preprocessed medical data based on the corresponding encoding method to obtain a corresponding feature vector; S103, fusing the corresponding feature vectors through multiple fusion mechanisms to obtain a fused feature vector, and inputting the fused feature vector into a trained diagnosis and treatment recommendation model; S104 , controlling the diagnosis and treatment recommendation model to process the fused feature vector to obtain a full-process diagnosis and treatment recommendation plan with explanatory information, so as to perform a full-process treatment on the gastric cancer patient based on the full-process diagnosis and treatment recommendation plan.

[0026] The full-process intelligent gastric cancer diagnosis and treatment recommendation system described in this application includes a host and a user interface module. The host is standardizedly connected to the HIS / LIS / PACS / pathology system in the hospital to obtain multi-dimensional medical data of gastric cancer patients, and processes the multi-dimensional medical data to generate a full-process diagnosis and treatment recommendation plan with explanatory information, and displays the full-process diagnosis and treatment recommendation plan with explanatory information on the user interface module. The user interface module is used for interactive use by attending physicians to output diagnosis and treatment suggestions, risk maps, survival prediction maps, etc. It is deployed in a cloud-edge collaborative manner, with image preprocessing / Radiomics extraction modules deployed on edge nodes and training / inference services (multi-GPU) deployed on central servers; Docker containerization and Kubernetes elastic scaling are supported.

[0027] In step S101, the present application divides the medical data of gastric cancer patients into multiple data dimensions according to their source system. The data dimensions include clinical information, nutritional scores, laboratory indicators, molecular markers, imaging omics, and pathological omics, as shown in Table 1. Table 1 also provides a column of medical data to illustrate the medical data to be obtained in this data dimension. The medical data of gastric cancer patients are obtained from multiple data dimensions to overcome the problem of single data in the existing medical system. Based on the data dimension, the data type corresponding to the medical data can be determined, that is, there is a corresponding relationship between the data dimension and the data type. Based on the data dimension, the corresponding data type can be determined. type, the data types include structured type, imaging type and pathological image type respectively. The medical data of structured type corresponds to the data dimensions of clinical information, nutritional score, laboratory index and molecular markers. The imaging type corresponds to the data dimension of imaging genomics. The pathological image type corresponds to the data dimension of pathological genomics. This application also establishes a mapping relationship between data type, preprocessing method and encoding method. Different data types correspond to different preprocessing methods and encoding methods. Based on the data type of the medical data, the preprocessing and encoding of the medical data can be realized, thereby realizing targeted processing of multimodal medical data and ensuring the accuracy of the medical data. Table 1 Data dimensions, medical data and source systems .

[0028] For example, the initial diagnosis data of a patient are shown in Table 2: Table 2 Medical data of a patient

[0029] In step S102, after determining the corresponding preprocessing method and encoding method based on the data type of the medical data, the application respectively calls the corresponding preprocessing method and encoding method. The medical data is preprocessed based on the preprocessing method, and the preprocessed medical data is encoded based on the corresponding encoding method. For example, the encoding method of the medical data of the structured type including clinical information, nutritional score, laboratory index, and molecular marker is two-layer full connection layer embedding encoding using MLP, and the output dimension is 128. After standardization processing, the medical data of the image type corresponding to the imageomics is compressed to a 128-dimensional vector through a three-layer FC layer. The medical data of the pathology image type corresponding to the pathologyomics is preprocessed to obtain a ResNet50 or EfficientNet, or Vision Transformer feature vector, and then the feature vector is reduced to 128 dimensions, thereby obtaining a feature vector corresponding to each data type, and the effectiveness and accuracy of the encoding are improved.

[0030] In the specific embodiment of step S102, there is an embodiment that the preprocessing of the medical data based on the preprocessing method includes: S1021, match the mapping relationship between the data type and the preprocessing method and data type, and obtain a matching result; S1022, determine the preprocessing method of the medical data based on the matching result, and execute the preprocessing method.

[0031] In steps S1021-S1022, after the data type of the medical data is determined, the data type is matched with the mapping relationship between the preprocessing mode and the data type to obtain a matching result; if the matching fails, it indicates that a new data type is generated, and at this time, the user interface can be displayed to remind the attending physician to process, or the similarity between the new data type and the above three data types is automatically analyzed, and it is divided into the data type with the highest similarity to determine the corresponding preprocessing mode; if the matching is successful, it indicates that there is a corresponding preprocessing mode, the preprocessing mode of the medical data is determined, and the preprocessing mode is executed. For example, the medical data of the image type corresponding to the imageomics such as CT / MRI image is resampled in standard size after being imported from the DICOM format, so as to adjust all images to a unified voxel size of 1*1*1 mm³, which can be processed by using a trilinear interpolation algorithm; and all images are forced to be center cropped into a standard size of 256*256*128 to ensure that the size of the input to the diagnosis and treatment recommendation model is consistent; the medical data of the pathology image type corresponding to the pathologyomics such as H&E section is cut into a small block of 512*512 pixels by using a method such as a sliding window method, and the invalid block with a background proportion greater than 80% needs to be filtered out, and finally the color normalization is performed by using the Macenko method to unify the color tone; the medical data of the structured type including clinical information, nutritional score, laboratory index and molecular marker has two preprocessing modes, the first one is a numerical variable such as age and albumin level, which is normalized by using formula (1) to map to the interval of 0-1 to prevent inconsistent feature scales, and formula (1) is expressed as: (1); wherein, Xnorm is the normalized medical data, X is the original medical data, Xmin is the minimum value in the original medical data, Xmax is the maximum value in the original medical data; the second one is a category variable such as gender and TNM stage, which is converted into a sparse binary vector based on one-hot encoding, such as gender (Male / Female) -> [1,0] or [0,1].

[0032] The preprocessing of a patient's medical data is as follows: Structured data processing (clinical, nutritional, and molecular markers): Age and PNI values ​​are normalized by Min-Max and scaled to [0, 1]; TNM stage is encoded as a one-hot vector: [T3, T4a, T4b] → [1, 0, 0]; HER2 status (IHC 3+) is annotated as an independent feature bit.

[0033] Image preprocessing: DICOM format CT images were resampled to 1 × 1 × 1 mm voxel size using SimpleITK and cropped to 256 × 256 × 128 dimensions using CenterCrop. 102-dimensional radiomics features (texture, shape, and density statistics) were extracted using pyradiomics.

[0034] Pathology image preprocessing: WSI was used to cut 512 × 512 pixel blocks, and >80% of invalid background images were removed; color was normalized (Macenko method); and a 2048-dimensional pathology semantic feature vector was extracted by pre-training ResNet50.

[0035] In step S103, after obtaining the feature vector of each medical data in each data type, the obtained feature vectors are fused through the neighborhood weighting mechanism, the modality perception weight mechanism and the cross-feature combination mechanism, and the feature vectors output by the modality encoder corresponding to each encoding method are uniformly mapped to the 768-dimensional shared semantic space to obtain the fused feature vector. The weights of each modality are standardized by Linear Projection (linear transformation), that is, each feature vector has a corresponding weight. For multi-source heterogeneous medical data, a three-level fusion strategy of modality-specific encoder + cross-modal Transformer fusion layer + graph structure enhancement is adopted, in which the feature vectors of all modalities are processed in the input shared semantic space to obtain the feature sequence of all modalities; the cross-modal Transformer fusion layer performs cross-modal self-attention interaction through 6 layers of TransformerEncoder; and the Encoder structure of each layer is based on multi-head attention (8 heads), residual connection + It is composed of layer normalization (LayerNorm) and feedforward neural network (FFN) to achieve deep semantic alignment and complementarity of different modal information, improve reasoning accuracy and robustness, and input the fused feature vector into the trained diagnosis and treatment recommendation model; the diagnosis and treatment recommendation model is obtained by combining multiple sub-models to achieve predictions in different directions, ensuring the comprehensiveness and reliability of the obtained full-process diagnosis and treatment recommendation plan.

[0036] In the specific implementation of step S103, there is an embodiment: Figure 2 As shown, the corresponding feature vectors are fused by multiple fusion mechanisms to obtain a fused feature vector, including: S1031. Constructing patient feature maps based on the corresponding feature vectors and screening out multiple target feature vectors; the target feature vectors have dynamic weights; S1032. Fusing multiple target feature vectors, the patient feature map, and the remaining feature vectors through a neighborhood weighting mechanism to obtain a fused feature vector.

[0037] In steps S1031-S1032, when performing feature fusion, the present application constructs patient feature maps based on the corresponding feature vectors, that is, the dimension of the patient feature map is determined by the data type. In this application, there are 3 data types, and the dimension of the patient feature map has 3 dimensions. A radar chart can be used to intuitively display the numerical conditions of each feature, wherein the nodes of the patient feature map are represented by the rows of feature vectors of each modality. The patient feature map is based on the correlation coefficient (Pearson R>0.5) are connected, and while constructing the patient feature map, multiple target feature vectors are also screened out from the feature vector; the multiple target feature vectors are respectively the imaging features corresponding to imaging omics, the pathological features corresponding to pathological omics, and the molecular features corresponding to molecular markers. The target feature vector has a dynamic weight, wherein the dynamic weight is calculated based on the information entropy within the modality corresponding to the target feature vector. The multiple target feature vectors are weighted and fused according to the modality perception weight mechanism, and the multiple target feature vectors, the patient feature map and the remaining feature vectors are fused through the neighborhood weighting mechanism to obtain a fused feature vector, wherein the data dimension of clinical information and the molecular marker dimension introduce a cross feature combination (Feature Cross): For example, [HER2 expression] × [T stage] constitutes a new cross-feature, indicating the invasive risk of a specific molecular subtype. This adaptive combination strategy effectively improves feature complementarity, ultimately increasing the prediction accuracy of the diagnosis and treatment recommendation model by 6.8%. Cross-modal attention guidance mechanism: Transformer not only performs cross-modal modeling, but also guides attention to biologically reasonable modal pairs based on the graph structure, thereby improving fusion effectiveness; adaptive adjustment of modal uncertainty: For patient samples with missing data or low modality quality, uncertainty assessment is introduced to dynamically reduce its influence in the fusion feature to ensure overall reasoning stability.

[0038] In step S104, after obtaining the fusion feature vector and fusing it, the diagnosis and treatment recommendation model is controlled to process the fusion feature vector to obtain a full-process diagnosis and treatment recommendation plan with explanatory information. The diagnosis and treatment recommendation model is composed of 5 sub-modules, including individualized treatment recommendation, adverse reaction prediction, dynamic evaluation of efficacy, recurrence prediction and re-treatment recommendation after recurrence, thereby realizing the full process from diagnosis and treatment plan recommendation to postoperative prediction and recurrence management. Each sub-module focuses on completing a key clinical task and uses a customized AI modeling method to achieve the recommendation goal with the best effect, so as to perform full-process treatment on the gastric cancer patient based on the full-process diagnosis and treatment recommendation plan.

[0039] In the specific implementation of step S104, there is an embodiment: Figure 3 As shown, the control of the diagnosis and treatment recommendation model to process the fused feature vector to obtain a full-process diagnosis and treatment recommendation plan with explanatory information includes: S1041, calling multiple submodules of the diagnosis and treatment recommendation model to process the fused feature vector respectively to obtain corresponding sub-processing results; S1042. Integrate the sub-processing results corresponding to the multiple sub-modules to obtain the full-process diagnosis and treatment recommendation plan.

[0040] In steps S10410 - S1042, after receiving the fused feature vector, the Su Sohu fused feature vector is matched with multiple sub-modules of the diagnosis and treatment recommendation model, that is, the multiple sub-modules of the diagnosis and treatment recommendation model are set with matching rules. Based on the matching rules, if they match, the multiple sub-modules of the diagnosis and treatment recommendation model are called to process the fused feature vector separately to obtain corresponding sub-processing results, that is, each sub-module can output corresponding sub-processing results. Then, according to the preset integration rules, the sub-processing results corresponding to the multiple sub-modules are integrated to obtain the full-process diagnosis and treatment recommendation plan. A PDF version of the decision support report can also be generated based on the full-process diagnosis and treatment recommendation plan, including recommended treatment plans, risk warnings, expected efficacy, etc., to facilitate the attending physician to review.

[0041] This application can also implement a back-tracking reinforcement learning decision tree based on the full-process diagnosis and treatment recommendation plan; annotate the selection reason and candidate treatment plan at each node; support path editing and manual adjustment to facilitate physician correction; output patient-specific Kaplan-Meier survival curves, annotated confidence intervals, turning points, prediction error bands, and predict 3-year overall survival rates. It also supports interactive interpretation and adjustment functions for physicians, allowing them to view the inference change curves under different feature settings.

[0042] In a specific implementation of step S1041, there is an embodiment in which the calling of the multiple submodules of the diagnosis and treatment recommendation model respectively processes the fusion feature vector to obtain corresponding sub-processing results, including: S10411. Based on the multiple key clinical tasks of the full-process diagnosis and treatment recommendation plan, design corresponding construction mechanisms; S10412. Construct and train multiple sub-modules through construction mechanisms corresponding to the multiple sub-modules to process the fused feature vectors respectively.

[0043] In steps S10411-S10412, based on multiple key clinical tasks of the full-process diagnosis and treatment recommendation plan, the multiple key clinical tasks are individualized treatment recommendation, adverse reaction prediction, dynamic evaluation of efficacy, recurrence prediction and re-treatment recommendation after recurrence. The task objectives of each of the above sub-modules are different. Therefore, a corresponding construction mechanism is designed based on each key clinical task, that is, it is constructed based on the PPO or DQN strategy network, and multiple sub-modules are constructed and trained through the construction mechanisms corresponding to multiple sub-modules. During the training process, the patient's historical trajectory data is used to construct a reward function, and long-term survival is used as the optimization goal. Experiments are also carried out on a large number of test data sets and verification data sets to meet actual needs, so as to process the fusion feature vectors separately.

[0044] The personalized treatment recommendation module uses a reinforcement learning policy network (PPO) to infer candidate treatment options, outputs treatment strategies with the goal of maximizing 3-year OS, and outputs a strategy confidence interval for each recommendation to quantify the credibility of the recommendation. The processing flow uses the current patient state as the environment State and all possible treatment combinations as Actions. The reward function is based on a weighted design of long-term OS, DFS, and QoL. The policy network Actor outputs the probability distribution of treatment selection, and the Critic predicts long-term rewards. It also uses a unique "virtual patient trajectory replay" technology to increase the training sample size and avoid overfitting.

[0045] The adverse reaction prediction module is based on a multi-task learning network (MTL) to simultaneously predict six major adverse reactions. It also implements the fusion of focal loss and attention, achieving a higher recall rate for rare adverse reactions such as postoperative anastomotic fistula, and enhancing the ability to identify rare complications. Its processing flow is based on a shared feature extractor (Shared Encoder); each adverse reaction corresponds to an independent output head; each output head uses a Sigmoid binary classifier; the Attention mechanism assists in locating key feature areas, and the unique "Adaptive Loss Balancing" mechanism automatically balances the learning rates of common and rare complications.

[0046] The dynamic efficacy evaluation module uses Bi-GRU+Attention to extract the efficacy evolution trend and thus output the efficacy category (PR / SD / PD) and efficacy curve. Its input is time series data (key indicators before, during and after treatment), and the output is the efficacy level (PR / SD / PD classification) + trend change curve. Its training method is: multi-time step sequence modeling and Attention mechanism to extract key time slices to obtain the inflection point of efficacy change. It also predicts the efficacy level based on Fully Connected and introduces the "automatic detection of efficacy stage turning points" function to assist doctors in adjusting plans in advance.

[0047] The recurrence management module uses a graph neural network (GAT) to infer the recurrence and metastasis map. Its model is an LSTM encoder + DeepSurv survival model, and it outputs the estimated recurrence time based on Cox proportional hazard modeling. Its input is the comprehensive characteristics after treatment; the output is the patient's expected survival curve. Its training method is time-event dual-task modeling and negative log-likelihood (NLL) loss optimization, integrating deep learning time series features with traditional Cox regression hazard proportional theory to improve the accuracy of survival prediction.

[0048] The post-relapse retreatment recommendation module recommends retreatment options based on the UCB multi-arm Bandit strategy. Its model is a Graph Attention Network (GAT). Its output is a distribution of metastasis probabilities for each organ and a prediction of recurrence time. Training is based on node classification tasks (for organ recurrence) and edge weight learning (for potential metastasis pathways). The proprietary "metastasis trend trajectory visualization" function outputs a map of potential recurrence diffusion pathways. Based on nationwide data, a metastasis network diagram (organs as nodes, metastases as edges) is constructed. Using GAT or GNNExplainer architectures, potential recurrence pathways are identified, enabling early warning of micrometastasis trends, achieving exploration-exploitation self-balancing, and dynamically adapting to the recurrence environment.

[0049] The above sub-modules also have corresponding core evaluation indicators, as shown in Table 3.

[0050] Table 3 Submodules and corresponding core evaluation indicators .

[0051] In a specific implementation of step S1042, there is an embodiment in which the integration is based on the sub-processing results corresponding to the multiple sub-modules to obtain the full-process diagnosis and treatment recommendation plan, including: S104211. Generating multiple candidate treatment plans based on the multiple sub-processing results, and evaluating the multiple candidate treatment plans to obtain corresponding evaluation results; S104212. Compare the evaluation results corresponding to the multiple candidate treatment plans to select the full-process diagnosis and treatment recommendation plan.

[0052] In steps S104211-S104212, multiple candidate treatment options are generated based on the multiple sub-processing results, such as candidate treatment option A generated for a gastric cancer patient: direct surgery; candidate treatment option B: neoadjuvant SOX option + surgery; candidate treatment option C: neoadjuvant SOX + anti-HER2 targeted therapy → surgery → supplementary immunotherapy; and the multiple candidate treatment options are evaluated by the long-term benefit value of each candidate treatment option calculated by the 3-year survival rate + QoL comprehensive score to obtain corresponding evaluation results, candidate treatment option A: 0.652; candidate treatment option B: 0.713; candidate treatment option C: 0.842 (highest); then the evaluation results corresponding to the multiple candidate treatment options are compared to select the full-process diagnosis and treatment recommendation plan, and candidate treatment option C becomes the full-process diagnosis and treatment recommendation plan, the full-process diagnosis and treatment recommendation plan: C (neoadjuvant SOX + targeted → surgery → supplementary immunotherapy), predicted complete remission (PR) probability: 87%. This application breaks through the limitations of static recommendation algorithms and introduces reinforcement learning strategy optimization algorithms (such as PPO) to simulate future states from multiple candidate treatment options and maximize long-term prognostic benefits, establish an intelligent closed loop of "strategy-benefit-evolution", and have the ability to learn from doctors' experience and surpass human levels.

[0053] In a specific implementation of step S1042, there is another embodiment in which: integrating the sub-processing results corresponding to the multiple sub-modules to obtain the full-process diagnosis and treatment recommendation plan further includes: S104221. Calculate and sort the contribution of multiple medical data in the full-process diagnosis and treatment recommendation plan to select multiple key medical data; S104222. Based on the multiple key medical data, generate multiple explanatory information to visually explain the full-process diagnosis and treatment recommendation plan.

[0054] In steps S104221-S104222, the contribution of medical data to the full-process diagnosis and treatment recommendation plan is automatically calculated based on the SHAP method, such as the contribution of CT texture heterogeneity (Radiomics feature): 34%; the contribution of HER2 expression status: 29%; the contribution of PNI score: 17%; and the contribution of Ki-67 proliferation index: 11%. The contribution is directly associated with the dynamic weights of the multiple target vectors, and is sorted based on the contribution to select multiple key medical data, i.e., the medical data with the top N contribution rankings. Based on the multiple key medical data, multiple explanatory information is generated to visually explain the full-process diagnosis and treatment recommendation plan. The explanatory information includes a feature contribution graph (barplot) and a local explanatory power graph, as well as a SHAP value graph, a feature contribution radar graph, a path evolution decision tree, etc., to assist doctors in tracing, quality control, and correction, displaying the reasons for selecting path C and the disadvantages compared to the excluded paths A / B. The 3-year overall survival rate based on the full-process diagnosis and treatment recommendation plan is predicted to be: 78.6% (95% CI: 71%-86%), risk of postoperative malnutrition: 35%; it can also be recommended to dynamically monitor PNI changes after surgery and provide nutritional intervention support when necessary. This application is the first to combine the multimodal SHAP feature contribution explanation diagram with the reinforcement learning decision evolution tree to improve doctors' transparent understanding of AI reasoning logic.

[0055] Example 2 This application also provides a full-process intelligent diagnosis and treatment recommendation device for gastric cancer, such as Figure 4 The figure shows a block diagram of a full-process intelligent gastric cancer diagnosis and treatment recommendation device. The functions implemented by this full-process gastric cancer intelligent diagnosis and treatment recommendation device correspond to the steps of executing a full-process gastric cancer intelligent diagnosis and treatment recommendation method on a terminal device. The device can be understood as a component of a server including a processor. The full-process gastric cancer intelligent diagnosis and treatment recommendation device described in this application is applied to a full-process gastric cancer intelligent diagnosis and treatment recommendation system. The device includes: Acquisition module 401 is configured to acquire multi-dimensional medical data of a gastric cancer patient from multiple data dimensions, determine the data type corresponding to the medical data, and determine a preprocessing method for the medical data based on the data type; different data types correspond to different preprocessing methods and encoding methods; An encoding module 402 is configured to preprocess the corresponding medical data based on the preprocessing method, and encode the preprocessed medical data based on the corresponding encoding method to obtain a corresponding feature vector; An input module 403 is configured to fuse the corresponding feature vectors through a plurality of fusion mechanisms to obtain a fused feature vector, and input the fused feature vector into a trained diagnosis and treatment recommendation model; The treatment module 404 is configured to control the diagnosis and treatment recommendation model to process the fusion feature vector to obtain a full-process diagnosis and treatment recommendation scheme with explanation information, and to perform full-process treatment on the gastric cancer patient based on the full-process diagnosis and treatment recommendation scheme.

[0056] In an implementation, the treatment module comprises: The calling module is configured to call a plurality of sub-modules of the diagnosis and treatment recommendation model to process the fusion feature vector to obtain corresponding sub-processing results. The integration module is configured to integrate the sub-processing results of the plurality of sub-modules to obtain the full-process diagnosis and treatment recommendation scheme.

[0057] In an implementation, the treatment module further comprises: The evaluation module is configured to generate a plurality of candidate treatment schemes based on the plurality of sub-processing results, and to evaluate the plurality of candidate treatment schemes to obtain corresponding evaluation results. The comparison module is configured to compare the evaluation results of the plurality of candidate treatment schemes to select the full-process diagnosis and treatment recommendation scheme.

[0058] In an implementation, the treatment module also comprises: The design module is configured to design a corresponding construction mechanism based on a plurality of key clinical tasks of the full-process diagnosis and treatment recommendation scheme. The training module is configured to construct and train a plurality of sub-modules through the corresponding construction mechanisms of the plurality of sub-modules to process the fusion feature vector.

[0059] In an implementation, the treatment module further comprises: The calculation module is configured to calculate and sort the contribution degrees of a plurality of medical data in the full-process diagnosis and treatment recommendation scheme to select a plurality of key medical data. The generation module is configured to generate a plurality of explanation information based on the plurality of key medical data to visually explain the full-process diagnosis and treatment recommendation scheme.

[0060] In an implementation, the input module comprises: The construction module is configured to construct a patient feature map based on the corresponding feature vectors and to screen a plurality of target feature vectors; the target feature vectors have dynamic weights. The fusion module is configured to fuse the plurality of target feature vectors, the patient feature map, and the remaining feature vectors through a neighborhood weighting mechanism to obtain a fusion feature vector.

[0061] In an implementation, the encoding module comprises: A matching module, configured to match the data type with the mapping relationship between the preprocessing method and the data type to obtain a matching result; An execution module is used to determine a preprocessing method for the medical data based on the matching result and execute the preprocessing method.

[0062] Example 3 The present application also provides an electronic device, such as Figure 5 As shown, it includes: a processor 501, a memory 502 and a bus 503, the memory 502 stores machine-readable instructions executable by the processor 501, and when the electronic device is running, the processor 501 and the memory 502 communicate through the bus 503, and when the machine-readable instructions are executed by the processor 501, any one of the steps of the full-process intelligent diagnosis and treatment recommendation method for gastric cancer is executed.

[0063] Example 4 The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program executes any one of the steps of the full-process gastric cancer intelligent diagnosis and treatment recommendation method.

[0064] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0065] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0066] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0067] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, platform server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0068] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A full-process intelligent diagnosis and treatment recommendation method for gastric cancer, characterized by: An intelligent gastric cancer diagnosis and treatment recommendation system applied to the entire process includes: Acquire medical data of gastric cancer patients from multiple data dimensions, determine the data type corresponding to the medical data, and determine a preprocessing method for the medical data based on the data type; different data types correspond to different preprocessing methods and encoding methods; preprocessing the corresponding medical data based on the preprocessing method, and encoding the preprocessed medical data based on the corresponding encoding method to obtain a corresponding feature vector; fusing the corresponding feature vectors through multiple fusion mechanisms to obtain a fused feature vector, and inputting the fused feature vector into a trained diagnosis and treatment recommendation model; The diagnosis and treatment recommendation model is controlled to process the fused feature vector to obtain a full-process diagnosis and treatment recommendation plan with explanatory information, so as to perform full-process treatment on the gastric cancer patient based on the full-process diagnosis and treatment recommendation plan.

2. The method according to claim 1, characterized in that The step of controlling the diagnosis and treatment recommendation model to process the fused feature vector to obtain a full-process diagnosis and treatment recommendation plan with explanation information includes: Calling multiple submodules of the diagnosis and treatment recommendation model to process the fused feature vector respectively to obtain corresponding sub-processing results; The sub-processing results corresponding to the multiple sub-modules are integrated to obtain the full-process diagnosis and treatment recommendation plan.

3. The method according to claim 2, characterized in that The integration is based on the sub-processing results corresponding to the multiple sub-modules to obtain the full-process diagnosis and treatment recommendation plan, including: generating a plurality of candidate treatment plans based on the plurality of sub-processing results, and evaluating the plurality of candidate treatment plans to obtain corresponding evaluation results; Compare the evaluation results corresponding to the multiple candidate treatment options to select the full-process diagnosis and treatment recommendation option.

4. The method according to claim 2, characterized in that The calling of the multiple submodules of the diagnosis and treatment recommendation model to respectively process the fused feature vector to obtain corresponding sub-processing results includes: Based on the multiple key clinical tasks of the full-process diagnosis and treatment recommendation plan, design corresponding construction mechanisms; A plurality of sub-modules are constructed and trained through construction mechanisms corresponding to the plurality of sub-modules to process the fused feature vectors respectively.

5. The method according to claim 2, characterized in that The integrating the sub-processing results corresponding to the multiple sub-modules to obtain the full-process diagnosis and treatment recommendation plan also includes: Calculating and ranking the contribution of multiple medical data in the full-process diagnosis and treatment recommendation plan to select multiple key medical data; Based on the multiple key medical data, multiple explanatory information is generated to visually explain the full-process diagnosis and treatment recommendation plan.

6. The method according to claim 1, characterized in that The fusing of the corresponding feature vectors by multiple fusion mechanisms to obtain a fused feature vector includes: Based on the corresponding feature vectors, patient feature maps are constructed and multiple target feature vectors are screened out; the target feature vectors have dynamic weights; A fused feature vector is obtained by fusing multiple target feature vectors, the patient feature map and the remaining feature vectors through a neighborhood weighting mechanism.

7. The method according to claim 1, characterized in that The preprocessing of the corresponding medical data based on the preprocessing method includes: Matching the data type with the mapping relationship between the preprocessing method and the data type to obtain a matching result; Based on the matching result, a preprocessing method for the medical data is determined, and the preprocessing method is executed.

8. A full-process intelligent gastric cancer diagnosis and treatment recommendation device, characterized by: An intelligent gastric cancer diagnosis and treatment recommendation system for the entire process, comprising: an acquisition module, configured to acquire multi-dimensional medical data of gastric cancer patients from multiple data dimensions, determine the data type corresponding to the medical data, and determine a preprocessing method for the medical data based on the data type; different data types correspond to different preprocessing methods and encoding methods; an encoding module, configured to preprocess the corresponding medical data based on the preprocessing method, and encode the preprocessed medical data based on the corresponding encoding method to obtain a corresponding feature vector; An input module, configured to fuse the corresponding feature vectors through a plurality of fusion mechanisms to obtain a fused feature vector, and input the fused feature vector into a trained diagnosis and treatment recommendation model; The treatment module is used to control the diagnosis and treatment recommendation model to process the fusion feature vector to obtain a full-process diagnosis and treatment recommendation plan with explanatory information, so as to perform full-process treatment on the gastric cancer patient based on the full-process diagnosis and treatment recommendation plan.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of a full-process intelligent diagnosis and treatment recommendation method for gastric cancer as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of a full-process gastric cancer intelligent diagnosis and treatment recommendation method as described in any one of claims 1 to 7.

Images