A surgical decision support method and related device based on context-aware subgraph activation and multimodal parallel reasoning
By using context-aware subgraph activation and multimodal parallel reasoning, local subgraphs are dynamically activated for parallel reasoning, which solves the problems of high latency, disconnect between risk assessment and operation recommendation, and lack of traceability of decision-making in existing surgical decision-making systems, and achieves safe and efficient surgical operation support.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-09
- Publication Date
- 2026-07-10
AI Technical Summary
Existing surgical decision-making systems suffer from high latency in full-map reasoning, lack of risk linkage filtering, multi-criteria ranking, and complete decision traceability capabilities, resulting in high response latency, disconnect between risk assessment and operational recommendations, lack of decision traceability, and the risk of catastrophic forgetting in online updates.
The method employs context-aware subgraph activation and multimodal parallel reasoning. By acquiring real-time intraoperative data and calculating context feature vectors, it dynamically activates local subgraphs for parallel reasoning. Combined with anatomical risk assessment and multi-criteria ranking, it outputs safe operation steps in real time and records the complete decision-making traceability chain.
It achieves secure and reliable decision support with millisecond-level response time, reduces inference latency, enhances risk linkage and decision traceability, improves operational safety and efficiency, and supports postoperative compliance auditing.
Smart Images
Figure CN122369813A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of surgical decision support and medical artificial intelligence, and in particular to a surgical decision support method and related apparatus based on context-aware subgraph activation and multimodal parallel reasoning. Background Technology
[0002] During surgical procedures, decision support systems need to comprehensively process anatomical structures, intraoperative images, and patient data within millisecond response times to provide surgeons with safe and reliable operational suggestions in real time. Existing surgical decision support systems face core challenges such as high latency in full-map reasoning, insufficient risk linkage, and lack of decision traceability.
[0003] Existing surgical decision support methods suffer from the following problems: First, they employ full-graph reasoning instead of context-aware subgraph activation, resulting in low reasoning efficiency and high response latency. Second, they lack multimodal parallel reasoning mechanisms, leading to cumulative latency in sequential reasoning between image and knowledge graphs. Third, they fail to link anatomical risk assessment results with the candidate set of operational steps in real time for risk filtering. Fourth, they lack a multi-criteria decision ranking mechanism that integrates Safety(a), Efficiency(a), and Personalization(a). Fifth, they lack real-time linkage output with the surgical navigation interface. Sixth, they lack a complete decision traceability chain. Seventh, the reasoning framework of existing methods carries the risk of catastrophic forgetting of old knowledge after updates to new data. Summary of the Invention
[0004] To address the above technical issues, this application provides a surgical decision support method and related apparatus based on context-aware subgraph activation and multimodal parallel reasoning.
[0005] Firstly, a surgical decision support method based on context-aware subgraph activation and multimodal parallel reasoning is provided, comprising the following steps: S1. Acquire real-time intraoperative laparoscopic video frames, force sensor readings, and patient vital signs data. Map these data to a unified dimension feature vector using the corresponding encoders, then concatenate and fuse them to obtain a contextual feature vector. S2. Calculate the cosine similarity between the context feature vector and the embedding vector of each node in the surgical knowledge graph, retain the nodes whose similarity exceeds the activation threshold θ_act and their K-hop neighbor nodes to obtain the activation subgraph; S3. Using the activated subgraph as input, a parallel reasoning architecture with a shared graph attention network encoder and two independent reasoning branches is adopted to simultaneously output the dissection risk assessment results and the candidate set of operation steps; S4. Based on the anatomical risk assessment results, perform risk filtering on the candidate set of operation steps, remove candidate operation steps involving high-risk anatomical areas, and obtain a set of safe operation steps; S5. Calculate a multi-criteria comprehensive score for each operation step a in the set of safe operation steps. Where Safety(a) is the safety score, Efficiency(a) is the efficiency score, Personalization(a) is the individualization matching score, λ1+λ2+λ3=1, and the recommended operation steps are obtained by sorting them in descending order of Score(a); S6. Output the recommended operation step sequence to the surgical navigation interface through a standardized interface for AR overlay display, and record the context feature vector, activation subgraph snapshot, anatomical risk assessment result and recommended operation step sequence of each decision as a complete decision traceability link.
[0006] In any embodiment of this application, the laparoscope video frames in the context feature vector are extracted into 256 to 512-dimensional spatial semantic feature vectors using ViT-B / 16, the force sensor readings are normalized into 16 to 64-dimensional temporal feature vectors, and the vital signs data are compressed into 8 to 32-dimensional scalar feature vectors; the three types of vectors are concatenated and compressed into a unified context feature vector of 128 to 256 dimensions.
[0007] In any embodiment of this application, the activation threshold θ_act ranges from 0.60 to 0.85, the K value is recommended to be 2 to 4, and the maximum number of activated nodes is 20% of the total number of nodes in the entire graph; the activated subgraph is dynamically recalculated each time a new scenario is input.
[0008] In any embodiment of this application, the multi-criteria comprehensive scoring is performed based on the following formula: , Where λ1+λ2+λ3=1, and λ1 is not less than 0.40; the recommended steps are sorted in descending order of Score(a), and the first 3 to 5 are output.
[0009] In any embodiment of this application, the parallel inference graph attention network encoder has 4 to 8 attention heads and 2 to 4 layers; the two branches share graph encoder parameters; and the total inference latency does not exceed 50ms.
[0010] In any embodiment of this application, the complete decision tracing link is stored in JSON-LD format, including context feature vectors (timestamps), activation subgraph snapshots, risk assessment results, and Score(a) scores for each step, supporting postoperative compliance audit queries.
[0011] In a second aspect, a surgical decision support device based on context-aware subgraph activation and multimodal parallel reasoning is provided, comprising: a context feature extraction unit, a subgraph activation unit, a parallel reasoning unit, a risk filtering unit, a multi-criteria sorting unit, and a navigation output tracing unit, which respectively execute steps S1 to S6 of the method described in the first aspect.
[0012] Thirdly, an electronic device is provided, including a processor and a storage unit, the storage unit being used to store computer program code containing computer instructions, wherein when the processor executes these instructions, the electronic device performs the method described in the first aspect.
[0013] Fourthly, a computer-readable storage medium is provided, wherein a computer program is stored therein, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method described in the first aspect.
[0014] Fifthly, a computer program product is provided, the computer program product comprising a computer program or instructions that, when the computer program or instructions are run on a computer, cause the computer to perform the method described in the first aspect.
[0015] It should be understood that the above general descriptions and subsequent specific descriptions are for illustrative and explanatory purposes only and do not impose any limitations on this application.
[0016] In this application, compared with the prior art, the technical problem to be solved by this application is: existing surgical decision-making systems suffer from high latency in full-graph inference, lack of risk linkage filtering, multi-criteria ranking, and complete decision tracing capabilities. The technical means adopted in this application and their non-obvious aspects compared with the prior art include: First, this application, based on cosine similarity-based context-aware subgraph activation, only activates local subgraphs related to the current context to perform inference processing, addressing the limitations of existing methods' low efficiency and high response latency in full-graph inference. This design is not seen in existing methods without subgraph activation for full-graph inference. Second, this application, based on a shared graph attention network encoder and a dual-branch parallel inference framework, simultaneously outputs both risk assessment and operation candidate results to address the limitation of accumulated latency in existing methods' serial inference. This design is not seen in serial inference methods without parallel inference branches. Third, this application, based on real-time linkage of anatomical risk assessment results with the operation step candidate set, performs risk filtering to remove high-risk operations, addressing the limitation of existing methods' risk assessment and operation recommendation being disconnected. This design is not seen in existing recommendation systems lacking risk linkage filtering. Fourth, this application, based on a comprehensive approach combining Safety(a) / Efficiency(a) / Pers The onalization(a) three-criterion weighted scoring (Score(a)) addresses the limitation of existing methods' incomplete single-criterion recommendations in ordering safety steps. This design is not seen in existing recommendation methods that only use a single confidence level for ranking. Fifth, this application addresses the limitation of existing methods' recommendation results being detached from the navigation interface by performing AR overlay output of recommended operation steps in real-time linkage with the surgical navigation interface. This design is not seen in existing methods lacking navigation interface linkage. Sixth, this application addresses the limitation of existing methods lacking decision tracing capabilities by recording a complete decision tracing link of contextual feature vector snapshots, activation subgraph snapshots, risk assessment, and recommendation sequence scoring. This design is not seen in existing methods without a complete decision tracing link. Seventh, this application addresses the limitation of existing methods' online updates that risk catastrophic forgetting by supporting online updates that reinforce EWC protection for the inference framework. This design is not seen in ordinary online learning methods without EWC protection. The related devices described in this application include devices, electronic devices, computer-readable storage media, and computer program products. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be explained below.
[0018] Figure 1 This is a flowchart illustrating a surgical decision support method based on context-aware subgraph activation and multimodal parallel reasoning, provided as an embodiment of this application.
[0019] Figure 2This is a schematic diagram of the structure of a surgical decision support device based on context-aware subgraph activation and multimodal parallel reasoning, provided in an embodiment of this application.
[0020] Figure 3 This is a schematic diagram of the hardware architecture of an electronic device provided in an embodiment of this application. Detailed Implementation
[0021] To enable those skilled in the art to more fully understand the technical solution of this application, the technical solution of this application will be explained in detail and clearly with reference to the accompanying drawings.
[0022] In the specification, claims and related drawings of this application, the terms "first" and "second" are used only to distinguish different elements and do not imply any particular order.
[0023] Please see Figure 1 , Figure 1 This is a flowchart illustrating a surgical decision support method based on context-aware subgraph activation and multimodal parallel reasoning, provided for an embodiment of this application. The method includes steps S1 to S6.
[0024] S1. Contextual Feature Extraction: Acquire real-time intraoperative laparoscopic video frames, force sensor readings, and patient vital signs data, and splice and fuse them to obtain a contextual feature vector.
[0025] In this embodiment, the intraoperative real-time multimodal data is synchronously acquired at a frame rate of not less than 10Hz. Laparoscopic video frames are processed using ViT-B / 16 to extract 256 to 512-dimensional spatial semantic feature vectors; force sensor readings are normalized to the [0,1] interval to form 16 to 64-dimensional temporal feature vectors; patient vital signs data (heart rate, blood pressure, blood oxygen) form 8 to 32-dimensional scalar feature vectors. These three types of vectors are concatenated and compressed through a linear projection layer to form a 128 to 256-dimensional unified contextual feature vector.
[0026] S2, Subgraph Activation: Calculate the cosine similarity between the context feature vector and the graph node embedding vector, retain nodes that exceed the activation threshold θ_act and their K-hop neighborhoods, and obtain the activated subgraph.
[0027] In this embodiment, the activation threshold θ_act is recommended to be between 0.60 and 0.85, the K value is recommended to be between 2 and 4, and the maximum number of activated nodes is 20% of the total number of nodes in the entire graph. The activation subgraph is dynamically recalculated each time a new contextual feature vector is input.
[0028] The context-aware subgraph activation mechanism addresses the problem of excessive computational load in full graph reasoning leading to excessive intraoperative response delay. The dynamic subgraph activation based on cosine similarity is not a conventional step in full graph traversal in general graph reasoning.
[0029] S3, Parallel Reasoning: Performs parallel reasoning with activation subgraphs as input, and outputs the dissection risk assessment results and a candidate set of operation steps.
[0030] In this embodiment, the parallel inference framework includes a shared graph attention network encoder (4 to 8 attention heads, 2 to 4 layers) and two independent inference branches: the anatomical risk branch outputs the risk level (high / medium / low) of each anatomical region; the operation step branch outputs a set of candidate operation steps (including step confidence). The two branches are executed in parallel, with a total inference latency of no more than 50ms.
[0031] S4. Risk Filtering: Based on the dissection risk assessment results, perform risk filtering on the candidate set of operation steps to obtain a set of safe operation steps.
[0032] In this embodiment, candidate operation steps involving high-risk anatomical areas (blood vessels, nerves) are directly removed; steps related to medium-risk areas are labeled with risk warning tags but retained; steps in low-risk areas are not processed.
[0033] S5. Multi-criteria sorting: The set of safe operating steps is sorted by a multi-criteria comprehensive score Score(a) to obtain the recommended sequence of operating steps.
[0034] In this embodiment, a multi-criteria comprehensive score is calculated for each candidate step a in the set of safe operation steps.
[0035] The overall score is calculated based on the following formula: , Where Safety(a) is the safety score, Efficiency(a) is the efficiency score, and Personalization(a) is the patient individualization matching score, λ1+λ2+λ3=1, and λ1 is not less than 0.40; the recommended operation steps are arranged in descending order of Score(a), and the first 3 to 5 steps are output.
[0036] The multi-criteria comprehensive scoring and ranking mechanism addresses the problem that single-criteria recommendation cannot simultaneously guarantee safety, efficiency, and personalized matching. The weighted scoring of the three criteria of Safety, Efficiency, and Personalization is not a conventional step in general recommendation systems that is based solely on a single confidence score for ranking.
[0037] S6. Navigation Output Traceability: Outputs the recommended operation step sequence to the surgical navigation interface and records the complete decision traceability link.
[0038] In this embodiment, the recommended operation step sequence is pushed to the surgical navigation system in real time via a RESTful API, and the recommended operation target area is highlighted in the laparoscopic field of view using AR overlay. The complete decision traceability link includes context feature vectors (including timestamps), activation subgraph snapshots, risk assessment results, and recommended step sequences (including Score(a) scores for each step), stored in JSON-LD format, supporting postoperative compliance audit queries.
[0039] Clinical Application Example 1: Taking laparoscopic left hemicolectomy (average 95 minutes) as an example, the average number of activated nodes in the S2 context-aware subgraph was 12% of the entire atlas, with an average activation time of 8ms. The average total latency of S3 parallel inference was 32ms, meeting the 50ms upper limit. S4 risk filtering identified the inferior mesenteric artery as a high-risk area, filtering out 3 candidate operation steps involving this area. S5 multi-criteria ranking output the top 5 recommended steps, with an surgeon adoption rate of 87%. S6 decision tracing fully recorded 423 recommendation events throughout the entire process.
[0040] Clinical Application Example 2: Taking laparoscopic distal pancreatectomy as an example, an abnormal increase in the force sensor reading occurred during the operation, triggering the reactivation of the system's subgraph, which activated a high-risk subgraph containing splenic artery and splenic vein nodes. The set of safe operating procedures after S4 risk filtering did not include direct manipulation of the splenic vessels. S5 recommended the first three steps as low-risk exposure procedures, which the surgeon fully adopted. The decision response time was 40ms, below the 50ms delay limit.
[0041] Please see Figure 2 , Figure 2 A schematic diagram of a surgical decision support device based on context-aware subgraph activation and multimodal parallel reasoning is provided for an embodiment of this application. The surgical decision support device 1 based on context-aware subgraph activation and multimodal parallel reasoning includes: The context feature extraction unit 11 is used to acquire real-time intraoperative laparoscopic video frames, force sensor readings and patient vital signs data, and splice and fuse them to obtain a context feature vector.
[0042] Subgraph activation unit 12 is used to calculate the cosine similarity between the context feature vector and the graph node embedding vector, retain nodes that exceed the activation threshold θ_act and K-hop neighborhoods, and obtain the activated subgraph.
[0043] Parallel reasoning unit 13 is used to perform parallel reasoning with activation subgraphs as input and output the dissection risk assessment results and the candidate set of operation steps.
[0044] Risk filtering unit 14 is used to perform risk filtering on the candidate set of operation steps based on the dissection risk assessment results to obtain a set of safe operation steps.
[0045] The multi-criteria sorting unit 15 is used to sort the set of safe operating steps by multi-criteria comprehensive score Score(a) to obtain the recommended operating step sequence.
[0046] The navigation output traceability unit 16 is used to output the recommended operation step sequence to the surgical navigation interface and record the complete decision traceability link.
[0047] Please see Figure 3 , Figure 3 This is a schematic diagram of the hardware architecture of an electronic device provided in an embodiment of this application. The electronic device 2 includes a processor 21, a memory 22, an input device 23, and an output device 24. The processor 21, the memory 22, the input device 23, and the output device 24 are communicatively connected via a bus. The memory 22 is used to store computer program code, which contains computer instructions. When the processor 21 executes the instructions, the electronic device 2 executes the method.
[0048] The various embodiments in the application have their own focus. For ease of description and simplification, the same or similar content may not be described in detail in different embodiments.
[0049] The method described in this application can be implemented based on hardware, software, or a combination of hardware and software.
Claims
1. A surgical decision support method based on context-aware subgraph activation and multimodal parallel reasoning, characterized in that, The steps include the following: S1. Acquire real-time intraoperative laparoscopic video frames, force sensor readings, and patient vital signs data. Map these data to a unified dimension feature vector using the corresponding encoders, then concatenate and fuse them to obtain a contextual feature vector. S2. Calculate the cosine similarity between the context feature vector and the embedding vector of each node in the surgical knowledge graph, retain the nodes whose similarity exceeds the activation threshold θ_act and their K-hop neighbor nodes to obtain the activation subgraph; S3. Using the activated subgraph as input, a parallel reasoning architecture with a shared graph attention network encoder and two independent reasoning branches is adopted to simultaneously output the dissection risk assessment results and the candidate set of operation steps; S4. Based on the anatomical risk assessment results, perform risk filtering on the candidate set of operation steps, remove candidate operation steps involving high-risk anatomical areas, and obtain a set of safe operation steps; S5. Calculate a multi-criteria comprehensive score for each operation step a in the set of safe operation steps. ， Where Safety(a) is the safety score, Efficiency(a) is the efficiency score, Personalization(a) is the individualization matching score, λ1+λ2+λ3=1, and the recommended operation steps are obtained by sorting them in descending order of Score(a); S6. Output the recommended operation step sequence to the surgical navigation interface through a standardized interface for AR overlay display, and record the context feature vector, activation subgraph snapshot, anatomical risk assessment result and recommended operation step sequence of each decision as a complete decision traceability link.
2. The method according to claim 1, characterized in that, In the context feature vector, the endoscopic video frame is extracted with 256 to 512-dimensional spatial semantic feature vectors by ViT-B / 16, the force sensor readings are normalized to 16 to 64-dimensional temporal feature vectors, and the vital signs data are compressed to 8 to 32-dimensional scalar feature vectors. The three types of vectors are concatenated and compressed to a unified context feature vector of 128 to 256 dimensions by a linear projection layer.
3. The method according to claim 1, characterized in that, The activation threshold θ_act ranges from 0.60 to 0.85, the K value of the K-hop neighborhood ranges from 2 to 4, and the maximum number of activated nodes is 20% of the total number of nodes in the entire graph; the activated subgraph is dynamically recalculated each time a new context feature vector is input.
4. The method according to claim 1, characterized in that, The multi-criteria comprehensive scoring is performed based on the following formula: ， Where λ1+λ2+λ3=1, and λ1 is not less than 0.40; the recommended operation step sequence is arranged in descending order of Score(a), and the first 3 to 5 steps are output.
5. The method according to claim 1, characterized in that, The parallel inference graph attention network encoder has 4 to 8 attention heads and 2 to 4 layers; the two inference branches share the graph encoder parameters; the total inference latency does not exceed 50ms.
6. The method according to claim 1, characterized in that, The complete decision tracing link includes a timestamped contextual feature vector, a complete snapshot of activated subgraph nodes and edges, anatomical risk assessment results, and a recommended sequence of operation steps (including Score(a) for each step), which is serialized and stored in JSON-LD format to support postoperative compliance audit queries.
7. A surgical decision support device based on context-aware subgraph activation and multimodal parallel reasoning, characterized in that, include: The context feature extraction unit (11) is used to perform step S1 as described in claim 1; Subgraph activation unit (12) is used to perform step S2 as described in claim 1; Parallel inference unit (13) is used to perform step S3 as described in claim 1; Risk filtering unit (14) is used to perform step S4 as described in claim 1; A multi-criteria sorting unit (15) is used to perform step S5 as described in claim 1; The navigation output tracing unit (16) is used to perform step S6 as described in claim 1.
8. An electronic device, characterized in that, include: A processor and a storage unit for storing computer program code, the code containing computer instructions, wherein when the processor executes these instructions, the electronic device performs the method of any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program containing program instructions that, when executed by a processor, cause the processor to perform the method described in any one of claims 1 to 6.
10. A computer program product, characterized in that, The computer program product includes a computer program or instructions that, when executed on a computer, cause the computer to perform the method described in any one of claims 1 to 6.