Endoscopic surgery navigation task planning method, system and computing device

Abstract

The application relates to the technical field of surgical navigation, in particular to a laparoscopic surgery navigation task planning method and system and a computing device. The method comprises the following steps: acquiring a structured task state object representing a current intraoperative state of laparoscopic surgery and a pre-constructed directed acyclic surgery task graph; the task graph comprises a plurality of task nodes with precondition and completion condition, and the nodes are connected through a directed task relationship with a transfer condition. The task state object is matched with the precondition and completion condition of each task node, the current task node and its completion state are determined, the target task node is determined, and a task-level planning result for indicating the current execution task and navigation prompt strategy is generated. The application can match the navigation prompt with the actual surgery process, improve the consistency and traceability of task planning, help doctors to identify the current and subsequent tasks in time, reduce the interference of errors, repetitions or outdated prompts on judgment, and provide auxiliary protection for reducing the intraoperative risk of patients.

Description

Technical Field

[0001] This application relates to the field of surgical navigation technology, specifically to a method, system, and computing device for laparoscopic surgical navigation task planning. Background Technology

[0002] Laparoscopic surgical navigation systems can reconstruct three-dimensional anatomical structures based on preoperative CT or MRI images and provide anatomical displays and location cues by combining intraoperative video, spatial registration, and instrument positioning information. However, existing systems focus primarily on structural display and instrument positioning, and adjustments to the task based on the current task, completion status, subsequent structures of concern, and risks still mainly rely on the surgeon's experience and judgment.

[0003] With the development of large language models, visual language models, and multi-agent technologies, systems such as SurgRAW and SurgBox can generate suggestions based on surgical videos, scene descriptions, or user questions. These systems typically use large language models or visual language models as their core, employing chain-like reasoning or multi-agent collaborative reasoning. However, they lack a computable data structure consisting of task nodes, preconditions, completion conditions, and task relationships. Their natural language results are difficult to use as structured input for navigation modules, and they struggle to reliably determine the current and subsequent tasks and trace the progression path.

[0004] Regarding tool invocation, some systems allow the model to autonomously determine the tools and their invocation order, lacking standardized constraints on input data, parameter mapping, invocation processes, and result merging rules. This makes it difficult to form a reusable and pre-auditable execution flow, potentially leading to redundant planning, tool omissions, or changes in the order of tools. In terms of memory mechanisms, some systems only retain dialogue text, lacking structured tracking of task status, risk events, navigation prompts, and doctor feedback. This easily results in duplicate prompts, outdated prompts, or premature task jumps.

[0005] Furthermore, smoke, bleeding, occlusion, and registration deviations can reduce data reliability. Without safety checks and degradation mechanisms for perceived and registration quality, the system may output error messages based on low-quality data; if it cannot receive doctor feedback and update the task status and replan accordingly, the system will remain in open-loop operation, potentially interfering with doctor judgment. Summary of the Invention

[0006] To address the aforementioned issues, this application provides a method, system, and computing device for laparoscopic surgical navigation task planning. It employs a Planner-Worker layered decoupled architecture, using a pre-constructed surgical task graph as the planning basis and a structured task state object as the unified input. Through rule-based task planning, standardized tool calls, multi-dimensional security verification, and closed-loop feedback replanning, the entire chain design achieves traceable, highly reliable, and highly secure laparoscopic surgical task-level navigation decision-making.

[0007] To achieve the above objectives, in a first aspect, embodiments of this application provide a method for planning laparoscopic surgical navigation tasks, comprising: acquiring a current task state object of laparoscopic surgery, wherein the current task state object is structured data representing the current intraoperative state of laparoscopic surgery; acquiring a pre-constructed surgical task graph, wherein the surgical task graph is a directed acyclic graph including multiple task nodes and directed task relationships with transition conditions between the multiple task nodes; wherein each task node corresponds to a surgical task and is configured with preconditions and completion conditions; determining the current task node in the surgical task graph by matching the preconditions of the multiple task nodes according to the current task state object; determining the task completion state of the current task node according to the current task state object and the completion conditions of the current task node; determining the target task node in the surgical task graph according to the task completion state and the directed task relationships; and generating a task-level planning result corresponding to the target task node according to the current task state object and the target task node, wherein the task-level planning result is used to indicate the current execution task and navigation prompt strategy of laparoscopic surgical navigation.

[0008] To achieve the above objectives, in a second aspect, embodiments of this application provide a laparoscopic surgery navigation task planning system, comprising: a state acquisition module, used to acquire the current task state object of the laparoscopic surgery, wherein the current task state object is structured data representing the current intraoperative state of the laparoscopic surgery; a task graph acquisition module, used to acquire a pre-constructed surgical task graph, wherein the surgical task graph is a directed acyclic graph, including multiple task nodes and directed task relationships with transition conditions between the multiple task nodes, each task node corresponding to a surgical task, and each task node is configured with preconditions and completion conditions; a task localization module, used to determine the current task node in the surgical task graph by matching the preconditions of the multiple task nodes according to the current task state object; a task state determination module, used to determine the task completion state of the current task node according to the current task state object and the completion conditions of the current task node; a task advancement module, used to determine the target task node in the surgical task graph according to the task completion state and the directed task relationships; and a planning result generation module, used to generate a task-level planning result corresponding to the target task node according to the current task state object and the target task node, wherein the task-level planning result is used to indicate the current execution task and navigation prompt strategy of the laparoscopic surgery navigation.

[0009] To achieve the above objectives, in a third aspect, embodiments of this application provide a computing device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the laparoscopic surgical navigation task planning method provided by any of the optional implementations of the first aspect.

[0010] This application's embodiments divide laparoscopic surgery into multiple surgical tasks and construct a surgical task graph using task nodes with prerequisites, completion conditions, and progression relationships. During task planning, the current task node and its completion status are first determined based on the current task state object. Then, the target task node is determined by combining the task completion status and task relationships, thereby generating a task-level planning result. Thus, the task-level planning result corresponds to the current intraoperative state, task conditions, and task progression path, helping to improve the consistency and traceability of task planning. It also provides a structured basis for determining the currently relevant task and generating subsequent navigation prompts, reducing the uncertainty caused by black-box reasoning in the model.

[0011] Simultaneously, task planning and tool execution are decoupled. Reusable task templates pre-define input data, tool call order, parameter mapping, and result merging rules, reducing repetitive planning and tool call discrepancies, improving navigation response efficiency and execution consistency, and facilitating verification and optimization of the execution process before clinical application. Structured task context memory continuously records surgical stages, task status, risk events, navigation prompts, and physician feedback, reducing duplicate prompts, failure prompts, and task error jumps, maintaining the temporal continuity of task planning.

[0012] Furthermore, this embodiment performs security checks on navigation prompts based on perceived quality, registration quality, and knowledge status, and performs degradation processing when data reliability is insufficient. Simultaneously, it updates the task status and re-plans based on feedback from doctors regarding confirmation, ignoring, delaying, or closing of navigation prompts. This reduces the risk of erroneous prompts due to low-quality data, enabling doctors to promptly identify current tasks and subsequent concerns, reducing interference from invalid prompts on clinical judgment, and providing auxiliary protection for reducing intraoperative risks for patients, aligning with the clinical principle of doctor-led, system-assisted management. Attached Figure Description

[0013] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A general architecture diagram of a laparoscopic surgery navigation task planning system provided for some embodiments of this application; Figure 2 A schematic diagram of the overall flow of a laparoscopic surgical navigation task planning method provided for some embodiments of this application; Figure 3 A schematic diagram of a pre-construction process for a surgical task map is provided for some embodiments of this application; Figure 4A schematic diagram of a current task node determination process provided for some embodiments of this application; Figure 5 A schematic diagram of a target task node determination process provided for some embodiments of this application; Figure 6 A flowchart illustrating the conversion of task-level planning results into navigation prompts is provided for some embodiments of this application; Figure 7 A flowchart illustrating the invocation of a Worker tool and the execution of a task template is provided for some embodiments of this application; Figure 8 A schematic diagram of a closed-loop replanning process based on doctor feedback is provided for some embodiments of this application; Figure 9 This is a schematic diagram of the structure of a computing device provided for some embodiments of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are all within the protection scope of this application.

[0016] In recent years, laparoscopic surgical robots have become an indispensable and highly valuable intelligent tool in minimally invasive surgical practice. Working in synergy with robotic arms and imaging technology, laparoscopic surgical robots provide precise and effective support for complex laparoscopic surgical scenarios such as urinary system tumor resection, gynecological disease treatment, thoracic and abdominal organ surgery, and gastrointestinal interventional procedures, and are widely used in multiple specialties.

[0017] Laparoscopic surgical robots can integrate laparoscopic surgical navigation systems. These systems provide surgeons with real-time anatomical positioning references by spatially registering preoperative medical images with actual intraoperative anatomical structures. They also offer navigation prompts and information on the surgeon's current surgical status, making them a crucial technology for improving the accuracy and safety of laparoscopic surgery.

[0018] The laparoscopic surgery navigation task planning method provided in this application can be applied to intraoperative navigation workstations, surgical robot navigation systems, augmented reality navigation systems, or other computing devices capable of receiving intraoperative status and outputting navigation information. This method can be used for partial nephrectomy, as well as other laparoscopic surgeries with divisible surgical tasks and task progression relationships.

[0019] Taking partial nephrectomy as an example, the surgical procedure may include perirenal exposure, identification and control of the renal hilar vessels, confirmation of tumor boundaries, tumor resection, wound hemostasis, management of the collecting system, renal parenchyma suturing, and final examination. Different surgical stages correspond to different structures of concern, instrument movements, and risk events. For example, in the renal hilar vessel identification and control stage, the focus can be on the renal artery, renal vein, and occlusion instruments; in the tumor resection stage, the focus can be on the tumor boundaries, resection level, collecting system, and bleeding points; in the wound management stage, the focus can be on suspected open collecting system, wound bleeding, and renal parenchymal defects.

[0020] In some implementations, large language models or visual language models can generate text responses based on surgical videos, scene descriptions, or user questions.

[0021] Large language models are models used to process and generate natural language content, while visual language models are models that can jointly process image or video information and language information.

[0022] Chain-of-Thought, also known as chain thinking, is a method of generating results through multiple intermediate reasoning steps.

[0023] Multi-agent, also known as multi-agent, is a method in which multiple agents with different roles or functions exchange information and collaboratively generate results.

[0024] General surgical question-and-answer or reasoning systems such as SurgRAW and SurgBox can use the above methods to handle surgical questions.

[0025] However, the reasoning process of the aforementioned systems is usually completed within the model itself. Their natural language reasoning results do not automatically form a computable surgical task graph composed of task nodes, preconditions, completion conditions, and task relationships. Therefore, it is difficult to stably determine the current task, task completion status, and subsequent tasks based on the same intraoperative state, and it is also difficult to trace the state information, conditional judgments, and task progression paths upon which navigation suggestions are based. Furthermore, their text output usually cannot be directly used as structured input for registration and positioning modules, augmented reality display modules, or other navigation execution modules, making it difficult to achieve structured integration between task-level planning results and the navigation system.

[0026] In some implementations, task planning and tool invocation are coupled, with the model selecting external tools, determining the invocation order, and organizing tool parameters in real-time based on the current prompts and context. This lacks standardized constraints on input data, tool invocation order, parameter mapping, and result merging rules. Therefore, when faced with the same or similar navigation tasks, the system may repeatedly perform tool selection and process planning, or omit necessary tools or change the invocation order due to contextual changes, affecting execution efficiency and result consistency. Furthermore, it is inconvenient to validate, audit, and optimize the tool execution process before clinical application.

[0027] Ordinary dialogue history primarily records natural language interaction content and typically cannot form structured intraoperative task context memory. It struggles to continuously record the current surgical stage, current task node, task completion status, recent risk events, output navigation prompts, and doctor feedback. When the intraoperative status continuously changes, the system may repeatedly output already processed prompts, continue outputting expired prompts, or prematurely switch to other tasks before the current task is completed, thus affecting the temporal continuity of task planning.

[0028] Furthermore, intraoperative video may be affected by smoke, bleeding, or obstructions; there may be registration discrepancies between preoperative information and the intraoperative space; and information from different knowledge sources may conflict with each other. If the system cannot perform safety checks on navigation prompts based on perception quality, registration quality, and knowledge status, and cannot perform degradation processing such as prompt suppression, range shrinking, or mode switching when data reliability is insufficient, it may generate erroneous prompts with inappropriate confidence based on low-quality data, interfering with the doctor's judgment. If the system cannot receive feedback from the doctor on navigation prompts such as confirmation, ignoring, delaying, or turning them off, and update the task status and replan the task accordingly, it will be difficult to form a closed-loop navigation decision-making process involving the doctor.

[0029] The laparoscopic surgical navigation task planning method provided in this application can be applied to a multi-layer closed-loop navigation system. This architecture includes a semantic map layer, an intraoperative perception layer, a registration and localization layer, an intelligent decision-making layer, and an enhanced display layer.

[0030] The system comprises three layers: a semantic map layer providing preoperative anatomical structure and spatial risk information; an intraoperative perception layer acquiring information on surgical stages, instruments, anatomical structures, and risk events based on intraoperative video; a registration and positioning layer establishing the correspondence between preoperative information and intraoperative space; an intelligent decision-making layer generating task-level planning results by combining intraoperative status, surgical task map, and knowledge retrieval results; and an enhanced display layer outputting corresponding navigation prompts. Registration deviations can be fed back to the semantic map layer to correct the map. Surgeons can re-enter the perception process through intraoperative video, and task planning can trigger knowledge retrieval, adjusting subsequent planning or retrieval strategies based on surgeon feedback. The layers can exchange data using a standardized message format defined by JSON Schema, forming a closed-loop navigation process combining map correction, perception decision-making, prompt feedback, and knowledge enhancement.

[0031] The laparoscopic surgery navigation task planning method provided in this application embodiment can be specifically executed by the intelligent decision layer in a multi-layer closed-loop navigation system.

[0032] This application embodiment uses the current task status object and the surgical task graph as processing objects, so that the determination and advancement of task nodes have a status basis and task path that can be read by the computing device.

[0033] Planner-Worker is an implementation architecture that separates planning and execution. The Planner, or task planning module, determines the current task node, target task node, tool invocation requirements, and navigation prompt strategies. The Worker, or tool execution module, invokes standardized navigation tools or executes task templates based on the task-level planning results. Planner and Worker are optional implementation names and are not limited to specific large models, intelligent agent frameworks, or software products.

[0034] Figure 1 This is a schematic diagram of the overall architecture of a laparoscopic surgical navigation task planning system provided for some embodiments of this application.

[0035] like Figure 1 As shown, the intraoperative perception layer, registration and positioning layer, preoperative navigation semantic map and knowledge graph module provide intraoperative perception information, spatial registration information, structural and risk information and knowledge information, respectively. The computing device organizes the above information with the task context memory into the current task state object.

[0036] The task planning module determines the current task node, task completion status, and target task node based on the current task status object and the pre-built surgical task graph, and generates task-level planning results.

[0037] The tool execution module invokes standardized navigation tools or matches and executes task templates based on the task-level planning results to obtain navigation processing results. Subsequently, candidate navigation prompts are generated based on the task-level planning results and navigation processing results. The candidate navigation prompts are then subjected to security verification and downgrade processing based on the current task status object to obtain the target navigation prompt and send it to the navigation display terminal.

[0038] The computing device can also receive feedback from doctors on target navigation prompts, update the task context memory, generate an updated current task state object, and trigger replanning, thereby forming a closed-loop navigation decision-making process.

[0039] Figure 2 This is a general flowchart of a laparoscopic surgical navigation task planning method provided for some embodiments of this application.

[0040] like Figure 2 As shown, the laparoscopic surgery navigation task planning method provided in some embodiments of this application may include the following steps: Step S110: Obtain the current task status object for laparoscopic surgery.

[0041] For example, the current task state object is structured data that represents the current intraoperative state of laparoscopic surgery, which is used to provide a unified input for subsequent task node location, completion condition judgment and task advancement.

[0042] The computing device can receive data from the intraoperative perception layer, registration and localization layer, preoperative navigation semantic map, knowledge graph module, and task context memory, and organize the received data into key-value objects, JSON objects, message objects, or other structured objects with clearly defined fields and field types. Each of these data sources can provide current-moment perception information, spatial information, knowledge information, and task history. Therefore, the current task state object is not a free text description of the surgical scene, but rather a readable collection of states that can be used to check task conditions.

[0043] In some embodiments, the current task state object may include one or more state dimensions as shown in Table 1 below.

[0044] Table 1

[0045] In some embodiments, the current task status object can be represented using a unified key-value structure or a JSON object, with each field having a clearly defined data type. The current task status object can serve as a unified input for the task planning module, enabling task node location, completion condition checks, risk event assessment, and subsequent task advancement to all be processed based on the same set of intraoperative status fields. This facilitates tracing the data source and status conditions upon which the task-level planning results are based.

[0046] In this embodiment, organizing intraoperative data from different sources and with different representations into a unified current task status object allows the conditions of subsequent task nodes to be matched with the same set of current status fields, thereby providing a data foundation for task planning according to the determined task conditions.

[0047] Step S120: Obtain the pre-constructed surgical task map.

[0048] For example, the surgical task graph is a directed acyclic graph, including multiple task nodes and directed task relationships with transition conditions between the multiple task nodes. Each task node corresponds to a surgical task and is configured with prerequisites and completion conditions.

[0049] Pre-construction refers to the process of determining the surgical tasks involved in laparoscopic surgery, the conditions for each task, and the relationships between them before the current task planning. This information is then constructed into a surgical task graph that can be read by the computing device. The surgical task graph can be represented by a directed acyclic graph G=(V,E), where V represents the set of task nodes and E represents the set of task relationships.

[0050] A task node is a data unit in a surgical task graph used to represent a fine-grained surgical task. Each task node corresponds to a surgical task. A task node may include at least one of the following: task identifier, task name, stage, prerequisites, completion conditions, attention structure, recommended instruments, risk structure, and risk event handling strategy.

[0051] A directed task relationship represents the progression relationship between different surgical tasks, which can be represented by a directed edge from a task node to its successor task node. The transition conditions set on the directed edge are used to limit the state conditions that need to be met when progressing from the current task node to the successor task node.

[0052] Preconditions represent the intraoperative states that must be met for a task to become the current or target task, while completion conditions represent the intraoperative states that must be met for a task to be determined as completed. Both preconditions and completion conditions can be expressed as Boolean expressions that evaluate corresponding fields in the current task state object.

[0053] Taking partial nephrectomy as an example, in some cases, the task node marked T303 can correspond to the "free renal artery trunk" task, which belongs to the vascular control stage. The prerequisites include the renal hilum region being visible and the dissecting forceps or ultrasonic scalpel being in place. The completion conditions include the renal artery free length being not less than 2cm and receiving confirmation from the doctor. The structure of interest is the renal artery trunk, and the recommended instruments include dissecting forceps and ultrasonic scalpel.

[0054] For example, task relationships represent the progression relationships between different surgical tasks. In the case of a directed graph, task relationships can be represented by directed edges from a task node to its successor task node, with task transition conditions set on the directed edges.

[0055] Taking partial nephrectomy as an example, in some cases, after the task of "identifying the renal artery and renal vein" is completed, subsequent tasks such as "freeing the main renal artery" can be performed along the corresponding directed edge.

[0056] Taking partial nephrectomy as an example, the partial nephrectomy task graph may include 6 stages, 32 task nodes and 58 directed edges.

[0057] In this embodiment, the computing device obtains a unified planning basis consisting of task nodes, task conditions, and task relationships, enabling subsequent processing to determine tasks within a predefined task scope and advancement path, without having to temporarily infer surgical tasks from free text.

[0058] It should be noted that steps S110 and S120 can be executed sequentially or simultaneously. When steps S110 and S120 are executed sequentially, there is no restriction on the specific execution order.

[0059] Step S130: Based on the current task status object, match the preconditions of each of the multiple task nodes to determine the current task node in the surgical task graph.

[0060] For example, the current task node is a task node determined based on the current intraoperative state, used to represent the surgical task that is currently being performed or should be processed.

[0061] The computing device can read the preconditions of multiple task nodes in the surgical task graph, evaluate each precondition using the current intraoperative state represented by the current task state object, and then determine the current task node based on the evaluation results of each task node. The current task state object provides the intraoperative state required to evaluate the preconditions, and the preconditions indicate under what intraoperative states the corresponding task node can be identified as the current task node.

[0062] Taking partial nephrectomy as an example, in some examples, the current task status object can indicate that the renal hilum region is currently visible, the dissection forceps are currently being used, and the previous task has been completed.

[0063] The computing device matches the above-mentioned intraoperative states with the preconditions of each task node in the surgical task diagram, and determines the current task node based on the matching results.

[0064] In this embodiment, establishing a correspondence between the current intraoperative state and the preconditions of each task node in the surgical task diagram enables the determined current task node to have a clear state basis.

[0065] Step S140: Determine the task completion status of the current task node based on the current task status object and the completion conditions of the current task node.

[0066] For example, the task completion status is used to indicate whether the surgical task corresponding to the current task node meets its completion conditions. The computing device can read the various completion conditions of the current task node, substitute the corresponding field values ​​in the current task status object into the completion conditions for evaluation, and then determine the task completion status of the current task node based on the evaluation result.

[0067] The completion conditions may involve at least one of the following: structural identification results, structural exposure results, operation completion results, risk event status, and physician confirmation.

[0068] Taking partial nephrectomy as an example, in some examples, the completion condition for T301 "Identify renal artery and renal vein" is that both the renal artery and renal vein have been identified. When the current task status object only indicates that the renal hilum region is visible but no specific blood vessel has been identified, this completion condition is False, and the task completion status of T301 can be determined as incomplete; when the current task status object indicates that both the renal artery and renal vein have been identified, this completion condition is True, and the task completion status of T301 can be determined as completed.

[0069] In this way, the current intraoperative status is checked by using the pre-set completion conditions of the current task node, and the completion status of the current task is converted into a clear status result, providing a basis for determining the target task node in the future.

[0070] Step S150: Determine the target task node in the surgical task graph based on the task completion status and directed task relationship.

[0071] For example, the target task node is the task node that the navigation system needs to perform around, as determined in this round of task planning. It may be an incomplete current task or a subsequent task after completion.

[0072] The computing device can associate the task completion status of the current task node with the directed task relationships in the surgical task graph, and determine the target task node in the surgical task graph based on the association results.

[0073] Taking partial nephrectomy as an example, in some examples, when the current task node is T301 "identify the renal artery and renal vein", the computing device determines the target task node corresponding to this round of task planning based on the task completion status of T301 and the directed task relationship between T301 and other task nodes.

[0074] In this way, the relationship between whether the current task is completed and the progress of the surgical task is used as the basis for determining the target task node, so that the determination of the target task can correspond to the current task status and the task path in the task graph.

[0075] Step S160: Based on the current task status object and the target task node, generate the task-level planning result corresponding to the target task node.

[0076] For example, the task-level planning result is a structured navigation decision plan generated around the target task node, used to indicate the current execution task and navigation prompting strategy for laparoscopic surgical navigation. The current execution task indicates which surgical task the navigation system should currently assist in, and the navigation prompting strategy indicates the structures, risk sources, processing requirements, and prompting methods that need to be addressed around that surgical task.

[0077] The computing device can read the surgical task corresponding to the target task node, and combine it with the current intraoperative state represented by the current task state object to determine the task that needs to be executed and the navigation prompt strategy corresponding to the task, thereby generating a task-level planning result.

[0078] The task-level planning results can be presented using the data structure shown in Table 2 below.

[0079] Table 2

[0080] Taking partial nephrectomy as an example, in some examples, when the target task node is T301 "identify the renal artery and renal vein", the task-level planning result can record the current task as T301, the target structure as the main renal artery and renal vein, and record the prompt type, prompt priority and display method suggestions corresponding to T301.

[0081] The above fields allow the current task being performed, the structure to be focused on, and the navigation hint strategy to be used to be expressed in the same structured result.

[0082] The laparoscopic surgery navigation task planning method provided in this application first organizes the continuously changing intraoperative data into current task state objects that can be conditionally evaluated. Then, it determines the current task based on pre-set preconditions in the surgical task graph, determines the task completion status based on completion conditions, and determines the target task by combining directed task relationships. Finally, it generates a task-level planning result corresponding to the target task node. Thus, the navigation system can not only display the current structure, instruments, or surgical stage, but also determine the specific task to be performed, judge whether the task has been completed, and plan the target task. Since the current task, target task, and navigation prompt strategy can all be mapped to specific state fields, task conditions, and task graph paths, similar or related intraoperative states can obtain consistent planning results according to unified rules. Furthermore, it can trace the task conditions and progress paths upon which the planning results are based, thereby solving the problems of navigation systems struggling to continuously determine the current task and task progress status, and insufficient consistency and traceability of planning results.

[0083] Simultaneously, task planning and tool execution are decoupled. Reusable task templates pre-define input data, tool call order, parameter mapping, and result merging rules, reducing repetitive planning and tool call discrepancies, improving navigation response efficiency and execution consistency, and facilitating verification and optimization of the execution process before clinical application. Structured task context memory continuously records surgical stages, task status, risk events, navigation prompts, and physician feedback, reducing duplicate prompts, failure prompts, and task error jumps, maintaining the temporal continuity of task planning.

[0084] Furthermore, this embodiment performs security checks on navigation prompts based on perceived quality, registration quality, and knowledge status, and performs degradation processing when data reliability is insufficient. Simultaneously, it updates the task status and re-plans based on feedback from doctors regarding confirmation, ignoring, delaying, or closing of navigation prompts. This reduces the risk of erroneous prompts due to low-quality data, enabling doctors to promptly identify current tasks and subsequent concerns, reducing interference from invalid prompts on clinical judgment, and providing auxiliary protection for reducing intraoperative risks for patients, aligning with the clinical principle of doctor-led, system-assisted management.

[0085] Figure 3 This is a schematic diagram of a pre-construction process for a surgical task map provided for some embodiments of this application.

[0086] like Figure 3 As shown, in some embodiments, the pre-construction of the surgical task map can be achieved in the following ways: Step S010: Determine the multiple surgical tasks involved in the laparoscopic surgery, as well as the prerequisites and completion conditions for each surgical task.

[0087] For example, a surgical task is a fine-grained task unit divided according to the operational goals and processing content of laparoscopic surgery. When constructing a surgical task graph, multiple surgical stages can be determined according to the laparoscopic surgical procedure, and each stage can be further subdivided into multiple surgical tasks. Preconditions and completion conditions that can be evaluated based on the current task state object can be set for each surgical task.

[0088] Taking partial nephrectomy as an example, the surgical stages and task nodes can be shown in Table 3 below.

[0089] Table 3

[0090] In some examples, "freeing the main renal artery" can be identified as a surgical task within the vascular control phase. Prerequisites may include visibility of the renal hilum and the presence of dissecting forceps or ultrasonic scalpels. Completion criteria may include a renal artery freeing length of at least 2 cm and physician confirmation. This allows the continuous surgical procedure to be broken down into multiple surgical tasks with clearly defined entry and completion criteria.

[0091] Step S020: Generate multiple task nodes corresponding to multiple surgical tasks.

[0092] For example, generating a task node means creating a corresponding data unit for each surgical task and writing the task's identifier, name, stage, task conditions, and navigation attributes into that data unit. A task node may contain the attributes shown in Table 4 below.

[0093] Table 4

[0094] In some examples, the sample properties of task node T303 "Free Renal Artery Trunk" are shown in Table 5 below.

[0095] Table 5

[0096] Step S020 above can convert each surgical task and its prerequisites, completion conditions, concern structures and risk handling strategies into task nodes that can be read and evaluated by the computing device.

[0097] Step S030: Based on the progression relationship between multiple surgical tasks, determine the successor task node corresponding to each task node.

[0098] For example, the advancement relationship indicates which subsequent surgical tasks can be performed after a surgical task is completed, and the subsequent task node indicates the task node that can be reached from the current task node according to the advancement relationship.

[0099] The computing device can determine one or more successor task nodes for each task node based on the sequential and branching relationships of the tasks in the surgical procedure. In this way, the flow relationship between surgical tasks can be transformed into the sequential connection relationship between task nodes.

[0100] Taking partial nephrectomy as an example, in some cases, T303 "Free renal artery trunk" can be identified as the successor task node of T301 "Identify renal artery and renal vein".

[0101] Step S040: Establish a directed relationship from each task node to its corresponding successor task node, and set task transfer conditions for the directed relationship to obtain the surgical task graph.

[0102] For example, a directed relation is the concrete representation of a directed task relation in a directed graph implementation, consisting of a task node pointing to its successor task node, used to indicate the direction of task progression. Task transition conditions are state conditions set on the directed relation, used to limit the possible progression paths when multiple candidate successor task nodes exist. In this way, task nodes, progression directions, and path selection conditions can be combined into a surgical task graph that can be planned, traversed, and evaluated online.

[0103] In some examples, the surgical task graph is a directed acyclic graph G=(V,E), where V is the set of task nodes and E is the set of directed relations. The directed relation e=(v_i,v_j) indicates that task v_i can be transferred to task v_j after completion.

[0104] A task node can have multiple candidate successor task nodes. The specific successor task node to which a task node is entered is determined by the corresponding task transition conditions and the current task state object.

[0105] For example, a directed relationship from T301 to T303 can be established, and this advancement path can be adopted when T301 is completed and the prerequisites of T303 are met.

[0106] In this embodiment, by constructing a surgical task graph, the surgical procedure is transformed into task nodes, task conditions, and directed progression relationships, providing a data foundation for matching intraoperative states with task nodes and selecting paths after task completion. When the computing device reads and traverses the surgical task graph, it can locate the current task node, check the task completion status, and select the successor task node.

[0107] Figure 4 This is a schematic diagram of a current task node determination process provided for some embodiments of this application.

[0108] like Figure 4 As shown, in some embodiments, step S130 can be implemented in the following ways: Step S131: Determine the current surgical stage based on the current task status object.

[0109] For example, the current surgical stage represents the stage range of the current surgical process, and is used to filter task nodes related to the current surgical process from the entire surgical task graph.

[0110] The computing device can read the phase probability distribution P_phase and the phase transition flag flag_transition from the phase state of the current task state object. The phase probability distribution represents the probability of the phase identification result at each candidate surgical phase, and the phase transition flag indicates whether the intraoperative sensing layer has detected a change in the surgical phase.

[0111] In some examples, when flag_transition=True and the maximum stage probability P_max in the stage probability distribution is not less than 0.7, the stage corresponding to the maximum stage probability can be identified as the new current surgical stage; otherwise, the original surgical stage is maintained and no stage change is made.

[0112] Taking partial nephrectomy as an example, in some cases, when the current task status object indicates that the current stage is P3 (renal hilar vessel identification and control stage), the stage confidence is 0.85, and flag_transition=False, the computing device maintains the P3 stage. This avoids frequently switching the task search range due to fluctuations in a single stage identification result.

[0113] Step S132: Determine multiple candidate task nodes corresponding to the current surgical stage from the surgical task graph.

[0114] For example, candidate task nodes are task nodes selected from the surgical task graph subgraph corresponding to the current surgical stage and are waiting for precondition checks.

[0115] A surgical task graph subgraph is a local surgical task graph composed of task nodes belonging to the same surgical stage and related directed task relationships. After determining the current surgical stage, the computing device reads the surgical task graph subgraph corresponding to that stage and identifies the task nodes within it as candidate task nodes. In this way, the scope of the precondition check can be narrowed down from all task nodes to the task nodes corresponding to the current stage.

[0116] Taking partial nephrectomy as an example, in some cases, when the current surgical stage is P3, task nodes such as "identifying the renal artery and renal vein" and "freeing the main renal artery" in stage P3 can be identified as candidate task nodes.

[0117] Step S133: Determine the current task node from the multiple candidate task nodes based on the current task status object and the preconditions of each candidate task node.

[0118] For example, the computing device substitutes the field values ​​in the current task status object into the preconditions of each candidate task node for evaluation, and determines the current task node from multiple candidate task nodes based on the evaluation results of the preconditions corresponding to each candidate task node.

[0119] The precondition checking function can be `EvaluatePrecondition(v_i.preconditions, task_state)`, where `v_i.preconditions` represents the preconditions of task node `v_i`, and `task_state` represents the current task state object. Task nodes that satisfy all preconditions can form a candidate current task node set `C_current`.

[0120] Taking partial nephrectomy as an example, in some cases, among multiple candidate task nodes corresponding to stage P3, the computing device checks the preconditions of task nodes such as T301 and T303, and determines the current task node based on the matching results of each precondition with the current task state object. Thus, the task search scope is first determined in the surgical stage, and the preconditions then refine the stage-level state into task-level states, avoiding the merging of multiple different surgical tasks within the same stage into a single coarse-grained stage result.

[0121] In this embodiment, all task nodes are first narrowed down to the candidate set corresponding to the current stage, and then the current intraoperative state is matched with the preconditions of the task nodes, so that the current task not only has a stage basis, but also a specific state condition basis, thus improving the stability and interpretability of fine-grained task localization.

[0122] In some embodiments, step S133 may further include steps S1331 and S1332.

[0123] Step S1331: If the current task status object indicates the existence of at least one active risk event, determine the risk priority score for each active risk event.

[0124] For example, an active risk event refers to an intraoperative risk event that still exists and needs to be addressed, and a risk task node refers to a task node pre-set in the surgical task graph for handling the corresponding risk event. The risk event handling strategy of the task node can record the correspondence between risk event types and risk task nodes.

[0125] In some examples, the risk priority score RiskScore_i for each active risk event can be calculated using the following formula: Formula 1; In Formula 1, severity_i represents the severity of the i-th type of event, and risk_weight_i represents the weight of that event in the safety constraints. Exemplary weights for bleeding, suspected open access to the collection system, smoke obstruction, blurred vision, and instrument obstruction can be 1.0, 0.9, 0.5, 0.6, and 0.3, respectively.

[0126] When any risk priority score is not lower than 0.6, the corresponding risk task node can be triggered according to the risk event handling strategy in the task node.

[0127] Step S1332: Determine the risk task node corresponding to the active risk event whose risk priority score meets the preset risk triggering condition as the candidate risk task node, and determine the candidate risk task node with the highest risk priority score as the current task node.

[0128] For example, when the risk priority scores of multiple active risk events respectively meet the preset risk triggering conditions and trigger multiple risk task nodes, the computing device compares the risk priority scores corresponding to each candidate risk task node and determines the candidate risk task node with the highest risk priority score as the current task node. Thus, when multiple risk events are active simultaneously, the priority risk task can be determined based on the severity of the risk events and the preset risk weight, avoiding the inability to determine the current task node when multiple risk task nodes simultaneously meet the triggering conditions.

[0129] For example, if the severity of a bleeding event is 0.7 and its weight is 1.0, its risk priority score is 0.7, which can trigger the T501 "Intraoperative Bleeding Management" risk task node. Risk task nodes take precedence over regular task nodes; even if the prerequisites for a regular task node have been met, the risk task node will be determined as the current task node first.

[0130] Step S1333: If the current task status object indicates that there are no active risk events, determine the current task node according to the task order of multiple candidate task nodes in the surgical task graph.

[0131] For example, the task order represents the sequence of multiple surgical tasks determined by the surgical task graph, which can be represented by the topological order in the surgical task graph. When there are no active risk events and the preconditions of multiple regular task nodes are met simultaneously, the task node with the earlier topological order can be selected.

[0132] For example, when both T301 and T303 meet the prerequisites and T301 has a higher topological order, T301 is determined as the current task node. This process ensures that task selection in the absence of risk intervention returns to a pre-determined normal task path, thus allowing objects with the same current task state to obtain the current task node according to a stable task order.

[0133] In this embodiment, when an active risk event occurs, task selection shifts from normal progression to a risky task. Once the risk is resolved or no longer exists, the normal order in the task graph is restored. Thus, the risk status alters the intermediate result of task node selection, allowing navigation planning to prioritize risk handling without losing the normal task path.

[0134] In some embodiments, the current task node can also be verified based on the current instrument, action, and structural state. Let the set of observed instruments be `observed_instruments`, the current task node be `v_current`, and the recommended instrument set for the current task node be `v_current.recommended_instruments`. The instrument consistency score `Score_consistency` can be obtained using the following formula: Formula 2.

[0135] For example, the instrument set observed_instruments can be {scissors, suction device}, the current task node v_current can be "tumor boundary confirmation", and the recommended instrument set v_current.recommended_instruments of the current task node can be {scissors, electrocoagulation device, suction device}.

[0136] In the example above, the intersection of the two sets is {scissors, suction device}, and the union is {scissors, electrocautery device, suction device}. Therefore, the device consistency score is 2 / 3. The higher the score, the more consistent the currently detected device is with the predefined recommended devices of the current task node.

[0137] When the consistency score is less than 0.4, the confidence level of the current task can be reduced, and the source of inconsistency can be recorded in the current task status object or task-level planning results. In this way, the current task is not only determined by stages and sequence, but can also be subject to reverse verification by the actual instrument status.

[0138] Figure 5 This is a schematic diagram of a target task node determination process provided for some embodiments of this application.

[0139] like Figure 5 As shown, in some embodiments, step S150 may include steps S151 to S153.

[0140] Step S151: If the task completion status indicates that the current task node is not completed, then the current task node is determined as the target task node.

[0141] For example, "current task node not completed" means that at least one of the completion conditions for the task node has not yet been met by the corresponding field in the current task status object. In this case, the current task node remains in progress, and the target task node remains the current task node. This maintains the continuity of task execution and avoids premature progress when the task completion conditions have not been met.

[0142] Taking partial nephrectomy as an example, in some cases, when the completion condition of T301, "both renal artery and renal vein have been identified", has not yet been met, T301 will continue to be identified as the target task node.

[0143] Step S152: If the task completion status indicates that the current task node has been completed, determine the successor task node of the current task node based on the directed task relationship.

[0144] For example, a successor task node is a task node that is connected to the current task node through a directed task relationship and can be entered after the current task node is completed.

[0145] The computing device can read one or more successor task nodes based on the directed task relationships between the current task node and other task nodes. In this way, the task completion status can be converted into a definite result for subsequent optional task nodes.

[0146] Taking partial nephrectomy as an example, in some cases, T301 is determined to be completed, and subsequent task nodes, including T303 "free renal artery trunk", can be obtained according to the directed task relationship.

[0147] Step S153: Determine the target task node from the successor task nodes based on the current task status object and the prerequisites of the successor task node.

[0148] For example, the computing device substitutes the field values ​​in the current task state object into the preconditions of each subsequent task node for evaluation, and determines the target task node based on the evaluation results of each subsequent task node. In this way, the target task node can correspond to the preconditions that have been met in the current intraoperative state.

[0149] Taking partial nephrectomy as an example, in some cases, when the prerequisites for T303, "the renal hilum region is visible" and "the dissecting forceps or ultrasonic scalpel is in place", are both met, T303 can be determined as the target task node based on the evaluation result.

[0150] In this embodiment, the task will not jump prematurely if the current task node is not completed, and it will not simply enter the next task node according to a fixed sequence number after the current task node is completed. Instead, it will further check the prerequisites of the subsequent task node so that the task progress matches the actual intraoperative state.

[0151] Figure 6 This is a schematic diagram illustrating a process for converting task-level planning results into navigation prompts, provided for some embodiments of this application.

[0152] like Figure 6 As shown, in some embodiments, after obtaining the task-level planning results, the method provided in this application embodiment may further include: Step S210: Perform navigation processing based on the task-level planning results to obtain the navigation processing results.

[0153] For example, navigation processing is a process of obtaining navigation prompts and generating the required information based on the task-level planning results, and the navigation processing result is the data output by this process.

[0154] The computing device can perform corresponding data processing based on the current task and navigation prompt strategy indicated by the task-level planning results, and organize the obtained task-related information into navigation processing results, thereby converting the planning requirements at the task level into data results for prompt generation.

[0155] Taking partial nephrectomy as an example, in some cases, when the task-level planning results indicate that the current focus should be on the renal artery trunk and renal vein, the computing device can perform corresponding navigation processing around the above target structures to obtain structural information, spatial information or risk information for generating navigation prompts.

[0156] Step S220: Generate candidate navigation prompts based on the task-level planning results and navigation processing results.

[0157] For example, a candidate navigation hint is a navigation hint object that is formed before security checks are completed.

[0158] The computing device determines the target task node and initial navigation prompt strategy corresponding to the candidate navigation prompts based on the task-level planning results, and generates corresponding prompt information using the navigation processing results.

[0159] Candidate navigation prompts can include prompt type, prompt content, target audience, risk level, display method, prompt confidence level, and prompt basis.

[0160] The prompt type indicates the function of the prompt. The prompt content carries the information provided to the doctor. The target object identifies the structure or area to which the prompt points. The risk level indicates the current level of risk. The display method indicates the presentation format of the prompt. The prompt confidence level indicates the reliability of the prompt's result. The prompt basis records the data basis provided by the task-level planning results and navigation processing results.

[0161] For example, candidate navigation suggestions can adopt the data structure shown in Table 6 below.

[0162] Table 6

[0163] Taking partial nephrectomy as an example, in some cases, when T301 is identified as the target task node and the corresponding navigation processing results are obtained, candidate navigation prompts can be generated to help identify the renal artery and renal vein. The target object is the renal hilum region, and the corresponding navigation processing results are recorded in the prompt basis. In this way, the target task node and navigation processing data can be combined into candidate navigation prompts with clearly defined fields.

[0164] Step S230: Perform a security check on the candidate navigation prompts based on the current task status object, and adjust the candidate navigation prompts according to the security check results to obtain the target navigation prompt.

[0165] For example, safety constraints are preset conditions that must be met when outputting candidate navigation prompts, and safety verification is the process of determining whether candidate navigation prompts meet safety constraints based on the current task state object. The safety verification result indicates whether candidate navigation prompts need to be adjusted, and the target navigation prompt is the prompt formed after adjusting the candidate navigation prompts accordingly based on the safety verification result.

[0166] In some examples, when security verification results indicate that candidate navigation hints are not suitable for output according to the original navigation hint strategy, the computing device can adjust the corresponding fields of the candidate navigation hints to obtain the target navigation hint. This allows the target navigation hint to be adapted to the intraoperative state represented by the current task state object.

[0167] Step S240: Output target navigation prompts.

[0168] For example, outputting a target navigation prompt means sending the target navigation prompt to the corresponding navigation display terminal.

[0169] Target navigation prompts can be sent to the enhanced display layer as JSON objects, message objects, or other structured formats, and output in the manner indicated by the target navigation prompts. Display methods can include at least one of the following: text, icons, space overlays, arrows, voice, and picture-in-picture.

[0170] In this way, the task-level planning results that have passed security verification can be converted into prompts that can be received and displayed on the navigation display.

[0171] In this embodiment, the above process ensures that the task-level planning results are processed through navigation, candidate navigation prompts are generated, and security checks are performed before the target navigation prompts are formed. This avoids the task-level planning results being directly converted into navigation outputs without verification, and ensures that the target navigation prompts correspond to the target task nodes while retaining the constraints of the current data reliability.

[0172] Figure 7 This application provides a flowchart for calling a Worker tool and executing a task template, which is part of some embodiments of the present application.

[0173] like Figure 7 As shown, the tool execution module receives the task-level planning results and determines whether a task template can be matched from the pre-built task template library based on the current task status object and the task-level planning results.

[0174] If no task template is matched, the tool execution module determines the tool invocation requirements based on the task-level planning results, invokes the corresponding standardized navigation tool according to the tool invocation requirements, obtains the tool execution results, and determines them as the navigation processing results.

[0175] When a task template is matched, the tool execution module obtains the template input data from the current task status object, calls the standardized navigation tools according to the tool call order specified by the task template, and merges the execution results of each tool according to the result merging rules specified by the task template to obtain the task template execution result, and determines the task template execution result as the navigation processing result.

[0176] Subsequently, candidate navigation prompts are generated based on the task-level planning results and navigation processing results.

[0177] In some embodiments, step S210 may include steps S211 to S213.

[0178] Step S211: Determine tool call requirements based on task-level planning results.

[0179] For example, a tool invocation requirement is used to indicate which standardized navigation tools need to be invoked when performing navigation processing for a target task node, and may include the tool name and the order in which the tools are invoked.

[0180] The computing device can read the target structure, risk concern information, and task rule requirements from the task-level planning results and match them with the functions of various standardized navigation tools. In this way, the navigation requirements in the task-level planning results can be transformed into explicit tool execution projects.

[0181] In some examples, when the attention structure of the target task node lacks spatial information, it can be determined that a distance query tool needs to be invoked.

[0182] In some examples, when the current operating area is close to a risk structure, it can be determined that a risk assessment tool needs to be invoked.

[0183] In some examples, when the surgical task corresponding to the target task node requires surgical rules or anatomical knowledge, it can be determined that a knowledge retrieval tool needs to be invoked.

[0184] Taking partial nephrectomy as an example, in some cases, when the structure of interest in T301 lacks distance information and it is necessary to query the anatomical rules of the renal hilum, it can be determined to call the structure query tool, distance query tool and knowledge retrieval tool in sequence.

[0185] Step S212: Based on the tool call requirements, call the corresponding standardized navigation tool through the tool execution module.

[0186] For example, the tool execution module is a software functional unit that organizes and executes standardized navigation tools according to tool call requirements. It does not directly determine the current surgical task, nor does it independently generate the final medical judgment. Instead, it performs data processing specified by the task-level planning results.

[0187] When multiple tools are invoked sequentially, the output of the previous tool can be written into the input of the next tool according to parameter mapping rules. These rules define which output fields of the previous tool are used as input fields of the next tool. This ensures that the tools execute in the planned order and according to their parameter sources.

[0188] Taking partial nephrectomy as an example, in some cases, the structural identifier and location of the renal artery trunk can be obtained first by the structural query tool, and then the structural identifier and location can be passed to the distance query tool.

[0189] Step S213: Obtain the tool execution result returned by the standardized navigation tool, and determine the tool execution result as the navigation processing result.

[0190] For example, the tool execution result is structured data returned by a standardized navigation tool after processing.

[0191] In some examples, when a single standardized navigation tool meets the navigation processing requirements for this round, the result of that tool's execution can be directly determined as the navigation processing result.

[0192] In some examples, when multiple tools are invoked sequentially, the execution results of each tool can be aggregated and used as subsequent candidate navigation prompts to generate the navigation processing results used.

[0193] For example, standardized navigation tools can be shown in Table 7 below.

[0194] Table 7

[0195] In this embodiment, the task planning module is responsible for determining task nodes and tool call requirements, while the tool execution module is responsible for obtaining data according to the tool call requirements. The two are connected by the task-level planning results, thereby reducing the possibility of the tool execution module arbitrarily changing task nodes.

[0196] The tool execution results can be represented by the data structure shown in Table 8 below.

[0197] Table 8

[0198] Taking partial nephrectomy as an example, in some cases, the distance query tool can return that the tip of the dissecting forceps is about 4.8 mm from the main renal artery and about 6.2 mm from the renal vein, and record the execution status, distance data, result confidence and semantic map source in the tool execution results.

[0199] In this embodiment, the task-level planning results first determine the tool requirements, and the tool execution process is only responsible for obtaining the results according to the requirements, thus forming a clear division of labor between the planning task and the data execution. Different tools also use a unified result format, which can reduce the impact of tool omissions, unstable calling order, and inconsistent result formats on the generation of subsequent prompts.

[0200] In some embodiments, step S210 may further include steps S214 to S217.

[0201] Step S214: Based on the current task status object and the task-level planning results, determine the matching task template from the pre-built task template library.

[0202] For example, the task template library is a collection of predefined task templates used to store reusable tool combination processes for common navigation scenarios. A task template is a data unit in the task template library and may include a template identifier, template name, trigger conditions, input requirements, tool call order, result merging rules, security constraints, and output format.

[0203] The computing device matches the status fields in the current task status object, the target task nodes and risk concern information in the task-level planning results with the trigger conditions of each task template, and determines the task template whose trigger conditions are met as the matching task template. In this way, a pre-defined combination of tools can be selected for the current navigation scenario.

[0204] For example, an active bleeding event can be matched with the triggering conditions of a bleeding event handling prompt template.

[0205] The task template can use the data structure shown in Table 9 below.

[0206] Table 9

[0207] JSON Schema is a schema definition used to define the fields, types, and nesting relationships of structured JSON data. Through output format constraints, the processing results generated by different task templates can be read by the subsequent prompt generation process according to a stable data structure.

[0208] Step S215: Based on the task template, obtain the template input data from the current task status object, and call the corresponding standardized navigation tools according to the tool call order specified by the task template.

[0209] For example, the template input data are the status fields or planning fields required for the execution of the task template, and their specific scope is limited by the input requirements of the task template.

[0210] The computing device extracts relevant data from the current task state object based on the input requirements and sequentially calls standardized navigation tools according to the tool invocation order, while transferring data between tools according to parameter mapping rules. In this way, the current intraoperative state can be substituted into a standardized tool flow that matches the current scenario.

[0211] Taking partial nephrectomy as an example, in some cases, when executing the renal hilum vessel control prompt template, the tool can be called in the following order: "structure query - registration quality query - distance query - risk assessment - knowledge retrieval - prompt generation".

[0212] Step S216: According to the result merging rules defined by the task template, merge the tool execution results returned by each standardized navigation tool to obtain the task template execution result.

[0213] For example, the result merging rule is used to define how fields from multiple tool execution results are combined into a unified result; the task template execution result is structured data merged according to the rule and organized according to the template output format.

[0214] In some examples, for the same field, the result of a later-called tool can take precedence over the result of an earlier-called tool; for different fields, they can be directly merged. In this way, the data returned by multiple tools can be converted into a single processing result that can represent the current navigation scenario as a whole.

[0215] For example, when the structure query tool returns the structure location, the distance query tool returns the structure distance, and the risk assessment tool returns the risk level, the location, distance, and risk level fields can be merged into the same task template execution result.

[0216] Step S217: Determine the task template execution result as the navigation processing result.

[0217] For example, the execution result of a task template may include a template identifier, execution status, and output data.

[0218] The template identifier indicates which tool combination process was used, the execution status indicates whether the template was fully executed, and the output data carries the merged results organized according to the output format.

[0219] Once the execution result of the task template is determined as the navigation processing result, the candidate navigation prompt generation process can directly read the template identifier, execution status, and merged output data, without having to parse the execution results of each tool separately. In this way, the pre-encapsulated multi-tool processing flow and the subsequent candidate navigation prompt generation process can form a unified data interface.

[0220] The example task templates and their tool call order are shown in Table 10 below.

[0221] Table 10

[0222] In steps S214 to S217 above, the template triggering condition determines when the established process is activated, the tool call order and parameter mapping determine how to obtain data, and the result merging rule determines how to form a unified output. This enables common navigation tasks to be executed according to a predefined and verifiable process, improving processing consistency and maintainability.

[0223] In some embodiments, step S230 may include steps S231 to S233.

[0224] Step S231: Perform a safety check on the candidate navigation prompts based on at least one of the perception quality, registration quality, and knowledge state in the current task state object.

[0225] For example, perception quality represents the reliability of intraoperative perception results after being affected by factors such as smoke, bleeding, occlusion, or blurred vision. Registration quality represents the reliability of the spatial correspondence between the preoperative model and the intraoperative field of vision. Knowledge state represents the currently used rules, guidelines, similar cases, and their source credibility or conflict status. In this embodiment, these states can be used to determine whether candidate navigation prompts meet the conditions for outputting as is.

[0226] The computing device can verify candidate navigation prompts using the perception quality, registration quality, and knowledge state of the current task state object, thereby generating a safety verification result. This allows determination of whether the candidate navigation prompts currently have limitations related to data quality or knowledge state.

[0227] Step S232: Based on the security verification results, determine the prompt type, prompt priority, display method, prompt confidence level, or prompt content that needs to be adjusted.

[0228] For example, the adjustment object represents the field in the candidate navigation prompt that needs to be changed based on the security check result.

[0229] The computing device can determine at least one field that needs adjustment based on the security verification results, including the prompt type, prompt priority, display method, prompt confidence level, and prompt content. This allows the security verification results to be translated into explicit prompt field adjustment requirements.

[0230] Step S233: Adjust the candidate navigation prompts according to the determined adjustment objects to obtain the target navigation prompts.

[0231] For example, the computing device modifies the fields identified as requiring adjustment in the candidate navigation prompts according to the security verification results, while retaining other fields that do not need adjustment, thus forming the target navigation prompt. In this way, the security verification results can be reflected in the field changes of the target navigation prompts.

[0232] In some examples, when the security check results indicate that the prompt content needs to be adjusted, the prompt content field in the candidate navigation prompt can be modified, while other fields such as target task node and target object are retained.

[0233] In some embodiments, step S233 may include at least one of the following processes.

[0234] In some examples, when the perceived quality score Q_perception is less than 0.5, the prompt type is adjusted to a downgraded prompt, the prompt confidence is multiplied by 0.6, and uncertain expressions such as "suspected," "possible," or "recommend review" are added to the prompt content.

[0235] Q_perception is used to comprehensively characterize the reliability of the intraoperative perception results; 0.5 is an exemplary preset perception quality condition; and 0.6 is an exemplary confidence decay factor used when the perception quality is insufficient.

[0236] For example, when the confidence level of a candidate navigation cue is 0.85 and Q_perception = 0.48, the adjusted cue confidence level is 0.51. This processing reduces the certainty of the cue and reflects the intermediate state of insufficient perceived quality in the cue type, confidence level, and wording.

[0237] In other examples, when the registration quality score Q_registration is less than 0.6, the output of precise spatial location prompts is prohibited, and spatial display methods such as 3D highlighting or arrows are changed to text prompts, with a note indicating that the current registration accuracy is insufficient and the spatial information is for reference only.

[0238] Wherein, Q_registration represents the quality score of the spatial correspondence between the preoperative model and the intraoperative field of view, and 0.6 is an example of the preset registration quality condition.

[0239] For example, when the registration quality score is 0.38 and the spatial deviation is 11mm, "3D highlight + text" can be changed to "text", and a note stating that the spatial information is for reference only can be added to the prompt content. This processing reduces the spatial display capability as the registration reliability decreases, preventing low-quality registration from continuing to support accurate spatial prompts.

[0240] In this embodiment, different thresholds related to registration quality can be used for different judgments: 0.6 can be used as a safety verification threshold for whether to allow the output of precise spatial location prompts; 0.7 can be used as a threshold for suggesting the adoption or restoration of spatial overlay display; and 0.5 can be used as a threshold for forcibly maintaining text prompts during dynamic replanning. These thresholds correspond to different processing stages and are not set differently for the same judgment condition.

[0241] In other examples, when multiple sources of knowledge reach inconsistent conclusions on the same issue, candidate navigation suggestions are marked as pending confirmation, and the viewpoints of different sources are recorded in the suggestion content.

[0242] For example, when one guideline recommends a safe cutting edge of not less than 3 mm and another study recommends a safe cutting edge of not less than 5 mm, both viewpoints and their sources can be recorded simultaneously, rather than directly selecting one value as the definitive prompt.

[0243] The knowledge sources can include surgical guidelines, knowledge graph rules, and retrieved research materials; the pending confirmation status indicates that there is a conflict in the current knowledge results, and different sources and their conclusions need to be retained for further confirmation. This process ensures that knowledge conflicts are not merged into a single definitive conclusion and allows the prompt to retain the sources of conflict.

[0244] In some other examples, when candidate navigation suggestions involve high-risk structures such as the renal artery, abdominal aorta, renal vein, inferior vena cava, or collecting system, the suggestion will be prioritized by one level.

[0245] High-risk structures refer to anatomical structures that need to be avoided in the current surgical task or that may pose a high risk if damaged.

[0246] For example, if a candidate navigation cue has an initial priority of medium and involves the renal artery, its priority can be increased to high. This process prioritizes cuees involving high-risk structures over general task cuees during display and interaction.

[0247] Furthermore, when the confidence level is less than 0.7, conservative wording such as "It is recommended to confirm the accurate location of the target structure" or "There may be corresponding risks; please verify in conjunction with the intraoperative field of view" can be used. The thresholds, attenuation coefficients, and structure types mentioned above are all exemplary parameters and can be adjusted according to the type of surgery and the system calibration results.

[0248] In this embodiment, insufficient perception, insufficient registration, knowledge conflict, and high-risk structures are mapped to confidence level, wording, display mode, confirmation status, or priority changes, respectively. This allows the intermediate state of candidate navigation prompts to change in a targeted manner according to the source of risk, thereby reducing the possibility of low-quality evidence generating overly certain target navigation prompts.

[0249] Figure 8 This is a schematic diagram of a closed-loop replanning process based on doctor feedback, provided for some embodiments of this application.

[0250] like Figure 8 As shown, in some embodiments, the target navigation prompt output may further include steps S310 to S350.

[0251] Step S310: Receive feedback information regarding the target navigation prompt.

[0252] For example, the feedback information is the result of the doctor's interaction with the output target navigation prompt, which is used to indicate the doctor's opinion on the corresponding prompt or the current task node status.

[0253] Feedback can be received via voice commands or other interactive methods, and may include confirmation, ignore, postpone, close, and repeat.

[0254] The feedback types and their processing rules are shown in Table 11 below.

[0255] Table 11

[0256] For example, when a doctor provides voice feedback that "the renal artery and renal vein have been identified," this feedback can be identified as confirmation feedback; when a doctor does not accept the spatial prompt in the case of registration failure, this feedback can be identified as ignored feedback.

[0257] Step S320: Update the task context memory based on the feedback information.

[0258] For example, task context memory is structured data used to continuously save the current task node state and interaction history across multiple planning moments. It can record the current stage, current task node, exposed structures, completed operations, recent risk events, output prompts, feedback information, registration quality changes, and key time information. This memory is not ordinary dialogue history, but a structured intraoperative state that can be read by task conditions.

[0259] The computing device modifies the corresponding fields in the task context memory according to the processing rules corresponding to the feedback type, so as to transform a feedback message into a task history state that can continuously affect subsequent planning.

[0260] In some examples, upon receiving confirmation feedback, the completion condition for the current task node can be updated to True and the ignore count can be reset.

[0261] In some examples, when ignored feedback is received, the ignore count for the corresponding prompt can be increased.

[0262] In some examples, when a closure feedback is received, the closure status of the corresponding prompt type can be recorded.

[0263] Step S330: Obtain the updated current task status object.

[0264] For example, the updated current task state object is a structured object obtained by reorganizing the updated task context memory and the intraoperative state acquired at the next moment. It includes the updated task context memory and may also include the perception, registration, spatial risk, and knowledge state at the next moment. In this embodiment, historical state changes caused by feedback and real-time changes in the intraoperative environment can be incorporated into the same round of planning input.

[0265] In some examples, once the doctor confirms that T301 has been completed, the updated current task state object can simultaneously include the T301 completion condition being True and the instrument and anatomical exposure status at the next moment.

[0266] Step S340: Based on the updated current task status object and surgical task graph, redetermine the target task node and generate the updated task-level planning result.

[0267] For example, re-determining the target task node means re-executing the current task node location, completion condition check, and task advancement process so that the target task node reflects the updated task status.

[0268] The computing device re-matches the updated current task state object with the preconditions, completion conditions, and directed task relationships in the surgical task graph, and generates updated task-level planning results according to the matching results, so that the task-level planning results are updated with feedback information and changes in the current intraoperative status.

[0269] In some examples, when the T301 completion condition is updated to True and the preconditions for T303 are met, the target task node can be updated from T301 to T303; when the severity of the bleeding event drops to 0 and T501 is completed, the target task node can be updated to the task node corresponding to the tumor resection task in stage P6.

[0270] Step S350: Adjust subsequent navigation prompts based on the updated task-level planning results.

[0271] For example, subsequent navigation prompts are the next round of navigation prompts generated and output after the current task state object is updated.

[0272] The computing device adjusts the content and output strategy of subsequent navigation prompts based on the target task nodes, prompt types, prompt priorities, and display method suggestions in the updated task-level planning results, so as to implement the updated target task nodes and navigation prompt strategies in the subsequent navigation output.

[0273] In some examples, when the current task node is confirmed to be completed and the prerequisites for the subsequent task node are met, subsequent navigation prompts are generated around the subsequent task node. In some examples, spawning continues around the current task node even if the current task node is not completed.

[0274] In some examples, risk task prompts are output first when the risk priority score is not lower than 0.6.

[0275] In some examples, when the registration quality score drops to less than 0.5, subsequent navigation prompts are forcibly downgraded to text prompts.

[0276] In some examples, when similar navigation tips are ignored at least 3 times consecutively, the frequency and priority of the tips are reduced.

[0277] In some examples, when a duplicate or acknowledgment feedback is received, the completion conditions of the current task node are reassessed.

[0278] In this embodiment, feedback information and changes in the current intraoperative state can be written into the next round of planning input. This allows task completion, risk mitigation, navigation prompt ignoring, and changes in registration quality to affect subsequent target task nodes and navigation prompt strategies, thus forming a continuous closed loop. Feedback information is not merely saved as an interaction record; instead, it first modifies the task context memory, then enters the updated current task state object and influences target task nodes and navigation prompt strategies. This reduces repetitive prompts for completed tasks, outdated prompts for risks mitigated, and repeated output of long-ignored prompts.

[0279] In some embodiments, the laparoscopic surgery navigation task planning system may include a status acquisition module, a task map acquisition module, a task positioning module, a task status determination module, a task advancement module, and a planning result generation module.

[0280] The system comprises the following modules: a status acquisition module for obtaining the current task status object; a task graph acquisition module for obtaining a pre-constructed surgical task graph; a task location module for determining the current task node based on the current task status object and preconditions; a task status determination module for determining the task completion status based on the current task status object and completion conditions; a task progression module for determining the target task node based on the task completion status and directed task relationships; and a planning result generation module for generating task-level planning results based on the current task status object and the target task node.

[0281] In some embodiments, the system may further include a tool execution module, a task template execution module, a safety constraint module, a prompt output module, and a task context memory module. The tool execution module is used to invoke standardized navigation tools according to task-level planning results; the task template execution module is used to match and execute task templates; the safety constraint module is used to verify and adjust candidate navigation prompts; the prompt output module is used to output target navigation prompts; and the task context memory module is used to save task status, prompt records, and feedback results. These modules can be deployed on the same computing device or interact with each other via a message bus, network interface, or inter-process communication.

[0282] The aforementioned system modules sequentially receive the current task status object, task node, task completion status, target task node, and task-level planning result, ensuring that each module in the system claims corresponds to the aforementioned method steps. Since each module transmits intermediate results around the same task status and surgical task graph, the system achieves the same task planning consistency and traceability as the method embodiments.

[0283] The following describes the laparoscopic surgical navigation task planning method provided in the embodiments of this application with specific examples.

[0284] Example 1: Navigation and decision-making throughout the partial nephrectomy procedure.

[0285] This embodiment describes the operation process of the method of this application in robot-assisted laparoscopic partial nephrectomy in the order of "implementation environment - system initialization - renal hilum anatomy time point A - bleeding risk time point B - doctor feedback and replanning".

[0286] I. Implementation Environment.

[0287] The surgical platform utilizes the Agibot surgical robot system, and the endoscope is a 30-degree 3D high-definition laparoscope. The computing platform is an edge computing server, configured with an NVIDIA RTX 4090 GPU, an Intel Xeon W-2400 CPU, and 64GB of RAM, running Ubuntu 22.04 LTS. The data bus employs a publish-subscribe message bus based on ZeroMQ, with the intraoperative perception layer, registration and localization layer, semantic map module, and knowledge graph module each publishing status messages.

[0288] The pre-constructed nephrectomy task graph in this embodiment includes 6 stages, 32 task nodes, and 58 directed edges. The task template library predefines 5 task templates: a renal hilum vessel control prompt template, a tumor margin risk assessment template, a collection system open risk assessment template, a bleeding event handling prompt template, and a spatial overlay reliability judgment template. The aforementioned platform, quantities, and hardware / software configurations are intended to illustrate a feasible environment and do not constitute necessary limitations.

[0289] II. System Initialization.

[0290] Before the operation begins, the computing device loads the task graph and task template library into memory and initializes the task context memory to an empty state: Current stage = null, Current task = null, Output prompts = [], Doctor feedback = [], Registration quality history = [].

[0291] III. Time Node A: 20 minutes after the start of the surgery, during the renal hilum dissection stage.

[0292] First, the computing device receives the intraoperative status from each data source and constructs a current task status object. This object is shown in Table 12 below.

[0293] Table 12

[0294] This step organizes the phase, instruments, anatomical exposure, spatial risks, registration, and mission history obtained at the 20th minute into the current mission status object. Among them, the phase status is used to limit the mission search scope, the instrument status and anatomical exposure status are used to check the mission conditions, the event status is used to determine whether risky mission nodes need to be prioritized, the registration status is used to determine the display method of candidate navigation prompts, and the mission context memory status is used to inherit the completion results of the previous mission node.

[0295] Subsequently, the computing device performs task-level planning.

[0296] The phase transition flag is set to False, therefore phase P3 is maintained. The computing device traverses the task nodes in phase P3, checking the preconditions of each task node. The task node T301 for "Identifying the Renal Artery and Renal Vein" and the task node T303 for "Isolating the Main Renal Artery" both pass the precondition check. Since T301 is earlier in the topological order of the task graph, T301 is determined as the current task node.

[0297] The completion condition for T301 is "Both renal arteries and renal veins have been identified". The current status only indicates that the renal hilum region is visible, but no specific blood vessels have been identified. Therefore, this completion condition is False, and the task completion score is set to 0.5.

[0298] The current observation instrument is a dissection forceps. The recommended instrument set for T301 is a dissection forceps and an ultrasonic scalpel. The instrument status matches the task requirements, and the instrument identification confidence score for the dissection forceps is 0.88. There are currently no active risk events, therefore no risk task will be triggered.

[0299] The structures of interest for T301 are the main renal artery and renal vein. Since distance information for both is currently lacking, it is determined that structural and distance query tools need to be used. Simultaneously, to obtain the anatomical rules of the renal hilum, a knowledge retrieval tool needs to be used. The prompt type is determined to be a task prompt, with an initial priority of medium. Because the registration quality score of 0.82 is greater than 0.7, the recommended display method is spatial overlay and text. The task template was not matched at this time point.

[0300] The task-level planning results are shown in Table 13 below.

[0301] Table 13

[0302] The tool execution process is as follows: The structural query tool returns the location and geometric attributes of the renal artery trunk and renal vein in the semantic map; The distance query tool returned that the tip of the dissecting forceps was approximately 4.8 mm from the main renal artery and approximately 6.2 mm from the renal vein; The knowledge retrieval tool uses a three-channel RAG search to retrieve "renal hilum anatomy RAPN", returning a vascular atlas of the renal hilum region and knowledge that 15% to 25% of patients have accessory renal arteries; The prompt generation tool generates candidate navigation prompts based on the task-level planning results and the results of the aforementioned tools.

[0303] The candidate navigation prompts and the target navigation prompts after security verification are shown in Table 14 below.

[0304] Table 14

[0305] The overall perception quality score is 0.87 (not lower than 0.5), the registration quality score is 0.82 (not lower than 0.6), and the knowledge status indicates no knowledge source conflicts; therefore, the corresponding security checks are passed. Because the candidate navigation prompt involves blood vessels, its priority is increased from medium to high. The target navigation prompt is output to the enhanced display layer.

[0306] The doctor then confirmed via voice feedback that "the renal artery and renal vein have been identified." The computing device updated the completion condition of T301, "both the renal artery and renal vein have been identified," to True. In the next round of task planning, T301 was identified as the completed current task node. The prerequisite for its successor task node, T303, "free renal artery trunk," has been met, therefore T303 was identified as the new target task node.

[0307] IV. Time Node B: 50 minutes after the start of the surgery, during the tumor resection stage, active bleeding suddenly occurs.

[0308] The current task state object constructed by the computing device is shown in Table 15 below.

[0309] Table 15

[0310] The risk priority score for the bleeding event is: RiskScore_Bleeding = 0.7 × 1.0 = 0.7.

[0311] Since the score is not lower than 0.6, the computing device triggers the risk task node T501 "Intraoperative Bleeding Management" and postpones the routine tumor resection task node in stage P6. Based on the focus of the risk task node, it is determined that the intraoperative awareness query tool, risk assessment tool, knowledge retrieval tool, and prompt generation tool need to be invoked. The prompt type is risk prompt, and the prompt priority is high. The registration quality score is 0.65, which is greater than 0.5 but less than 0.7. Therefore, the recommended display method is text and icons, avoiding reliance on precise spatial overlay.

[0312] The task-level planning results are shown in Table 16 below.

[0313] Table 16

[0314] The candidate navigation suggestions generated based on the task-level planning results are shown in Table 17 below.

[0315] Table 17

[0316] Because the overall perception quality score of 0.48 is less than 0.5, the perception quality downgrade rule is triggered. The prompt type of the candidate navigation prompt is adjusted to a downgrade prompt, and the prompt confidence is multiplied by 0.6 to 0.51. The prompt content is updated to "Due to the current field of vision being affected by bleeding / smoke, the reliability of perception is reduced. Please judge in conjunction with the actual situation during the operation." The target navigation prompt is output as text and a red flashing icon. This processing retains the risk reminder of active bleeding while simultaneously indicating that the current perception result may be unstable.

[0317] After the doctor completes the bleeding management, they provide voice feedback stating "Bleeding under control." The computing device then updates the task context memory, reducing the severity of the bleeding event to 0 and marking the risk task node T501 as completed. Following replanning, the risk task node is removed, and the target task node is updated to the task node corresponding to the tumor resection task in phase P6.

[0318] As seen from time nodes A and B, this embodiment can determine the current task node based on preconditions and task order under normal surgical conditions, and advance along the directed task relationship after the current task node is completed. When active bleeding occurs, risk priority can be used to prioritize risk task nodes over normal task nodes, and the certainty of target navigation prompts can be reduced based on perception quality. Each task node selection, navigation prompt generation, and task recovery can be mapped to the current task state object, task conditions, and surgical task graph path, thereby enabling similar intraoperative states to obtain task-level planning results according to unified rules, while retaining the planning basis and progress process.

[0319] Example 2: Security degradation and closed-loop reprogramming under registration failure conditions.

[0320] This example illustrates how to adjust navigation prompts and restore the display mode after registration is restored when tissue deformation causes a significant decrease in the registration quality between the preoperative model and the intraoperative field of view.

[0321] The surgery progressed to stage P4, the stage of exposing the kidney and tumor area. After the kidney was fully freed, it shifted by more than 10 mm relative to its preoperative CT position, and the registration quality score plummeted from 0.85 to 0.38.

[0322] Registration failure refers to a situation where the spatial correspondence between the preoperative model and the actual intraoperative anatomy deteriorates to the point where it can no longer reliably support the original display method due to tissue displacement or deformation. In this embodiment, this state is represented by three fields: registration quality score, spatial deviation, and whether spatial overlay is allowed.

[0323] When constructing the current task status object, the registration status is: registration quality score = 0.38, spatial deviation = 11mm, and allowable spatial overlap = False.

[0324] Because Q_registration=0.38 is less than 0.5, a registration quality degradation is triggered: Change the current display mode of the prompt from "3D highlight + text" to "text"; The following message will be added to the end: "Note: The current registration accuracy is insufficient (deviation of approximately 11 mm). Spatial information is for reference only. Please refer to the actual anatomical dissection during the operation." Change the prompt type to a downgrade prompt.

[0325] The adjusted target navigation prompts are shown in Table 18 below.

[0326] Table 18

[0327] The physician ignores the target navigation prompt and chooses to continue based on the endoscopic view. The computing device records the ignored feedback, maintains the current stage and task node, but continuously suggests setting the display mode to text in subsequent task-level planning results. If the registration quality remains below 0.5 for more than 5 minutes, the computing device generates a "re-registration recommended" navigation prompt.

[0328] After the surgeon completed the critical operation and repositioned the kidney, the registration quality score recovered to 0.78. The computing device acquired the updated registration status in the next round, confirmed that the registration quality score was not lower than 0.7, and then restored the spatial overlay display capability. Thus, the display method dynamically adjusts as the registration quality decreases and recovers, and maintains continuity across multiple planning rounds through task context memory.

[0329] In this embodiment, when the registration quality drops from 0.85 to 0.38, the target navigation prompt is downgraded from spatial overlay to text with additional spatial information usage restrictions; after the registration quality recovers to 0.78, the system restores spatial overlay. This process ensures that the display method of the target navigation prompt corresponds to the current registration reliability level, and that both downgrading and recovery have clear registration status criteria, thereby reducing the risk of outputting overly definitive spatial prompts when registration fails.

[0330] Figure 9 This is a schematic diagram of a computing device provided for some embodiments of this application.

[0331] like Figure 9 As shown, the computing device 900 includes a processor 901 and a memory 902. Exemplarily, the computing device 900 may also include a communications interface 903 and a communications bus 904.

[0332] The processor 901, memory 902, and communication interface 903 communicate with each other via communication bus 904. The communication interface 903 may include a transmitter and receiver for communicating with other devices or communication networks, and may be a wired interface (port), such as a fiber distributed data interface (FDDI) or a gigabit Ethernet interface (GE).

[0333] In some embodiments, the processor 901 is used to execute program 905, specifically performing the relevant steps in the above method embodiments. Exemplarily, program 905 may include program code, which includes computer-executable instructions.

[0334] For example, processor 901 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement some embodiments of this application. Computing device 900 may include one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs. The CPU may be a single-core CPU or a multi-core CPU.

[0335] In some embodiments, memory 902 is used to store program 905. Memory 902 may include high-speed random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device.

[0336] Specifically, program 905 can be invoked by processor 901 and executed by computing device 900 using the above method.

[0337] Some embodiments of this application provide a computer-readable storage medium storing at least one executable instruction that, when executed on a computing device 900, causes the computing device 900 to perform the above-described method.

[0338] For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device.

[0339] This application provides a chip system in several embodiments, which is applied to a server. The chip system includes one or more interface circuits and one or more processors. The interface circuits and processors are interconnected via lines. The interface circuits are used to receive signals from the server's memory and send signals to the processors, the signals including computer instructions stored in the memory. When the server processor executes the computer instructions, the server performs the steps of the methods shown in the above-described method embodiments.

[0340] The beneficial effects that the readable storage medium provided in some embodiments of this application can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0341] It should be noted that, in this application, terms such as "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0342] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0343] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0344] In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc. The above embodiments are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the technical solutions of this application should be included within the scope of protection of this application.

Claims

1. A method for planning navigation tasks in laparoscopic surgery, characterized in that, include: Obtain the current task status object of the laparoscopic surgery, wherein the current task status object is structured data representing the current intraoperative status of the laparoscopic surgery; Obtain a pre-constructed surgical task graph, which is a directed acyclic graph including multiple task nodes and directed task relationships with transition conditions between the multiple task nodes; wherein, each task node corresponds to a surgical task and is configured with prerequisites and completion conditions. Based on the current task status object, match the preconditions of each of the multiple task nodes, and determine the current task node in the surgical task graph; Based on the current task status object and the completion conditions of the current task node, determine the task completion status of the current task node; Based on the task completion status and the directed task relationship, the target task node is determined in the surgical task graph; Based on the current task status object and the target task node, a task-level planning result corresponding to the target task node is generated. The task-level planning result is used to indicate the current execution task and navigation prompt strategy of laparoscopic surgery navigation.

2. The laparoscopic surgery navigation task planning method according to claim 1, characterized in that, Based on the current task state object, the current task node is determined in the surgical task graph by matching the preconditions of each of the multiple task nodes, including: The current surgical stage is determined based on the current task status object; Multiple candidate task nodes corresponding to the current surgical stage are determined from the surgical task graph; The current task node is determined from the multiple candidate task nodes based on the current task status object and the respective prerequisites of the multiple candidate task nodes.

3. The laparoscopic surgery navigation task planning method according to claim 2, characterized in that, Determining the current task node from the plurality of candidate task nodes includes: If the current task status object indicates the existence of at least one active risk event, a risk priority score is determined for each of the active risk events. The risk task node corresponding to the active risk event whose risk priority score meets the preset risk triggering condition is determined as the candidate risk task node, and the candidate risk task node with the highest risk priority score is determined as the current task node; if the current task status object indicates that there is no active risk event, the current task node is determined according to the task order of the multiple candidate task nodes in the surgical task graph.

4. The laparoscopic surgery navigation task planning method according to claim 1, characterized in that, Based on the task completion status and the directed task relationship, the target task node is determined in the surgical task graph, including: If the task completion status indicates that the current task node is not completed, the current task node is determined as the target task node; When the task completion status indicates that the current task node has been completed, the successor task node of the current task node is determined according to the directed task relationship, and the target task node is determined from the successor task nodes according to the current task status object and the preconditions of the successor task node.

5. The laparoscopic surgical navigation task planning method according to any one of claims 1 to 4, characterized in that, The method further includes: Navigation processing is performed based on the task-level planning results to obtain the navigation processing results; Candidate navigation prompts are generated based on the task-level planning results and the navigation processing results; The candidate navigation prompts are subjected to security verification based on the current task status object, and the candidate navigation prompts are adjusted according to the security verification results to obtain the target navigation prompts; Output the target navigation prompt.

6. The laparoscopic surgery navigation task planning method according to claim 5, characterized in that, Navigation processing is performed based on the task-level planning results to obtain navigation processing results, including: Determine tool invocation requirements based on the task-level planning results; Based on the tool invocation requirements, the corresponding standardized navigation tool is invoked through the tool execution module; Obtain the tool execution result returned by the standardized navigation tool, and determine the tool execution result as the navigation processing result; The standardized navigation tools include at least one of the following: structural query tool, distance query tool, risk assessment tool, intraoperative perception query tool, registration quality query tool, knowledge retrieval tool, and prompt generation tool.

7. The laparoscopic surgery navigation task planning method according to claim 6, characterized in that, Based on the task-level planning results, navigation processing is performed to obtain navigation processing results, which also include: Based on the current task status object and the task-level planning result, a matching task template is determined from the pre-built task template library; Based on the task template, obtain template input data from the current task status object, and call the corresponding standardized navigation tools according to the tool call order defined by the task template; According to the result merging rules defined by the task template, the tool execution results returned by each standardized navigation tool are merged to obtain the task template execution result; The execution result of the task template is determined as the navigation processing result; The task template includes at least one of the following: triggering conditions, input data, tool call order, parameter mapping rules, result merging rules, security constraints, and output format.

8. The laparoscopic surgery navigation task planning method according to claim 5, characterized in that, Performing a security check on the candidate navigation prompts based on the current task status object, and adjusting the candidate navigation prompts based on the security check result, including: Based on at least one of the perception quality, registration quality, and knowledge status in the current task status object, the candidate navigation prompt is subjected to security verification. Based on the security verification results, at least one of the following: prompt type, prompt priority, display method, prompt confidence level, and prompt content of the candidate navigation prompts is adjusted.

9. The laparoscopic surgery navigation task planning method according to claim 8, characterized in that, Adjusting the candidate navigation prompts based on the security verification results includes at least one of the following: If the perceived quality does not meet the preset perceived quality conditions, the confidence level of the candidate navigation prompt is reduced, and the prompt content of the candidate navigation prompt is adjusted to uncertain prompt content. If the registration quality does not meet the preset registration quality conditions, the display mode of the candidate navigation prompt will be adjusted from spatial display mode to non-spatial display mode. If the knowledge status indicates that multiple knowledge sources conflict, the candidate navigation prompt will be marked as pending confirmation. If the candidate navigation prompt involves a preset high-risk structure, the prompt priority of the candidate navigation prompt shall be increased.

10. The laparoscopic surgery navigation task planning method according to claim 5, characterized in that, The method further includes: Receive feedback information regarding the target navigation prompts; Update the task context memory based on the feedback information; Obtain the updated current task state object, which includes the updated task context memory; Based on the updated current task status object and the surgical task graph, the target task node is redefined, and an updated task-level planning result is generated. Adjust subsequent navigation prompts based on the updated task-level planning results.

11. A laparoscopic surgery navigation task planning system, characterized in that, include: The status acquisition module is used to acquire the current task status object of the laparoscopic surgery, wherein the current task status object is structured data that represents the current intraoperative status of the laparoscopic surgery. The task graph acquisition module is used to acquire a pre-constructed surgical task graph. The surgical task graph is a directed acyclic graph, which includes multiple task nodes and directed task relationships with transition conditions between the multiple task nodes. Each task node corresponds to a surgical task and is configured with prerequisites and completion conditions. The task location module is used to determine the current task node in the surgical task graph by matching the preconditions of each of the multiple task nodes according to the current task status object. The task status determination module is used to determine the task completion status of the current task node based on the current task status object and the completion conditions of the current task node. The task advancement module is used to determine the target task node in the surgical task graph based on the task completion status and the directed task relationship. The planning result generation module is used to generate a task-level planning result corresponding to the target task node based on the current task status object and the target task node. The task-level planning result is used to indicate the current execution task and navigation prompt strategy of laparoscopic surgery navigation.

12. A computing device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, implements the laparoscopic surgical navigation task planning method according to any one of claims 1 to 10.

Images