Anthropomorphic robotically-enabled natural orifice surgical system and method

Abstract

The application provides a humanoid embodied intelligent natural orifice surgery robot system and a control method thereof. The system is constructed based on a perception-cognition-decision-execution-feedback embodied intelligent closed loop architecture, comprising: an intelligent assessment module for disease assessment and natural orifice surgery indication determination of a patient; a surgery planning module for reconstructing a three-dimensional path model of a target natural lumen and generating a personalized surgery planning scheme; a humanoid natural orifice execution module for controlling a flexible surgical instrument to reach a target site through a natural lumen of a human body and perform a surgical operation; and an auxiliary rehabilitation module for generating a personalized postoperative rehabilitation scheme. In this way, the travel strategy and operation scheme are autonomously adjusted according to the real-time environmental changes in the lumen, and the preoperative assessment and planning and postoperative rehabilitation management are integrated.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and more specifically, to a humanoid intelligent transnatural cavity surgical robot system and method. Background Technology

[0002] Natural orifice surgery is a minimally invasive surgical procedure that enters the body through natural orifices (such as the mouth, nose, anus, urethra, etc.) for diagnosis and treatment. Compared with traditional open surgery and percutaneous minimally invasive surgery, it has significant advantages such as less trauma, faster recovery, and minimal postoperative scarring. It has been widely used in clinical applications in the fields of digestive endoscopy, bronchoscopy, and urological endoscopy.

[0003] Existing flexible endoscopic assistive devices are still limited to master-slave remote operation, unable to autonomously adjust their travel strategies and operating plans according to real-time changes in the intraluminal environment, and even less able to integrate preoperative assessment and planning with postoperative rehabilitation management. Therefore, there is an urgent need for a new type of surgical robot system that can combine humanoid intelligent technology with natural orifice surgery. Summary of the Invention

[0004] To address the aforementioned problems, the first aspect of this application provides a humanoid embodied intelligent transoral surgical robot system. The system is constructed based on an embodied intelligent closed-loop architecture of perception-cognition-decision-execution-feedback, comprising: The intelligent assessment module is used to conduct targeted disease assessments and determine the indications for transoral orifice surgery based on the patient's medical data and vital signs, generate personalized auxiliary assessment results for the patient, and display them. The surgical planning module is used to reconstruct a three-dimensional path model of the target natural cavity based on the patient's medical imaging data, perform targeted transoral surgical planning for the patient based on the three-dimensional path model, generate a personalized surgical planning scheme, and display it. The humanoid natural cavity execution module is used to control flexible surgical instruments to reach the target site through the human body's natural cavities according to the surgical plan, acquire surgical field images through intracavitary visual perception, and perform surgical operations. The auxiliary rehabilitation module is used to generate and display personalized postoperative rehabilitation plans for patients based on their medical data and postoperative assessment results.

[0005] A second aspect of this application provides a control method for a humanoid intelligent transoral surgical robot system, applied to the aforementioned humanoid intelligent transoral surgical robot system, comprising: Based on the patient's medical and physical examination data, targeted disease assessments and indications for transoral orifice surgery are determined, generating and displaying personalized auxiliary assessment results. Based on the patient's medical imaging data, a three-dimensional path model of the target natural cavity is reconstructed. Based on the three-dimensional path model, a targeted surgical plan is made for the patient through the natural cavity, generating a personalized surgical plan, which is then displayed. According to the surgical plan, the flexible surgical instruments are controlled to reach the target site through the body's natural cavities, and the surgical field image is obtained through visual perception within the cavity and the surgical operation is performed. Based on the patient's medical data and postoperative assessment results, a personalized postoperative rehabilitation plan is generated and displayed.

[0006] A third aspect of this application provides an electronic device comprising: a memory and a processor; the memory for storing a program; and the processor, coupled to the memory, for executing the program to implement the control method of the humanoid intelligent transoral surgical robot system described above.

[0007] The fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the control method of the humanoid intelligent transoral surgical robot system described above.

[0008] The humanoid intelligent transoral surgical robot system provided in this application has the following beneficial effects: It can autonomously adjust its travel strategy and operation plan based on real-time changes in the intraluminal environment, and can integrate preoperative assessment planning and postoperative rehabilitation management.

[0009] By adopting an embodied intelligent closed-loop architecture of perception-cognition-decision-execution-feedback, the transoral surgical robot is upgraded from a passive execution tool to an intelligent surgical system with autonomous decision-making capabilities. It can autonomously adjust its movement strategy and operation plan according to the real-time environmental changes within the cavity, reducing its dependence on the surgeon's personal operation skills.

[0010] By docking the humanoid robot body with the operating table, eliminating the robot's degrees of freedom of movement, and suppressing environmental vibrations, a stable operating base is provided for the flexible surgical arm to perform precise operations within the cavity, effectively isolating external environmental interference from affecting surgical precision. A cavity creation module establishes and maintains the cavity space required for surgical operations within the patient's body cavity, providing ample operating space for the flexible surgical arm.

[0011] By using a humanoid dual-arm surgical endoscope structure with a control arm and a push arm on the humanoid torso, the dexterous hand at the end of the control arm manipulates the flexible continuum handle, and the dexterous hand at the end of the push arm pushes and retracts the flexible continuum, the coordinated hand movements of a surgeon are simulated, and precise control of the flexible surgical arm is achieved.

[0012] Through the electromagnetic positioning, real-time registration of preoperative models, and intelligent guidance of bifurcation within the lumen of the intraluminal navigation module, the flexible surgical arm achieves precise positioning and path guidance within the lumen, effectively solving the problem of lack of three-dimensional navigation in traditional natural orifice surgeries.

[0013] By using VR / AR interactive modules to build a virtual luminal environment before surgery for doctors to rehearse the surgical path, and by overlaying navigation information, lesion identification results and planned paths onto the surgical field image in an augmented reality manner during surgery, the visualization level and operational accuracy of the surgery are significantly improved.

[0014] By autonomously performing cognitive understanding and decision-making reasoning on multimodal sensory information within the lumen through the embodied intelligent decision-making module, the system generates travel and operation strategies and autonomously iterates and corrects them based on execution feedback, thus realizing intelligent autonomous decision-making for transluminal cavity surgery.

[0015] The safety monitoring module monitors the contact force between the flexible surgical arm and the lumen wall and the bending curvature of the flexible arm in real time. When the monitored parameters exceed the preset safety range, a graded safety response is executed to ensure the safety of intraluminal operations. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below.

[0017] Figure 1 This is a schematic diagram of the overall architecture of the humanoid intelligent transoral surgical robot system provided in the embodiments of this application; Figure 2 A schematic diagram of the humanoid robot body provided in the embodiments of this application; Figure 3 This is a schematic diagram of the humanoid double-arm manipulation mirror structure provided in the embodiments of this application; Figure 4 This is a schematic diagram of the intraluminal navigation module architecture provided in an embodiment of this application; Figure 5 A schematic diagram of the architecture of the embodied intelligent decision-making module provided in the embodiments of this application; Figure 6 This is a schematic diagram of the security monitoring module architecture provided in an embodiment of this application; Figure 7 A flowchart illustrating the control method for a humanoid intelligent transoral surgical robot system provided in this application embodiment; Figure 8 This is a schematic diagram of an electronic device architecture provided in an embodiment of this application. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of this application more apparent and understandable, specific embodiments of this application will be described in detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0019] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application shall have the ordinary meaning as understood by one of ordinary skill in the art to which this application pertains.

[0020] like Figure 1 As shown, this application provides a humanoid embodied intelligent transoral surgical robot system. The system is built upon an embodied intelligent closed-loop architecture of perception-cognition-decision-execution-feedback, and includes: The intelligent assessment module is used to conduct targeted disease assessments and determine the indications for transoral orifice surgery based on the patient's medical data and vital signs, generate personalized auxiliary assessment results for the patient, and display them. The surgical planning module is used to reconstruct a three-dimensional path model of the target natural cavity based on the patient's medical imaging data, perform targeted transoral surgical planning for the patient based on the three-dimensional path model, generate a personalized surgical planning scheme, and display it. The humanoid natural cavity execution module is used to control flexible surgical instruments to reach the target site through the human body's natural cavities according to the surgical plan, acquire surgical field images through intracavitary visual perception, and perform surgical operations. The auxiliary rehabilitation module is used to generate and display personalized postoperative rehabilitation plans for patients based on their medical data and postoperative assessment results.

[0021] Specifically, the embodied intelligent closed-loop architecture is the core architecture that upgrades the transoral surgical robot from a tool that passively executes the doctor's control instructions to an intelligent agent with the ability to perceive the environment, understand the cognition, make autonomous decisions, execute precisely, and provide feedback on the results.

[0022] The design philosophy of this architecture is based on the theory of embodied intelligence, which states that intelligent agents must acquire and utilize knowledge through direct interaction with the physical environment.

[0023] In the scenario of surgery through natural cavities, the physical environment is the complex and curved natural cavities of the human body. The intelligent agent directly interacts with the cavity environment through a flexible surgical arm, perceives visual, force and positional information within the cavity, and after cognitive reasoning and decision planning, controls the flexible surgical arm to perform corresponding movement and operation actions, and continuously corrects and optimizes the operation strategy through a feedback mechanism.

[0024] The effect of this closed-loop architecture is that it enables the transoral surgical robot to autonomously navigate, operate, and respond to emergencies in complex cavitary environments.

[0025] Specifically, the intelligent assessment module is the system's preoperative assessment entry point, responsible for comprehensively analyzing the patient's multi-dimensional medical data to determine whether the patient is suitable for natural orifice surgery.

[0026] The medical data includes, but is not limited to, the patient's medical history, laboratory test results, imaging data, and endoscopic examination reports.

[0027] The vital signs data include the patient's vital signs, organ function indicators, and assessment results of the lumen mucosa status.

[0028] The intelligent assessment module uses a multimodal basic model to fuse and analyze the above-mentioned multi-source data and perform cognitive reasoning to generate personalized auxiliary assessment results for natural orifice surgery. These assessment results include not only disease diagnosis and staging, but also assessment conclusions on lumen traversability, lesion accessibility, and lumen operation risk level.

[0029] The effect of this feature is that it provides a scientific and comprehensive preoperative assessment basis for subsequent surgical planning, avoiding intraoperative accidents caused by insufficient assessment.

[0030] Specifically, the surgical planning module is responsible for constructing a three-dimensional path model of the target natural lumen based on the patient's medical imaging data, and formulating a detailed surgical planning scheme based on this model.

[0031] This module first uses the patient's CT image data to construct a complete three-dimensional path model of the target natural lumen through a three-dimensional reconstruction algorithm. This model accurately reflects the lumen's course, bifurcation structure, diameter changes, and degree of curvature.

[0032] Then, path planning, equipment selection, and operation strategy design are carried out on the three-dimensional path model.

[0033] The personalization of the surgical planning scheme is essentially achieved by generating surgical parameters autonomously based on the patient's individual luminal anatomy characteristics using a multimodal basic model, rather than using a preset universal parameter template.

[0034] The effect of this feature is that it provides detailed path guidance and operational instructions for the precise movement of the flexible surgical arm during surgery, significantly improving the predictability and safety of the surgery.

[0035] Specifically, the humanoid natural cavity execution module is responsible for controlling the flexible surgical instruments to enter through the body's natural cavities and reach the target site to perform surgical operations.

[0036] This module includes a humanoid robot body and a flexible surgical system mounted on it.

[0037] The humanoid robot can autonomously move to the surgical position and dock with the operating table. After docking, the robot eliminates the degrees of freedom of movement through a degree-of-freedom locking mechanism and suppresses vibration interference from the surgical environment through an active vibration isolation device, providing a stable operating base for the flexible surgical arm to perform precise operations within the cavity.

[0038] The flexible surgical system includes a flexible surgical arm module and an intraluminal navigation module. During the operation, the flexible surgical arm is inserted through the natural cavity entrance and travels along the planned path to the target site under the guidance of the intraluminal navigation module. The surgical field image is acquired through intraluminal visual perception, and the surgical operation is performed by the end effector.

[0039] The effect of this feature is that it enables precise navigation and operation of flexible surgical instruments within complex, curved lumens.

[0040] Specifically, the auxiliary rehabilitation module is responsible for generating personalized postoperative rehabilitation plans based on the patient's medical data and postoperative assessment results.

[0041] This module comprehensively considers factors such as surgical method, degree of lumen damage, and individual patient differences. It uses a multimodal basic model for reasoning and analysis to generate personalized rehabilitation plans that include dietary management, activity guidance, medication regimens, and follow-up plans.

[0042] The effect of this feature is that it enables closed-loop management of the entire process from preoperative assessment to postoperative rehabilitation, avoiding information gaps between postoperative rehabilitation and intraoperative treatment.

[0043] The beneficial effects of the aforementioned humanoid embodied intelligent transluminal cavity surgical robot system are as follows: through the embodied intelligent closed-loop architecture of perception-cognition-decision-execution-feedback, the transluminal cavity surgical robot possesses autonomous decision-making and adaptive adjustment capabilities; through the intelligent assessment module, it achieves multi-dimensional preoperative indication determination; through the surgical planning module, it achieves personalized surgical planning based on a three-dimensional path model; through the humanoid transluminal cavity execution module, it achieves precise navigation and operation of flexible surgical instruments within the cavity; and through the auxiliary rehabilitation module, it achieves personalized postoperative rehabilitation management.

[0044] In a preferred embodiment, the humanoid transoral cavity execution module includes: The humanoid robot body is used to dock with the operating table berth, and after docking is completed, the body's motion degrees of freedom are eliminated and environmental vibrations are suppressed; The cavity creation module is used to create and maintain the cavity space required for surgical operations within the patient's body cavity; The flexible surgical arm module, mounted on the humanoid robot body, is driven by a continuum mechanism and is used to acquire surgical field images within the lumen, insert through a natural cavity entrance and travel along the lumen path to the target site, and perform surgical operations within the cavity space.

[0045] Specifically, the humanoid robot body adopts a humanoid robot structure design, including an autonomous mobile chassis, a humanoid torso, and an operating arm system mounted on the torso.

[0046] The autonomous mobile chassis integrates omnidirectional wheels and a lidar navigation system, enabling the humanoid robot to navigate and move autonomously within the operating room to reach the target parking position next to the operating table.

[0047] The docking mechanism is located at the connection between the chassis and the torso. It adopts a combination design of precision guide pins and locking buckles. After the robot reaches the target position, the guide pin of the docking mechanism is precisely inserted into and locked with the pre-set docking hole on the side of the operating table, realizing a rigid connection between the humanoid robot body and the operating table.

[0048] After docking at the berth, the degree-of-freedom elimination mechanism locks the omnidirectional wheel assembly and the degrees of freedom of each joint of the autonomous mobile chassis through electromagnetic brakes, so that the humanoid robot body changes from a mobile state to a fixed state, eliminating positional drift caused by chassis micro-movement or joint loosening.

[0049] The vibration suppression mechanism adopts a design that combines an active vibration isolation platform and a passive damper. The active vibration isolation platform senses vibration signals from the ground and the operating table in real time through an accelerometer and generates reverse vibration through a piezoelectric actuator to actively dampen vibrations; the passive damper absorbs high-frequency vibration components.

[0050] The effect of this feature is that it eliminates interference from the body's movement and environmental vibrations for the precise operation of the flexible surgical arm within the lumen, enabling the positioning accuracy of the surgical arm's end to reach the sub-millimeter level.

[0051] Specifically, the cavity creation module is responsible for creating and maintaining the cavity space required for surgical operations within the patient's body cavity.

[0052] After entering the body cavity via natural orifice surgery, sufficient space needs to be created in the target surgical area to allow the flexible surgical arm to unfold and operate.

[0053] The cavity creation module includes an inflation / expansion device, a pressure monitoring and regulation unit, and a cavity space maintenance unit.

[0054] The inflation / expansion device injects CO2 gas or other medical gases into the target body cavity through an inflation tube inserted through a natural cavity, causing the collapsed body cavity to expand and form an operating space.

[0055] The pressure monitoring and regulation unit monitors the pressure inside the cavity in real time through a pressure sensor located at the far end of the inflation tube, and precisely regulates the inflation flow rate through a proportional valve to maintain the pressure inside the cavity within a preset safe range (usually 8 to 15 mmHg), thus avoiding gas embolism or tissue damage caused by excessive pressure.

[0056] The cavity space maintenance unit continuously monitors and maintains the stability of the cavity space during surgery, and automatically replenishes gas when gas leakage or pressure drop is detected.

[0057] The effect of this feature is that it provides a stable and safe cavity space for the operation of flexible surgical arms within the body cavity.

[0058] Specifically, the flexible surgical arm module is mounted on the manipulator arm of the humanoid robot body, driven by a continuum mechanism, and is the execution component for performing surgical operations through natural cavities. For example... Figure 2 As shown, it is a humanoid robot body with a standard interface, and the flexible surgical arm module is set on the operating arm of the humanoid robot body through the standard interface.

[0059] The continuum mechanism achieves multi-degree-of-freedom bending motion through the coordinated push and pull of multiple superelastic nickel-titanium alloy rods, enabling the flexible surgical arm to move flexibly within the curved cavity like a snake.

[0060] The outer diameter of the ultra-fine flexible body of the flexible surgical arm is controlled to an extremely small size, ensuring that it can pass through various natural cavities in the human body.

[0061] The distal end integrates a miniature image sensor and an LED illumination source to acquire surgical field images within the lumen.

[0062] The distal end is also equipped with replaceable end effectors (such as biopsy forceps, ablation electrodes, injection needles, etc.) for performing surgical procedures within the cavity space.

[0063] The proximal drive unit controls the pushing and pulling of each nickel-titanium alloy rod through a precision motor, enabling precise bending and directional movement of the flexible surgical arm.

[0064] The effect of this feature is that it enables the surgical arm to pass through complex, curved cavities that are inaccessible to traditional rigid instruments, and to perform a variety of surgical procedures once it reaches the target site.

[0065] The beneficial effects of the aforementioned humanoid robot body and flexible surgical arm structure are as follows: the docking mechanism enables precise connection between the humanoid robot body and the operating table; the degree-of-freedom elimination mechanism and vibration suppression mechanism provide a highly stable operating base for precision surgical operations; the cavity establishment module provides the necessary operating space for intracavitary surgical operations; and the flexible surgical arm driven by the continuum mechanism enables flexible movement within the cavity, acquisition of surgical field images, and execution of surgical operations.

[0066] like Figure 3 As shown, in a preferred embodiment, the humanoid transoral cavity execution module includes a control module, a humanoid torso, a manipulator arm, and a pusher arm; the manipulator arm and the pusher arm are mounted on the humanoid torso; the manipulator arm is equipped with a dexterous hand at its end for manipulating a flexible continuum handle, and the pusher arm is equipped with a dexterous hand at its end for pushing and retracting the flexible continuum to simulate the coordinated hand movements of a surgeon; the control module generates control commands based on the acquired surgical field image to control the manipulator arm and the pusher arm to perform surgical actions.

[0067] Specifically, the humanoid torso is the main structural framework of the humanoid transoral cavity execution module, and its design concept is to accurately simulate the body structure and operating posture of a human endoscopic surgeon.

[0068] The humanoid torso includes a height adjustment mechanism and a posture adjustment mechanism. The height adjustment mechanism uses an electric lifting column design, which can continuously adjust the height position of the torso within a preset height range to accommodate operating tables of different heights and patients in different positions.

[0069] The posture adjustment mechanism is located at the waist of the torso, allowing the torso to perform a certain range of forward, backward, and lateral movements to adjust the angle of the control arm and push arm relative to the natural cavity entrance, ensuring the optimal operating posture.

[0070] The anthropomorphic torso integrates a main control computing unit and a power distribution system for each joint actuator.

[0071] Specifically, the control arm is located on one side of the humanoid torso, simulating the operating hand used by a human endoscopist to manipulate the flexible endoscope handle.

[0072] The control arm adopts a multi-joint serial structure, including the shoulder joint (3 rotational degrees of freedom), elbow joint (1 rotational degree of freedom), and wrist joint (3 rotational degrees of freedom), for a total of 7 degrees of freedom, giving the control arm motion flexibility comparable to that of the human arm.

[0073] The end of the control arm is equipped with a multi-fingered humanoid dexterous hand, which includes a thumb and at least two pairs of fingers. Each finger contains two to three joint degrees of freedom, enabling precise grasping, pinching, and rotation.

[0074] The dexterous hand is used to manipulate the various control knobs and levers on the flexible continuum handle. By rotating the knobs, the bending direction and angle of the distal end of the flexible surgical arm are controlled, and by moving the levers, the opening and closing action of the distal end effector is controlled.

[0075] The joint drive of the control arm uses a high-precision servo motor in conjunction with a harmonic reducer to ensure the accuracy and smoothness of the control movements.

[0076] The effect of this feature is that it enables the robot to precisely manipulate the flexible continuum handle like a human surgeon, achieving precise control over the bending direction and angle of the flexible surgical arm and the movement of the end effector.

[0077] Specifically, the push arm is located on the other side of the humanoid torso, simulating the auxiliary hand used by human endoscopists to push and retract the flexible endoscope.

[0078] The push arm also adopts a multi-joint serial structure and has the same 7 degrees of freedom of motion as the control arm.

[0079] The end of the push arm is also equipped with a multi-fingered humanoid dexterous hand, which is used to grasp the proximal outer sheath of the flexible continuum. The gripping force of the dexterous hand controls the frictional gripping force on the outer sheath of the flexible continuum. The push arm moves back and forth linearly to realize the pushing (forward) and retraction (backward) operations of the flexible continuum in the lumen. The axial rotation of the flexible continuum can also be realized through the rotational action of the dexterous hand.

[0080] The pusher arm is also equipped with a force sensor to monitor the axial force and torque during the push process in real time. When an abnormal increase in push resistance is detected (which may indicate lumen blockage or instrument jamming), the pusher will automatically stop and report to the control module.

[0081] The effect of this feature is that it enables the robot to precisely control the propulsion distance, propulsion speed, and propulsion direction of the flexible continuum within the cavity.

[0082] Specifically, the dual-arm coordinated configuration of the control arm and the push arm simulates the two-handed coordinated operation mode of a human endoscopic surgeon performing transoral surgery.

[0083] When operating a flexible endoscope, human endoscopists typically hold the handle with their left hand to control the bending direction and the end instrument, while holding the endoscope sheath with their right hand to push and pull it back.

[0084] The humanoid dual-arm manipulation endoscope structure of this application accurately reproduces this two-handed collaborative mode: the control arm corresponds to the surgeon's control hand and is responsible for bending control and instrument manipulation on the flexible continuum handle; the push arm corresponds to the surgeon's push hand and is responsible for the advancement / retraction and axial rotation of the flexible continuum.

[0085] The two arms coordinate their movements in real time through a control module, ensuring the timing of the control and pushing actions.

[0086] For example, when passing through a curved section of the tube, the control arm first adjusts the bending direction so that the distal end is aligned with the direction of the tube after the bend, and then the push arm propels the flexible continuum through the curved section at an appropriate speed. The two actions are precisely coordinated to reduce unnecessary contact with the tube wall.

[0087] The effect of this feature is that it enables multi-dimensional and precise control of the flexible surgical arm through the coordinated operation of both arms, and its control precision and coordination surpass the manual operation level of human surgeons.

[0088] Specifically, the control module is responsible for generating control commands based on the acquired surgical field images and other sensory information to control the manipulator and pusher arms to perform surgical actions.

[0089] The control module receives real-time surgical field images from the intraluminal vision module, three-dimensional positioning data from the intraluminal navigation module, and feedback data from each joint and force sensor. Through visual servoing and motion planning algorithms, it calculates the target angles and velocities of each joint of the manipulator and pusher arm in real time, and issues control commands to drive the two arms to perform coordinated surgical actions.

[0090] The control module also interfaces with the embodied intelligent decision-making module, receiving high-level movement and operation strategies generated by the decision-making module and converting them into specific bi-arm movement commands.

[0091] The effect of this feature is that it enables intelligent closed-loop control from surgical field image perception to dual-arm operation execution.

[0092] The beneficial effects of the aforementioned humanoid dual-arm endoscopic structure are as follows: By using a humanoid torso to support the control and push arms, the dual-arm structure accurately simulates the coordinated hand-eye coordination of a human endoscopic surgeon, enabling the robot to naturally control the flexible continuum surgical arms; through the multi-finger precision manipulation capabilities of the dexterous end-effector, precise control of the flexible continuum handle knobs and outer sheath is achieved; through the visual servoing and motion coordination algorithms of the control module, precise timing coordination between the two arms is realized; and by simulating the coordinated hand-eye coordination of a surgeon, the robot can be directly controlled without modifying existing flexible endoscopic instruments, exhibiting strong compatibility.

[0093] like Figure 4As shown, in a preferred embodiment, the humanoid natural cavity execution module further includes an intraluminal navigation module. The intraluminal navigation module is used to provide real-time intraluminal positioning and path guidance during the movement of the flexible surgical arm, including: obtaining the three-dimensional position and orientation of the flexible surgical arm within the cavity through electromagnetic positioning; registering the preoperatively reconstructed three-dimensional cavity path model with the intraoperative positioning data in real time to generate a virtual intraluminal navigation view; and automatically guiding the selection of the correct branch direction according to the preoperative planned path when the flexible surgical arm reaches the cavity bifurcation point.

[0094] Specifically, the electromagnetic positioning system includes an electromagnetic field generator and a miniature electromagnetic sensor disposed at the distal end of the flexible surgical arm.

[0095] An electromagnetic field generator is placed at a fixed location near the operating table to generate a low-frequency alternating electromagnetic field with a known spatial distribution.

[0096] Miniature electromagnetic sensors contain multiple miniature induction coils that generate electrical signals in an electromagnetic field.

[0097] By solving the relationship between the induced electrical signal and the spatial distribution function of the electromagnetic field, the three-dimensional spatial position (x, y, z) and three-dimensional orientation (roll, pitch, yaw) of the sensor can be calculated, thus realizing the six-degree-of-freedom pose acquisition of the distal end of the flexible surgical arm in space.

[0098] The advantage of electromagnetic positioning technology is that it is not affected by visual obstruction. Even if the flexible surgical arm is located in an invisible area inside the lumen, it can still provide accurate position information.

[0099] Electromagnetic positioning typically has an accuracy of 1 to 2 mm and an update frequency of 30 to 60 Hz, meeting the accuracy and speed requirements for real-time navigation within the cavity.

[0100] The effect of this feature is that it provides line-of-sight-dependent three-dimensional positioning capability for intracavitary navigation.

[0101] Specifically, the real-time registration of the preoperative model and intraoperative data is a key process for spatially aligning the preoperatively reconstructed three-dimensional lumen path model with the intraoperative electromagnetic positioning data.

[0102] The registration process first establishes an initial rigid transformation relationship between the preoperative CT coordinate system and the intraoperative electromagnetic positioning coordinate system by using surface markers or intraluminal anatomical landmarks. Then, during the movement of the flexible surgical arm, the positional information of known anatomical landmarks (such as bifurcation points and carina) passed by the flexible surgical arm is used to continuously optimize the registration parameters online and compensate for non-rigid deformation, so as to compensate for the registration deviation caused by changes in patient position and respiratory movements.

[0103] Once registration is complete, the real-time position of the distal end of the flexible surgical arm in the electromagnetic positioning coordinate system can be mapped onto the preoperative three-dimensional path model, generating a virtual navigation view within the lumen.

[0104] The virtual navigation view displays the current position, orientation, and remaining path of the flexible surgical arm in the lumen path model in a three-dimensional visualization.

[0105] The effect of this feature is that it allows the surgeon and the system to intuitively understand the precise position of the flexible surgical arm in the three-dimensional lumen space.

[0106] Specifically, the automatic guidance at the bifurcation point of the lumen is an intelligent guidance function that is automatically activated by the intraluminal navigation module when the flexible surgical arm reaches the bifurcation point of the lumen.

[0107] This function first uses electromagnetic positioning data and intraluminal visual information to determine whether the flexible surgical arm is about to reach or has already reached a bifurcation point. Then, it matches the current position with the preoperative planned path to determine the target branch direction to be selected at the bifurcation point.

[0108] The guidance methods include: marking the direction of the target branch with a highlighted arrow on the virtual navigation view, marking the entrance of the target branch with an augmented reality overlay on the real-time endoscopic view, and informing the surgeon through voice prompts or guiding the flexible surgical arm to automatically turn to the target branch through direct control commands.

[0109] In bronchial systems with multiple bifurcations, such as the bronchial tree, this function is activated sequentially at each bifurcation point, ensuring that the flexible surgical arm penetrates step by step along the correct branch path.

[0110] The effect of this feature is that it effectively avoids the problem of accidentally entering the wrong branch at the bifurcation of the lumen, which is a common difficulty and error in traditional procedures during natural orifice surgery.

[0111] The beneficial effects of the aforementioned intraluminal navigation module are as follows: it achieves three-dimensional intraluminal positioning without relying on line of sight through electromagnetic positioning technology; it realizes the fusion and visualization of three-dimensional path model and real-time position through real-time registration of preoperative model and intraoperative data; and it solves the problem of direction selection in complex bifurcated lumens through automatic guidance at the bifurcation point of the lumen. The three working together make the movement of the flexible surgical arm in complex lumens safe, efficient and precise.

[0112] In a preferred embodiment, the system further includes a VR / AR interaction module, used to construct a virtual lumen environment based on the three-dimensional path model before surgery for doctors to perform surgical path rehearsals, and to overlay intraluminal navigation information, lesion identification results and surgical planning path onto the surgical field image in an augmented reality manner for real-time display during surgery.

[0113] Specifically, the preoperative VR simulation function is based on the three-dimensional lumen path model reconstructed by the surgical planning module, and uses virtual reality rendering technology to construct a highly realistic virtual lumen internal environment.

[0114] After wearing a VR headset, doctors can immerse themselves in the process of starting from the inlet of the lumen and virtually moving along the planned path to the target lesion, simulating the movement of the flexible surgical arm during surgery.

[0115] During the virtual rehearsal, doctors can observe the degree of curvature, diameter changes, bifurcation structures, and key anatomical landmarks of each segment of the lumen, and familiarize themselves with the characteristics and difficulties of the surgical path in advance.

[0116] VR preview also allows doctors to interactively adjust their walking speed, observation angle, and path selection, and can simulate the operation process after reaching the lesion in a virtual environment.

[0117] Data recorded from preoperative VR simulations can serve as prior knowledge input for embodied intelligent decision-making modules, improving the efficiency and accuracy of intraoperative autonomous decision-making.

[0118] The effect of this feature is that it enables doctors to have an intuitive and comprehensive understanding of the surgical path and procedure before surgery, reducing the risk of unexpected events during surgery.

[0119] Specifically, the intraoperative AR overlay function overlays various surgical aids onto the real-time surgical field image acquired by the intraluminal visual module in an augmented reality manner, forming an information-enhanced surgical view.

[0120] The overlaid information includes three categories: The first category is intraluminal navigation information, including the current position marker of the flexible surgical arm in the three-dimensional lumen model, the remaining distance and direction guidance to the target lesion, and the branch direction guidance annotation at the bifurcation of the lumen; the second category is lesion identification results, including the bounding box, segmentation mask and lesion type label of the lesion area in the lumen wall identified in real time by deep learning algorithm, which are marked on the surgical field image in a color semi-transparent overlay; the third category is the surgical planning path, including the remaining planned path from the current position to the target lesion, which is displayed in the surgical field image in a dashed overlay.

[0121] The rendering latency of AR overlay is controlled within 50 milliseconds to ensure real-time synchronization between the overlay information and the surgical field image.

[0122] The effect of this feature is that it provides the surgeon and the system with a wealth of auxiliary decision-making information without obscuring the surgical field.

[0123] The beneficial effects of the aforementioned VR / AR interaction modules are as follows: preoperative VR rehearsals allow doctors to familiarize themselves with the surgical path and procedure before surgery, reducing the cognitive load during surgery; and intraoperative AR overlay integrates navigation information, lesion identification results, and planned paths onto the surgical field image, significantly improving the visualization level and information acquisition efficiency of the surgery, and assisting surgeons and the system in making more accurate surgical decisions.

[0124] like Figure 5 As shown, in a preferred embodiment, the system further includes an embodied intelligent decision-making module, which is connected to the intelligent assessment module, surgical planning module, humanoid transluminal cavity execution module, and auxiliary rehabilitation module. This module is used to autonomously perform cognitive understanding and decision-making reasoning based on the intraluminal surgical task objectives, generate a flexible surgical arm's movement strategy and operation strategy, and distribute them to the humanoid transluminal cavity execution module. It also autonomously iterates and corrects the strategy based on execution feedback until the surgical task objectives are achieved.

[0125] Specifically, the embodied intelligent decision-making module adopts a multi-cognitive intelligent agent collaborative architecture, including multiple cognitive intelligent agents and a surgical situation sharing space, wherein the multiple cognitive intelligent agents include: The system comprises three intelligent agents: a perception-cognitive agent, which accesses intraoperative multimodal sensors, fuses multi-source perception data into a unified structured scene representation, and writes the scene representation into the surgical situation sharing space; a planning agent, which reads the scene representation and surgical task objectives from the surgical situation sharing space, generates surgical sub-task sequences and operation strategies hierarchically, and writes the operation strategies into the surgical situation sharing space; a motion agent, which reads the operation strategies from the surgical situation sharing space and converts the operation strategies into real-time motion control commands for the surgical operating device; and a safety agent, independent of the decision-making links of the perception-cognitive agent, planning agent, and motion agent, possessing an independent perception path, which performs security audits on the motion control commands and triggers execution veto when a security risk is detected; and the surgical situation sharing space, which serves as the communication medium between the multiple cognitive agents, employs an asynchronous read-write mechanism for information exchange and collaboration among the cognitive agents.

[0126] The perceptual and cognitive agent is responsible for the acquisition, fusion, and semantic processing of intraoperative multimodal perceptual data. This agent independently connects to the intraoperative multimodal sensing system, including endoscopic visual sensors, end-effector force / torque sensors, intraoperative endoscopic imaging equipment, and intracavitary navigation and positioning system.

[0127] The perceptual and cognitive intelligent agent, based on a multimodal data fusion algorithm, transforms heterogeneous sensor data into a unified structured scene representation. This structured scene representation encompasses the following semantic dimensions: tissue state description (including tissue type labeling, elastic modulus estimation, blood perfusion status, and damage severity grading), instrument pose information (including the six-degree-of-freedom pose and motion state vectors of each surgical instrument in the surgical area coordinate system), anatomical structure mapping (including the location of key anatomical landmarks, delineation of safety boundaries, and marking of surgical restricted areas), and abnormal event labeling (including spatiotemporal identifiers of abnormal states such as bleeding detection, vessel wall damage identification, and instrument slippage determination).

[0128] Scene representations are continuously refreshed at a fixed frequency and written into the scene state domain of the surgical situation sharing space. The other three cognitive agents obtain a unified understanding of the current surgical situation by reading the scene representations in the shared space, without having to process the raw sensor data independently, thus eliminating redundant computation and data consistency risks among multiple agents.

[0129] The planning and reasoning agent is responsible for the hierarchical decomposition of surgical tasks and the dynamic generation of operational strategies. This agent reads the structured scene representation provided by the perceptual and cognitive agent, as well as the preset surgical task objectives, from the shared surgical situation space, and executes hierarchical surgical planning and reasoning.

[0130] At the task level, the planning and reasoning agent decomposes the overall surgical goal into an ordered sequence of sub-tasks (e.g., localization → cavity advancement → lesion arrival → intracavitary manipulation → effect verification → instrument withdrawal). Explicit state transition conditions are defined between each sub-task, and the agent autonomously determines the completion status of the current sub-task and whether the triggering conditions for subsequent sub-tasks are met based on real-time changes in the scene representation.

[0131] At the policy level, the planning and reasoning agent generates specific operational policies for the currently executed subtask. These operational policies include positioning policies (precise spatial coordinates of the target point and optimal entry angle), entry policies (path selection, velocity curve planning, obstacle avoidance constraints), operation policies (operation type, force range, duration parameters), and force control policies (force threshold setting, compliance control parameters, adaptive gain coefficient). The generated candidate policies are written into the policy queue field of the shared space for the motion control agent to consume and execute.

[0132] When the scene representation deviates significantly, the planning and reasoning agent automatically triggers a policy revision mechanism to reassess the applicability of the current policy and generate an alternative policy. Leveraging the characteristics of an asynchronous architecture, the planning and reasoning agent can pre-generate operational policies for subsequent subtasks while the current subtask is being executed, achieving pipelined parallel processing of planning and execution.

[0133] The motion control agent is responsible for converting and issuing real-time motion commands from the operational strategy to the surgical actuator. This agent reads the operational strategy from the strategy queue domain of the surgical situation sharing space and transforms the abstract strategy into a sequence of real-time motion control commands for the flexible manipulator and end effector.

[0134] The motion control agent comprises three functional sub-modules: a trajectory planning module, which maps high-level operational strategies into time-series motion trajectories in joint space and Cartesian space, generating directly executable trajectories based on kinematic constraints, collision detection, and motion smoothness optimization criteria; a force-position hybrid control module, which dynamically switches between position control and force control modes according to the operation type—switching to force control mode for cavity propulsion operations to precisely constrain propulsion force, and switching to position control mode for alignment and positioning operations to ensure spatial positioning accuracy—with force-position control parameters adaptively adjusted based on real-time force feedback; and a real-time anomaly response module, which detects abnormal conditions such as sudden changes in contact force, instrument slippage, and accidental collisions with millisecond-level latency, and immediately executes local protective actions, including emergency braking, safe retraction, and joint locking, without waiting for intervention from other cognitive agents, thus endowing the motion control agent with independent local safety decision-making capabilities.

[0135] Before being sent to the surgical execution mechanism, the control commands generated by the motion control agent must be reviewed and approved by the safety guardian agent. Execution feedback data (including actual motion trajectory, end-effector contact force, execution status, and trajectory deviation) is written back to the execution feedback domain in the shared space for the perception and cognition agent to read and use for closed-loop updates of the scene state.

[0136] The safety-guarding intelligent agent is responsible for independent safety monitoring and risk veto throughout the entire surgical procedure. This agent is architecturally independent of the decision-making chain of the other three cognitive agents and possesses two key mechanisms: Independent Perception Path: The safety guardian directly accesses the raw sensor data stream, without relying on the scene representation generated by the perception and cognition agent in the surgical situation sharing space. This design ensures that even if the perception and cognition agent malfunctions, experiences latency, or data anomalies, the safety guardian can still independently complete the safety assessment of the surgical situation based on the raw data, forming a perception redundancy guarantee at the architectural level.

[0137] A veto mechanism: The safety-guarding intelligent agent continuously monitors three types of information sources in parallel: the content written by each cognitive agent in the shared space, the raw sensor data acquired by its own independent sensing pathway, and the consistency deviation between the two. A veto is triggered when any of the following risk conditions are detected: the operation strategy violates anatomical safety constraints (such as the instrument trajectory intruding into the safety envelope of critical blood vessels or nerves), the motion command exceeds the force / velocity / acceleration safety threshold, there is a significant consistency deviation between the self-perception results and the representation of the shared space scene (indicating that the sensing link may have failed), or the system's cumulative risk rating exceeds the preset safety threshold.

[0138] Upon triggering a veto, the safety guardian agent executes a tiered response strategy based on the risk level: Mild risk: Freezes current motion control commands, sends a strategy regeneration request to the planning and reasoning agent, pauses the surgical procedure but maintains the system's ready state, and automatically resumes execution after the new strategy passes safety review. Moderate risk: Downgrades the system control mode to physician-assisted mode, suspends the autonomous decision-making link, and allows the surgeon to determine the subsequent operation plan. Severe risk: Immediately triggers an emergency stop command, resets all motion control commands to zero, automatically retracts the surgical execution mechanism along a preset safe trajectory to a safe position, and the system enters a safety lock state, which can only be restarted after being manually unlocked by the physician.

[0139] The security guardian agent continuously writes real-time risk rating information into the risk rating domain of the shared space, which can be read by all cognitive agents, ensuring that the entire multi-agent system maintains a global consensus on the current security situation.

[0140] The surgical situation sharing space serves as the unified communication medium among the four cognitive agents (with the sole exception of the independent perception pathway of the security guardian agent). Essentially, it is a structured shared memory space with strict access control, internally divided into five functional data domains: Scene state domain: Written at a fixed frequency by the perceptual and cognitive agents, and accessed only by other agents. It stores a unified structured representation of the current surgical scene, covering tissue state, instrument pose, anatomical structure mapping, and abnormal event marking.

[0141] Task progress domain: Written, maintained, and updated by the planning and reasoning agent. It records the hierarchical execution progress of the surgical task, including the set of completed subtasks, the current execution status of subtasks, and the ordered queue of subtasks to be executed.

[0142] Policy queue field: written by the planning and reasoning agent and read by the motion control agent. The sequence of operation policies to be executed is managed using a first-in, first-out (FIFO) queue.

[0143] Risk rating domain: Written by the security guardian agent, all agents have read access. It records the system's overall risk rating index and the sub-assessment of specific risk factors in real time.

[0144] Execution feedback domain: written by the motion control agent and read by the perception and cognition agent for closed-loop updates of the scene state. It records the actual execution results of motion commands, including the actual motion trajectory, end-effector contact force, execution status flag, and deviation from the planned trajectory.

[0145] The shared space employs an asynchronous read-write mechanism, where each cognitive agent independently executes read and write operations according to its own operating cycle, without blocking, waiting, or synchronization lock dependencies. This mechanism enables the four cognitive agents to operate in true parallelism—the perception agent continuously refreshes the scene state, the planning and reasoning agent continuously generates operation strategies, the motion control agent continuously issues motion commands, and the security guardian agent continuously performs security monitoring—each working independently at its optimal operating frequency, achieving loosely coupled asynchronous information exchange and multi-agent collaboration through the shared space.

[0146] The implementation mechanisms of each cognitive agent are described in detail below. The multimodal fusion encoding of the perceptual cognitive agent is specifically implemented as follows: For natural cavity surgery scenarios, the visual input includes two visual sources: an endoscopic view and an external monitoring view. Independent visual Transformer backbone networks are configured for each source. The input image is segmented into labeled sequences by image blocks, and spatial feature representations are obtained through Transformer encoding. The two labeled sequences are concatenated into a unified long sequence with learnable viewpoint category labels added, and then fed into a cross-viewpoint Transformer fusion layer. A multi-head self-attention mechanism is used to enable each viewpoint label to pay attention to each other's spatial information. After fusion, the output vectors of each viewpoint category label position are extracted, concatenated, and compressed into a fixed-dimensional scene visual vector by a multilayer perceptron. The proprioceptive encoder receives the joint angles and angular velocities, end-effector forces and torques, and curvature distribution vectors uniformly sampled along the length of the flexible surgical arm. The three submodals are concatenated and fed into a multilayer fully connected network to output the proprioceptive embedding. The lumen wall contact force sensing signal is incorporated into the proprioceptive encoding as an additional safety dimension. The speech encoder is fine-tuned for the surgical speech domain based on the Whisper model encoder part, and relevant knowledge fragments are obtained from the surgical knowledge base by combining a retrieval enhancement generation mechanism. The final scene representation is composed of visual vectors, ontological perception embeddings, and retrieval context vectors, and is continuously written into the scene state domain of the surgical posture sharing space.

[0147] The planning and reasoning agent is a variant of the vision-language large-scale model architecture. The visual projection layer maps scene representations to the labeled embedding space of the language model, concatenates them with textual system prompts, and then feeds them into the Transformer decoder to generate an autoregressive operation strategy. Fine-tuning employs a low-rank adaptation method, training only the adapter weights and the visual projection layer, while freezing the weights of the base language model. For natural cavity surgery, a stage-adaptive meta-query mechanism maintains a dedicated stage state machine, including stages such as cavity entrance localization, lumen travel, lumen bifurcation selection, target region arrival, lesion treatment, and safe exit. Each stage corresponds to a set of learnable query vectors, which are injected as additional key-value pairs into the decoder's cross-attention layer, guiding the model's attention to automatically favor the key information of the current stage—the lumen travel stage focuses on the distance to the lumen wall and the anterior bifurcation structure, while the lesion treatment stage focuses on tissue texture and signs of bleeding. During stage switching, the meta-query vectors undergo a smooth transition via linear interpolation over several frames to prevent abrupt changes in attention patterns.

[0148] The motion control agent employs a conditional flow matching framework to train the action generation model, defining a linear interpolation path from random noise to the target action trajectory. A vector field network is trained to learn the velocity field direction from the noise to the real action. The vector field network uses a one-dimensional temporal UNet architecture, with policy conditions, scene conditions, and safety conditions injected layer by layer through a feature modulation mechanism. During inference, the action trajectory is obtained by progressively solving the learned vector field direction from the random noise using multi-step Euler integration. At each integration step, the latest conditions are read from the shared space to achieve dynamic coupling across agents. Considering the flexible body motion characteristics during natural cavity surgery, the action block size is dynamically adjusted according to the surgical stage: large blocks are used during the lumen travel stage to ensure motion continuity and flexible arm posture stability; small blocks are used during the lesion treatment stage to provide fine control; and minimal blocks are used for emergency response to achieve frame-by-frame real-time reaction. Crossfading is used to transition when the block size changes to avoid action discontinuity. During training, frequency domain regularization is applied to the generated trajectory, penalizing high-frequency components to suppress high-frequency jitter at the flexible arm end. The motion increment output by each step of the integral is processed serially by safety constraint projection: the pipe wall potential field projection prevents the flexible arm from intruding into the pipe wall, and force constraint truncation and velocity constraint truncation.

[0149] The safety guardian agent includes an adversarial diffusion module, sharing the same network architecture and conditional flow matching framework as the motion control agent, but with the opposite training objective—learning the action patterns most likely to lead to safety accidents given current scene conditions. The training dataset consists of a database of safety accidents during natural cavity surgeries, including action trajectories in accident scenarios such as lumen perforation, excessive compression of the tube wall, and flexible arm breakage. During inference, multiple failed trajectories are generated in parallel from multiple random seeds to form a dangerous trajectory envelope. The optimal trajectory output by the motion control agent is dynamically time-warped, and the distance between the failed trajectories and the optimal trajectory is measured. When the safe distance falls below a dynamic threshold, a graded safety response is triggered. In the contrastive learning dual-path verification mechanism, the safety guardian agent has a built-in independent lightweight safety perception encoder that directly extracts safety guidance features from raw sensor data. Cosine similarity is calculated in real-time with the scene representation of the perception and cognition agent. When the similarity is below a threshold for several consecutive frames, it is considered a semantic conflict and arbitration is triggered. During training, negative samples are specifically selected from hidden risk scenarios such as weak tube wall areas and densely vascularized sections, enabling the safety encoder to learn to generate divergent signals when the perception pathway appears normal but a hidden risk actually exists.

[0150] The forward prediction engine embedded in the surgical situation sharing space employs a lightweight causal Transformer architecture. It takes the recent frame-by-frame scene state embedding sequence and the current policy encoding as input, and regresses the scene state increments for each future time step through multiple prediction heads. For natural orifice surgeries, the degree of lumen deformation and the rate of change in the flexible arm curvature distribution are included as additional prediction variables in the prediction model. An online calibration mechanism maintains the prediction bias statistics using an exponential moving average and periodically updates the bias parameters of the prediction network output layer, enabling the world model to continuously adapt to the individual patient's lumen tissue biomechanics during surgery. When the prediction bias exceeds the normal range for several consecutive frames, the prediction is marked as failed, the motion control agent switches to a purely reactive control mode, and the safety guardian agent switches to a conservative threshold mode.

[0151] like Figure 6 As shown, in a preferred embodiment, the system further includes a safety monitoring module for real-time monitoring of the contact force between the flexible surgical arm and the lumen wall and the bending curvature of the flexible arm. When the monitored parameters exceed the preset safety range, a graded safety response is executed, which includes at least one of early warning prompts, motion restriction, and automatic degradation to the doctor's master control mode.

[0152] Specifically, the contact force monitoring function monitors the magnitude and direction of the contact force between the flexible surgical arm and the lumen wall in real time through a miniature force sensor array set at the distal end and side wall of the flexible surgical arm.

[0153] In natural orifice surgery, the flexible surgical arm will inevitably come into contact with the lumen wall as it moves through the lumen, but the contact force must be controlled within a safe range to avoid damage to the lumen wall mucosa and muscle layer.

[0154] The safety threshold for contact force with the lumen wall is set differently depending on the lumen type and the thickness of the lumen wall. For example, the safety threshold is relatively high for the thicker esophageal wall and relatively low for the thinner distal bronchial wall.

[0155] The force sensor array continuously collects contact force data at a sampling frequency of no less than 100Hz, and outputs a stable force signal after removing noise interference through a filtering algorithm.

[0156] The effect of this feature is to ensure the safety of the lumen wall in real time from a mechanical perspective, preventing damage or perforation of the lumen wall due to excessive contact force.

[0157] Specifically, the bending curvature monitoring function calculates the real-time bending curvature of each segment of the flexible surgical arm based on the bending morphology data provided by the flexible arm morphology sensing unit.

[0158] Each type of flexible continuum surgical arm has its maximum safe bending curvature (i.e. minimum safe bending radius). Exceeding this limit may cause fatigue damage to the continuum drive mechanism or extrusion deformation of the internal channels of the flexible body.

[0159] The flexible arm shape sensing unit can use distributed fiber Bragg grating (FBG) sensor technology, embedding micro-fiber sensors along the length of the flexible body, and calculating the curvature at that location by detecting the change in the reflected wavelength of each grating.

[0160] When the curvature of any segment is detected to be close to or exceed the safe range, the safety monitoring module immediately triggers the corresponding safety response.

[0161] The effect of this feature is to protect the flexible surgical arm from damage due to excessive bending from the perspective of instrument mechanics, while preventing compression damage to the lumen wall caused by excessive bending.

[0162] Specifically, the graded security response is a progressive security measure executed by the security monitoring module when it detects that the monitoring parameters exceed the preset security range.

[0163] The graded safety response includes three levels: Level 1 Response (Early Warning): When a minor abnormality is detected (such as contact force approaching 80% of the safety threshold, or bending curvature approaching 90% of the safety limit), an early warning is issued to the doctor via visual cues (flashing yellow indicator) and audible cues (low-frequency beep) on the control terminal display, reminding the doctor to pay attention to the current operating status; Level 2 Response (Motion Restriction): When a moderate abnormality is detected (such as contact force exceeding the safety threshold or bending curvature exceeding the safety limit), the advancement and continued bending movements of the flexible surgical arm are automatically restricted, allowing only retraction and reduction of bending angle to prevent further deterioration of the abnormal state; Level 3 Response (Automatic Degradation to Doctor-Controlled Mode): When a severe abnormality is detected (such as contact force continuously increasing and exceeding 150% of the safety threshold, or bending curvature severely exceeding the limit), the control mode is automatically downgraded from the current mode to doctor-controlled mode, and decision support information including details of the current abnormal state and suggested treatment strategies is pushed to the doctor, allowing the doctor to directly take over the operation.

[0164] The effect of this feature is that it enables graded and progressive safety response measures based on the severity of the abnormality, ensuring safety while avoiding excessive response that could affect the surgical process.

[0165] The beneficial effects of the aforementioned safety monitoring module are as follows: it achieves mechanical protection of the lumen wall through real-time contact force monitoring; it achieves mechanical safety assurance of the instrument and prevention of lumen wall compression damage through real-time bending curvature monitoring; and it achieves targeted and progressive implementation of safety measures through a graded safety response mechanism that includes early warning prompts, movement restrictions, and automatic demotion to the doctor's control mode, ensuring that the doctor always retains ultimate control while ensuring the safety of intraluminal operations.

[0166] The data flow and collaborative relationships between the embodied intelligent decision-making module and other modules in the system are as follows: Between the embodied intelligent decision-making module and the intelligent assessment module, the perceptual and cognitive intelligent agent obtains the patient's preoperative assessment results (including three-dimensional reconstruction of the luminal anatomy, lesion localization and grading, and luminal accessibility analysis) from the intelligent assessment module, and writes them into the surgical situation sharing space as global prior knowledge for continuous reference by the planning and reasoning intelligent agent and the safety protection intelligent agent during surgery; between the embodied intelligent decision-making module and the surgical planning module, the planning and reasoning intelligent agent obtains the surgical path planning scheme and operation sequence template through the natural orifice from the surgical planning module, parses them into staged sub-goals and caches them in the surgical situation sharing space, and the motion control intelligent agent gradually generates motion commands that conform to the luminal constraints based on these sub-goals. Between the humanoid natural cavity surgery execution module and the surgical execution module, the motion control agent sends real-time motion control commands to the flexible surgical arm drive unit and the push arm. The perception and cognition agent receives endoscopic images and force feedback from the intraluminal vision module and updates the luminal environment status in the surgical situation sharing space. The safety protection agent directly accesses the contact force sensor and bending curvature sensor through an independent perception channel to achieve real-time monitoring and intervention of the risk of luminal wall damage. Between the surgical execution module and the auxiliary rehabilitation module, after the surgery is completed, the planning and reasoning agent writes the intraoperative operation record, intraluminal operation parameters and postoperative luminal status assessment data into the archive area of ​​the surgical situation sharing space and transmits them to the auxiliary rehabilitation module as the basis for formulating the postoperative rehabilitation plan.

[0167] The beneficial effects of the aforementioned embodied intelligent decision-making module are as follows: Through the architecture of asynchronous reading and writing of the surgical situation sharing space by four intelligent agents, the decoupled parallel processing of perception, planning, control, and safety protection is realized, which significantly reduces the system response latency compared with the traditional serial decision pipeline; the perception and cognition intelligent agent adopts a multimodal fusion mechanism to uniformly encode endoscopic vision, instrument force perception, and lumen navigation information into the shared space, providing downstream intelligent agents with a consistent and real-time updated lumen environment representation; the planning and reasoning intelligent agent performs multi-step prospective prediction based on a causal reasoning world model, which can generate globally optimal operation strategies in lumen branch path selection and complex lesion treatment; the motion control intelligent agent transforms high-level operation commands into continuous trajectories that conform to the kinematic constraints of flexible instruments, realizing sub-millimeter-level precise control within the lumen; the safety protection intelligent agent operates with an independent perception pathway, without relying on the intermediate results of other intelligent agents, ensuring that it can still independently execute lumen wall safety protection when any intelligent agent malfunctions, realizing system-level fault isolation and safety fallback.

[0168] This application provides a control method for the aforementioned humanoid intelligent transnatural cavity surgical robot system. The specific solution of this method is as follows: Figure 7 As shown below, the control method of the humanoid intelligent transoral surgical robot system will be described in detail.

[0169] Combination Figure 7 As shown, the control method of the humanoid intelligent transoral surgical robot system includes: S101, based on the patient's medical data and physical signs, conducts targeted disease assessments and determines the indications for transoral orifice surgery, generates personalized auxiliary assessment results for the patient, and displays them; S102, Based on the patient's medical imaging data, reconstruct a three-dimensional path model of the target natural cavity, perform targeted transoral surgical planning for the patient based on the three-dimensional path model, generate a personalized surgical plan, and display it. S103, according to the surgical planning scheme, control the flexible surgical instruments to reach the target site through the body's natural cavities, obtain surgical field images through visual perception within the cavity, and perform surgical operations. S104 generates and displays a personalized postoperative rehabilitation plan for the patient based on their medical data and postoperative assessment results.

[0170] The control method of the humanoid intelligent transoral surgical robot system provided in the above embodiments of this application corresponds to the humanoid intelligent transoral surgical robot system provided in the embodiments of this application. Therefore, the specific content of the control method corresponds to the humanoid intelligent transoral surgical robot system. The specific content can be referred to the records in the humanoid intelligent transoral surgical robot system, which will not be repeated in this application.

[0171] The control method of the humanoid intelligent transoral surgical robot system provided in the above embodiments of this application is based on the same inventive concept as the humanoid intelligent transoral surgical robot system provided in the embodiments of this application, and has the same beneficial effects as the methods adopted, run or implemented by its stored application.

[0172] Based on the same inventive concept, another embodiment of this application provides an electronic device for implementing the control method of the humanoid intelligent transoral surgical robot system described in the above embodiments. Figure 8 As shown, the electronic device includes a memory 301 and a processor 303.

[0173] Memory 301 can be configured to store a program.

[0174] Additionally, memory 301 can also be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, contact data, phonebook data, messages, pictures, videos, etc.

[0175] Memory 301 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Processor 303, coupled to memory 301, is used to execute programs in memory 301 for: Based on the patient's medical and physical examination data, targeted disease assessments and indications for transoral orifice surgery are determined, generating and displaying personalized auxiliary assessment results. Based on the patient's medical imaging data, a three-dimensional path model of the target natural cavity is reconstructed. Based on the three-dimensional path model, a targeted surgical plan is made for the patient through the natural cavity, generating a personalized surgical plan, which is then displayed. According to the surgical plan, the flexible surgical instruments are controlled to reach the target site through the body's natural cavities, and the surgical field image is obtained through visual perception within the cavity and the surgical operation is performed. Based on the patient's medical data and postoperative assessment results, a personalized postoperative rehabilitation plan is generated and displayed.

[0176] In this application, Figure 8 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 8 The components shown.

[0177] The electronic device provided in this embodiment is based on the same inventive concept as the control method of the humanoid intelligent transoral surgical robot system provided in this application embodiment, and has the same beneficial effects as the methods adopted, run or implemented by the application stored therein.

[0178] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0179] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0180] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0181] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory. Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0182] This application also provides a computer-readable storage medium corresponding to the control method of the humanoid intelligent transoral surgical robot system provided in the foregoing embodiments, wherein a computer program (i.e., a program product) is stored thereon, and the computer program, when run by a processor, executes the control method of the humanoid intelligent transoral surgical robot system provided in any of the foregoing embodiments.

[0183] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, optical disc read-only memory (CDROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0184] The computer-readable storage medium provided in the above embodiments of this application and the control method of the humanoid intelligent transoral surgical robot system provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0185] It should be noted that numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0186] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, system, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, system, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, system, article, or apparatus that includes said element.

[0187] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A humanoid intelligent transoral surgical robot system, characterized in that, The system is built upon an embodied intelligence closed-loop architecture of perception-cognition-decision-execution-feedback, including: The intelligent assessment module is used to conduct targeted disease assessments and determine the indications for transoral orifice surgery based on the patient's medical data and vital signs, generate personalized auxiliary assessment results for the patient, and display them. The surgical planning module is used to reconstruct a three-dimensional path model of the target natural cavity based on the patient's medical imaging data, perform targeted transoral surgical planning for the patient based on the three-dimensional path model, generate a personalized surgical planning scheme, and display it. The humanoid natural cavity execution module is used to control flexible surgical instruments to reach the target site through the human body's natural cavities according to the surgical planning scheme, acquire surgical field images through intracavitary visual perception, and perform surgical operations. The auxiliary rehabilitation module is used to generate and display personalized postoperative rehabilitation plans for patients based on their medical data and postoperative assessment results.

2. The humanoid intelligent transoral surgical robot system according to claim 1, characterized in that, The humanoid trans-natural cavity execution module includes: The humanoid robot body is used to dock with the operating table berth, and after docking is completed, the body's motion degrees of freedom are eliminated and environmental vibrations are suppressed; The cavity creation module is used to create and maintain the cavity space required for surgical operations within the patient's body cavity; The flexible surgical arm module, mounted on the humanoid robot body, is driven by a continuum mechanism and is used to acquire surgical field images within the lumen, insert through a natural cavity entrance and travel along the lumen path to the target site, and perform surgical operations within the cavity space.

3. The humanoid intelligent transoral surgical robot system according to claim 1, characterized in that, The humanoid transoral cavity execution module includes a control module, a humanoid torso, a control arm, and a push arm; the humanoid torso is equipped with a control arm and a push arm; the end of the control arm is equipped with a dexterous hand for manipulating a flexible continuum handle, and the end of the push arm is equipped with a dexterous hand for pushing and retracting the flexible continuum, so as to simulate the coordinated hand movements of a surgeon. Based on the acquired surgical field images, the control module generates control commands to control the manipulator and pusher arms to perform surgical actions.

4. The humanoid intelligent transoral surgical robot system according to claim 2, characterized in that, The humanoid natural cavity execution module also includes an intraluminal navigation module, which provides real-time intraluminal positioning and path guidance during the movement of the flexible surgical arm. This includes: obtaining the three-dimensional position and orientation of the flexible surgical arm within the cavity through electromagnetic positioning; registering the preoperatively reconstructed three-dimensional cavity path model with the intraoperative positioning data in real time to generate a virtual intraluminal navigation view; and automatically guiding the selection of the correct branch direction based on the preoperative planned path when the flexible surgical arm reaches the cavity bifurcation point.

5. The humanoid intelligent transoral surgical robot system according to claim 1, characterized in that, The system also includes a VR / AR interaction module, which is used to construct a virtual lumen environment based on the three-dimensional path model before surgery for doctors to perform surgical path rehearsals, and to overlay intraluminal navigation information, lesion identification results and surgical planning path onto the surgical field image in an augmented reality manner for real-time display during surgery.

6. The humanoid intelligent transoral surgical robot system according to claim 1, characterized in that, The system also includes an embodied intelligent decision-making module, which is connected to the intelligent assessment module, surgical planning module, humanoid transluminal cavity execution module, and auxiliary rehabilitation module. It is used to autonomously perform cognitive understanding and decision-making reasoning of multimodal perception information within the lumen based on the surgical task objectives, generate the flexible surgical arm's movement strategy and operation strategy and send them to the humanoid transluminal cavity execution module, and autonomously iterate and correct the strategy based on execution feedback until the surgical task objectives are achieved.

7. The humanoid intelligent transoral surgical robot system according to claim 1, characterized in that, The system also includes a safety monitoring module for real-time monitoring of the contact force between the flexible surgical arm and the lumen wall and the bending curvature of the flexible arm. When the monitored parameters exceed the preset safety range, a graded safety response is executed. The graded safety response includes at least one of the following: early warning prompts, movement restrictions, and automatic degradation to the doctor's main control mode.

8. A control method for a humanoid intelligent transoral surgical robot system, applied to the humanoid intelligent transoral surgical robot system as described in any one of claims 1 to 7, characterized in that, include: Based on the patient's medical and physical examination data, targeted disease assessments and indications for transoral orifice surgery are determined, generating and displaying personalized auxiliary assessment results. Based on the patient's medical imaging data, a three-dimensional path model of the target natural cavity is reconstructed. Based on the three-dimensional path model, a targeted surgical plan is made for the patient through the natural cavity, generating a personalized surgical plan, which is then displayed. According to the surgical plan, the flexible surgical instruments are controlled to reach the target site through the body's natural cavities, and the surgical field image is obtained through visual perception within the cavity and the surgical operation is performed. Based on the patient's medical data and postoperative assessment results, a personalized postoperative rehabilitation plan is generated and displayed.

9. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores computer-executable instructions, and the processor executes the computer-executable instructions to implement the steps of the control method for the humanoid intelligent transnatural cavity surgical robot system of claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the control method for the humanoid intelligent transnatural cavity surgical robot system of claim 8.

Images