Humanoid robot surgery system

Abstract

In the humanoid robot surgery system provided by the embodiment of the present application, the mechanical arm is assembled on the trunk and is in communication connection with the host computer, the trunk is assembled on the moving mechanism, the trunk is provided with a host computer accommodating space for assembling the host computer in the host computer accommodating space, the man-machine interface is assembled on the trunk and is in communication connection with the host computer, the host computer is deployed with a generative surgery large model, and the surgery large model performs the following steps: constructing a three-dimensional physiological structure model through the man-machine interface to accept the basic medical record information, preoperative examination results and lesion image data of the patient; performing feature analysis on the three-dimensional physiological structure model to mark the target surgery site and accordingly to formulate the mechanical arm path; establishing a mapping relationship between the mechanical arm path and the position of the patient to generate the action trajectory sequence of the mechanical arm; and driving the moving mechanism to move towards the position of the patient to reach the specified point according to the action trajectory sequence, and controlling the mechanical arm to drive the surgical instrument to act to perform surgery on the target surgery site.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more particularly to a humanoid robotic surgical system. Background Technology

[0002] Surgical procedures occupy a vital position in modern medicine, and the precision and safety of these procedures have always been the goals pursued in the medical field. Over the past 30 years, surgical navigation and positioning robot technology has continued to develop and has been increasingly widely used in many fields such as neurosurgery, orthopedics, dental implantology, abdominal paracentesis, and bronchial navigation.

[0003] Currently, existing surgical robots have many limitations in clinical applications. Most surgical robots primarily focus on puncture navigation and positioning, with relatively low levels of automation and intelligence, and extremely limited interaction methods with surgeons. Furthermore, these robots are typically fixed in specific locations within the operating room, occupying a significant amount of surgical space, and during surgery, they can only perform simple positioning operations according to pre-set programs, making it difficult to flexibly adjust surgical strategies based on the actual intraoperative situation, thus offering limited overall assistance to clinical surgery. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a humanoid robotic surgical system to at least partially solve the above-mentioned problems.

[0005] According to a first aspect of the present invention, a humanoid robotic surgical system is provided, comprising: a robotic arm, a torso, a moving mechanism, a host computer, and a human-machine interface. The robotic arm is mounted on the torso and communicatively connected to the host computer. The torso is mounted on the moving mechanism. A host computer accommodating space is provided within the torso to accommodate the host computer. The human-machine interface is mounted on the torso and communicatively connected to the host computer. A generative surgical model is deployed on the host computer, and the surgical model performs the following steps to conduct a surgical procedure:

[0006] The human-machine interface is used to receive the patient's basic medical record information, preoperative examination results and lesion imaging data to construct a three-dimensional physiological structure model.

[0007] Feature analysis is performed on the three-dimensional physiological structure model to mark the target surgical site and formulate the robotic arm path accordingly;

[0008] Establish a mapping relationship between the robotic arm path and the patient's location to generate a sequence of robotic arm motion trajectories;

[0009] According to the sequence of motion trajectories, the moving mechanism is driven to move toward the patient's position to reach the designated point, and the robotic arm is controlled to drive the surgical instruments to perform surgery on the target surgical site.

[0010] The solutions in the embodiments of the present invention have at least the following technical advantages:

[0011] 1. Precision surgical planning and execution

[0012] The system uses a human-machine interface to acquire detailed patient information to construct a three-dimensional physiological structure model. Compared to existing surgical robots that can only perform simple positioning, this system can more accurately mark the target surgical site and plan the robotic arm path. This helps doctors understand the patient's condition more intuitively and accurately before surgery, plan the surgical procedure, and thus improve the precision of the surgery. For example, in complex abdominal surgeries, accurately identifying the lesion and its relationship with surrounding tissue structures can avoid misoperation and reduce surgical risks.

[0013] Based on accurate model analysis, the robotic arm motion trajectory sequence is generated, which can ensure that surgical instruments move along the predetermined optimal path during the operation, achieve precise operation, and overcome the problem that existing robots cannot cope with changes during the operation based on preset programs.

[0014] 2. Improve surgical efficiency

[0015] The mobile mechanism allows the robot to move freely and quickly reach the patient's designated location, eliminating the need for complex pre-operative layout adjustments required by traditional fixed robots. This saves surgical preparation time and improves overall surgical efficiency.

[0016] The generative surgical model on the host computer coordinates the work of various components, making the movements of the robotic arm, torso, and moving mechanism more efficient and smooth, avoiding the time waste caused by the relatively independent operation of each part of the traditional robot and the loose connection between them.

[0017] 3. Enhance human-machine collaboration

[0018] The human-machine interface facilitates interaction between doctors and the robot, allowing doctors to input commands and receive system feedback in a timely manner. Compared to the limited interaction methods of existing robots, this efficient communication method enables doctors to better control the surgical process and improves surgical collaboration.

[0019] The system can provide decision support to doctors based on their input and the intraoperative situation. For example, it takes into account the doctor's operating habits and experience when planning the robotic arm path, making the robot's assistance more in line with the doctor's needs and improving the tacit understanding between humans and machines.

[0020] 4. Optimize surgical adaptability

[0021] The system can adjust the robotic arm path and trajectory according to the actual situation during surgery, adapting to dynamic factors such as changes in patient position and tissue deformation, while existing robots are less flexible in this regard. This allows the surgery to proceed more smoothly and reduces the risk of surgical interruption or failure due to the inability to adjust the surgical strategy in a timely manner.

[0022] By establishing a mapping relationship between the robotic arm path and the patient's position, the system can quickly and accurately adjust the robotic arm movements under different surgical scenarios and individual patient differences, thereby improving the adaptability of the surgical system to various surgical situations.

[0023] It should be noted that the above-mentioned technical benefits are not to be achieved simultaneously in a single embodiment, but rather that a particular embodiment may only have some of the technical benefits. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the structure of a humanoid robotic surgical system according to an embodiment of the present invention. Detailed Implementation

[0025] Figure 1 This is a schematic diagram of the structure of a humanoid robotic surgical system according to an embodiment of the present invention. Figure 1 As shown, it specifically includes: a robotic arm 101, a torso 102, a moving mechanism 103, a host computer 104, and a human-machine interface 105. The robotic arm is mounted on the torso and communicates with the host computer. The torso is mounted on the moving mechanism. A host computer housing space is provided within the torso to house the host computer. The human-machine interface is mounted on the torso and communicates with the host computer. A generative surgical model is deployed on the host computer. The surgical model performs the following steps to conduct a surgical procedure:

[0026] The human-machine interface is used to receive the patient's basic medical record information, preoperative examination results and lesion imaging data to construct a three-dimensional physiological structure model.

[0027] Feature analysis is performed on the three-dimensional physiological structure model to mark the target surgical site and formulate the robotic arm path accordingly;

[0028] Establish a mapping relationship between the robotic arm path and the patient's location to generate a sequence of robotic arm motion trajectories;

[0029] According to the sequence of motion trajectories, the moving mechanism is driven to move toward the patient's position to reach the designated point, and the robotic arm is controlled to drive the surgical instruments to perform surgery on the target surgical site.

[0030] The humanoid robotic surgical system of this invention has several significant advantages: In terms of precise surgical planning and execution, the three-dimensional physiological structure model constructed with the help of the human-machine interface can accurately mark target areas, formulate robotic arm paths, and generate motion trajectory sequences, thereby improving surgical accuracy and overcoming the limitations of existing robots; In terms of improving surgical efficiency, the mobile mechanism can quickly take position, and the generative surgical model makes the collaboration of various components more efficient, saving preparation time; Enhanced human-machine collaboration is reflected in the convenient interaction of the human-machine interface and the decision support provided by the system, improving surgical coordination and human-machine tacit understanding; Optimized surgical adaptability is manifested in the ability to adjust the robotic arm path and movements according to the actual situation during surgery and patient differences, reducing the risk of surgical interruption or failure, and making the surgery more successful.

[0031] When constructing a three-dimensional physiological structure model from the patient's basic medical record information, preoperative examination results, and lesion imaging data through the human-computer interface, the generative surgical large model specifically includes the following processing steps:

[0032] (1) Data Acquisition and Preprocessing

[0033] 1. Interface data reception

[0034] The system utilizes a high-speed data bus connected to a human-machine interface (HMI) equipped with multiple communication protocol stacks to ensure seamless integration with various external devices, such as Hospital Information Systems (HIS), Laboratory Information Systems (LIS), and Picture Archiving and Communication Systems (PACS). For example, for DICOM-compliant imaging equipment, the HMI establishes a connection via the DICOM network communication protocol, enabling high-speed transmission of image data. Simultaneously, text data such as basic medical record information and preoperative examination results are received using the secure and reliable HL7 (Health Level-7) protocol. This design ensures accurate, complete, and efficient data transmission from external devices.

[0035] Upon receiving data, the human-machine interface performs preliminary format and integrity checks. For example, for DICOM image data, it checks whether key information in the file header (such as patient ID, examination date, and image modality) is complete; for text data, it verifies whether the format of the data fields conforms to predetermined specifications. Only data that passes the preliminary checks is further transmitted for subsequent processing.

[0036] 2. Data Validation and Cleaning

[0037] It incorporates a rigorous data validation rule base, built upon medical data standards and common error patterns. For example, for various indicators in routine blood tests, reasonable value ranges are set; if received data exceeds these ranges, it is marked as suspicious. For imaging data, the statistical characteristics of the images (such as mean and variance) are calculated and compared with reference values ​​for normal images to determine if any anomalies exist.

[0038] For data cleaning, machine learning-based interpolation algorithms are used to address missing data. For example, a regression model is trained using existing patient data to predict the values ​​of missing data based on other relevant patient characteristics (such as age, gender, and disease type). For noise interference, wavelet transform algorithms are used for filtering. Wavelet transform can analyze image data at different scales and frequencies, effectively removing noise while preserving image details. For instance, when processing Gaussian noise in MRI images, selecting appropriate wavelet basis functions and thresholds can significantly improve the signal-to-noise ratio.

[0039] (2) Image data registration and fusion

[0040] 1. Multimodal image registration

[0041] A registration algorithm based on maximizing mutual information is employed, which leverages the information redundancy between images of different modalities to achieve accurate registration. Specifically, for CT and MRI images, the images are first normalized to ensure a consistent grayscale range. Then, the MRI image is moved spatially with a certain step size, and the mutual information between the CT and MRI images is calculated after each movement. Mutual information is a measure of the degree of interdependence between two random variables; when the mutual information reaches its maximum value, the two images are considered to have achieved optimal spatial matching.

[0042] To improve the accuracy and efficiency of registration, the algorithm employs a multi-resolution strategy. It begins with coarse registration using low-resolution images to quickly determine the approximate range of transform parameters, then gradually increases the resolution to perform registration adjustments at a finer scale. Simultaneously, it utilizes an image pyramid structure to sample and process the image at different resolution levels, reducing computational load while maintaining registration accuracy. For example, in the initial low-resolution stage, the image is reduced to 1 / 8 of its original size, and the approximate registration position is found by quickly calculating mutual information. Then, in the high-resolution stage, the registration results are gradually refined.

[0043] 2. Image Fusion

[0044] For image fusion, a weighted fusion algorithm based on principal component analysis (PCA) is employed. First, PCA decomposition is performed on the registered CT and MRI images respectively to obtain their respective principal component vectors and eigenvalues. Then, weights are assigned to each principal component based on the importance and contribution of different tissues in different modalities of the images. For example, higher weights are assigned to skeletal structures in CT images, while higher weights are assigned to soft tissues in MRI images.

[0045] Finally, the weighted principal components are subjected to inverse PCA transformation to obtain the fused image. This fusion method can fully preserve the high-resolution information of bone structures in CT images and the rich detail information of soft tissues in MRI images. The generated fused image shows the lesion area and its relationship with surrounding tissues more clearly and accurately. For example, in brain tumor surgical planning, the fused image can clearly show the location and size of the tumor and its relationship with surrounding blood vessels and nerve tissues, providing an important basis for surgical path planning.

[0046] (3) Three-dimensional model reconstruction

[0047] 1. Segmentation Algorithm

[0048] A deep learning segmentation algorithm based on the 3D U-Net architecture is adopted. This network structure extends the 2D U-Net to three-dimensional space, enabling better handling of the contextual information of volume data. The network consists of two parts: an encoder and a decoder. The encoder extracts features from the image data step by step through a series of convolutional and pooling layers, gradually reducing the image resolution while increasing the number of feature channels, thereby obtaining more abstract and higher-level semantic features.

[0049] The decoder maps features back to the original image resolution through upsampling and convolutional layers, and fuses them with feature maps from the corresponding levels in the encoder to recover detailed image information. During training, the network is trained using a large amount of labeled medical image data. The loss function combines cross-entropy loss and Dice coefficient loss, enabling the network to learn accurate tissue segmentation boundaries. For example, when training a lung CT image segmentation model, the network can accurately identify different structures such as lung tissue, trachea, blood vessels, and lung tumors, achieving sub-millimeter level segmentation accuracy.

[0050] 2. 3D modeling technology

[0051] After segmentation, the Marching Cubes algorithm is used to reconstruct the surface of tissue and lesion regions represented by voxels. This algorithm extracts isosurfaces from the volume data and converts the voxel data into a 3D surface model represented by triangular facets. Specifically, the algorithm iterates through each voxel in the volume data, determines whether the voxel intersects with an isosurface based on the gray value of the voxel vertex, and if so, calculates the intersection points of the isosurface and the voxel edges through interpolation. Connecting these intersection points forms triangular facets, thereby constructing the surface of the 3D model.

[0052] For polygonal mesh representations, a modeling technique based on a half-edge data structure is used. This data structure can efficiently represent and process polygonal mesh models, facilitating model editing, modification, and optimization. After model construction, the Laplacian smoothing algorithm is used to smooth the model surface. By adjusting the vertex positions, the model surface becomes smoother and more natural. Simultaneously, a hole-filling algorithm is used to repair holes in the model, ensuring its integrity and topological correctness. For example, when constructing a 3D model of the heart, the smoothed and hole-filled model accurately represents the heart's external structure, facilitating intuitive observation and surgical planning by doctors.

[0053] (4) Model Validation and Optimization

[0054] 1. Model Validation

[0055] The system stores a database containing standard models of various anatomical structures, which are constructed based on a large amount of normal human anatomy data. During model validation, the constructed 3D physiological structure model is geometrically matched with the corresponding standard model. First, a feature point matching algorithm is used to determine the corresponding anatomical feature points in the two models, such as the joints of bones and the center points of organs. Then, the Hausdorff distance between the models is calculated, which measures the maximum mismatch between the two point sets.

[0056] Simultaneously, manual review is conducted using the expertise and experience of doctors. Doctors observe the model using specialized visualization software, checking the accuracy of anatomical structures and the reasonableness of lesion areas. If deviations are found during the verification process, such as significant differences in the shape or location of an organ compared to the standard model, relevant information is recorded, and the process is traced back to data acquisition, registration, and segmentation stages. By analyzing intermediate data, potential causes of the problem are identified, such as motion artifacts during image data acquisition or incorrect segmentation algorithm judgments in specific areas.

[0057] 2. Model Optimization

[0058] Based on the validation results, a gradient descent-based reinforcement learning algorithm was used to optimize and adjust the parameters and algorithm during the model construction process. For the threshold parameter in the image segmentation algorithm, the gradient direction of the parameter was calculated based on the error feedback from model validation, and then the threshold parameter was updated according to a certain learning rate to make the segmentation result closer to the real anatomical structure. For the weights of the deep learning model, the backpropagation algorithm was used to calculate the gradient of the loss function with respect to the weights, and the weights were updated using gradient descent to improve the prediction accuracy of the model.

[0059] As the system accumulates more surgical case data, the training dataset is continuously expanded, and the deep learning model is retrained and iteratively updated. Simultaneously, based on the characteristics of different case types (such as different disease types, different age groups of patients, etc.) and complex surgical scenarios (such as multi-organ surgeries, minimally invasive surgeries, etc.), the model's structure and parameter settings are adjusted to better adapt to various practical application needs and consistently maintain a high level of surgical assistance performance. For example, when handling complex abdominal multi-organ surgery cases, the optimized model can more accurately construct three-dimensional physiological structure models, providing more reliable planning and navigation support for the surgery.

[0060] In the above-described solution, this invention integrates basic medical record information, preoperative examination results, and lesion image data from multiple modalities. This innovation is fully reflected in the data processing workflow. Through meticulously designed data acquisition and preprocessing steps, it ensures that different types of data can be accurately and completely collected and integrated. During the data verification and cleaning stage, various advanced algorithms are employed to address the characteristics of multi-source data, such as machine learning-based interpolation algorithms to handle missing data. This approach considers not only the characteristics of individual image data but also the overall clinical information of the patient. This multi-source data fusion processing method breaks through the limitations of traditional surgical robots that rely solely on single images or simple positioning information, providing more comprehensive and in-depth information on the patient's physiological and pathological state for surgical planning, thus helping doctors develop more precise and personalized surgical plans.

[0061] Furthermore, the use of a deep learning segmentation algorithm based on the 3D U-Net architecture for image segmentation is one of the key innovations of this technology. Compared with traditional manual feature extraction or simple threshold segmentation methods, 3D U-Net can automatically learn complex feature patterns in image data. Especially when processing three-dimensional volume data, its context information extraction capability is stronger, enabling it to accurately distinguish different tissue types and lesion regions, and construct a more realistic and detailed three-dimensional physiological structure model. During training, the combination of cross-entropy loss and Dice coefficient loss effectively improves the model's accuracy in identifying segmentation boundaries. This big data-based learning method greatly enhances the accuracy and reliability of the model, providing a solid foundation for precise surgical procedures. For example, in neurosurgery, it can more accurately locate small brain tumors and their surrounding important neural structures, reducing surgical risks.

[0062] Furthermore, the adaptive optimization mechanism in the model building process is a key innovation of this invention. In the model validation and optimization phase, the accuracy of the model is comprehensively evaluated through a combination of comparison with a standard model and manual review by physicians. A gradient descent-based reinforcement learning algorithm is used to adjust the model parameters in real time, optimizing the model not only in current surgical cases but also iteratively updating the model building algorithm as surgical case data accumulates. This continuous learning capability enables the large surgical model to adapt to the ever-changing practical needs of the medical field, such as the emergence of new diseases and improvements in surgical techniques. When faced with individual differences among patients and complex cases, the model can continuously self-optimize, maintaining a high level of surgical assistance performance, providing a strong guarantee for improving surgical quality and success rates.

[0063] When the generative surgical model performs feature analysis on the three-dimensional physiological structure model to mark the target surgical site and determine the robotic arm path accordingly, the specific processing steps include the following:

[0064] (1) Feature analysis and target surgical site marking

[0065] 1. Multi-scale feature extraction

[0066] A three-dimensional wavelet transform algorithm was used to perform multi-scale decomposition of a three-dimensional physiological structure model. This algorithm decomposes the model into sub-bands of different scales and orientations in three-dimensional space, similar to wavelet decomposition of images, but taking into account the characteristics of volume data. At each scale, local geometric features of the model, such as mean curvature, Gaussian curvature, and shape index, were calculated. For example, for vascular structures, at finer scales, curvature features can highlight the tortuous parts and branching points of the vessels, areas that may require special attention during surgical procedures.

[0067] Simultaneously, a feature learning method based on a three-dimensional convolutional neural network (3D CNN) is employed. A 3D CNN architecture with multiple convolutional and pooling layers is constructed, with three-dimensional physiological structure model data as input. In the convolutional layers, small-sized three-dimensional convolutional kernels (e.g., 3x3x3) are used to traverse the model data and extract local features. As the network depth increases, the receptive field gradually expands, enabling the learning of more global feature relationships. For example, in brain surgery, the network can learn the relative positional relationships between different brain regions and their association with lesion areas. The geometric features extracted by wavelet transform and the features learned by the 3D CNN are fused in the fully connected layers of the network to obtain a comprehensive feature representation, providing rich information for the labeling of the target surgical site.

[0068] 2. Feature analysis based on anatomical knowledge

[0069] An integrated database containing numerous standard models of human anatomical structures and disease feature patterns has been developed, built upon authoritative medical textbooks, clinical experience, and extensive case data. During analysis, three-dimensional physiological structure models are registered and compared with the standard models in the database. For example, in cardiac surgery, the patient's three-dimensional heart model is registered with a standard model of a normal heart, and the differences between the two are calculated. Simultaneously, the database stores disease feature patterns of various heart diseases, such as the typical morphology of myocardial infarction areas and the geometric characteristics of valvular heart disease. By matching these patterns, areas where lesions may exist can be quickly located.

[0070] Based on the surgical type and specific requirements input by the doctor through the human-machine interface, relevant features are further filtered and focused. For example, in the case of coronary artery bypass surgery, the focus will be on the stenosis or blockage of the coronary artery, as well as surrounding vascular branches that can be used for bypass surgery. In this way, by utilizing anatomical knowledge and the doctor's intentions, the target surgical site can be accurately located.

[0071] 3. Target surgical site marking algorithm

[0072] Based on the preceding feature analysis results, a Conditional Random Field (CRF) model was used for target surgical site labeling. The CRF model can consider spatial relationships and contextual information between features to optimize the labeling results. First, the fused features were used as input to the CRF model, and a suitable energy function was defined, including a feature-based univariate potential (reflecting the probability that each voxel or patch belongs to the target surgical site, determined based on previously calculated feature values) and a spatial relationship-based binary potential (encouraging adjacent voxels or patches to have similar labels; for example, adjacent tissues belonging to the same organ should have the same label). Then, by solving the maximum a posteriori probability problem of the CRF model, the optimal labeling result was obtained, accurately marking the boundaries and extent of the target surgical site. For example, in lung surgery, it can accurately mark the safe resection range of the lung tumor and its surrounding area.

[0073] (2) Robotic arm path planning

[0074] 1. Environmental Modeling and Collision Detection

[0075] The robot's vision system (such as binocular or trinocular vision cameras) acquires environmental images within the operating room, and this information is combined with depth information from LiDAR to construct a 3D point cloud model of the surgical environment. Simultaneously, the robot's own geometric model (including the robotic arm, torso, and movement mechanisms) is also converted into a point cloud. By performing pose estimation and collision detection on the robot's point cloud model within the environmental point cloud model, the feasible motion space for the robot in the current environment is determined. For example, an octree-based collision detection algorithm is used, dividing the environmental and robot point clouds into octree structures of different resolutions. By rapidly traversing the octree nodes, the algorithm determines whether there is a collision risk between the robot and the environment.

[0076] 2. Path planning algorithm

[0077] The Rapid Exploratory Random Tree (RRT) algorithm and its improved versions (such as RRT*) are employed for robotic arm path planning. First, a search space is constructed between the robotic arm's starting position and the target surgical site (determined based on the marking results), randomly sampling points from the point cloud. Then, starting from the starting position, the sampled points are connected to the nearest node in the tree by continuously expanding the tree structure, while simultaneously checking for collisions with the environment. During the tree expansion process, the RRT* algorithm optimizes the path by reconnecting nodes to find shorter and safer paths. For example, in orthopedic surgery, to accurately deliver surgical instruments to the bone lesion site, the RRT* algorithm can quickly find a collision-free robotic arm movement path in complex surgical environments (including the patient's body, surgical equipment, etc.), and after finding a feasible path, it continues to optimize the path, reducing the distance and time of the robotic arm movement and improving surgical efficiency.

[0078] 3. Path optimization and security verification

[0079] For the initially planned robotic arm path, spline curve interpolation is used for smoothing. For example, cubic spline curves are used to interpolate the path points, making the robotic arm's movement smoother and more continuous, reducing joint impact and wear. Simultaneously, dynamic factors during the surgical process are considered, such as the patient's breathing movements and minute body displacements caused by heartbeat, and the path is dynamically adjusted. By acquiring real-time physiological monitoring data of the patient (such as respiratory rate and heart rate), the body's movement trends are predicted, and the robotic arm path is adjusted in advance to ensure that the surgical instruments are always accurately aligned with the target surgical site.

[0080] Before path execution, a comprehensive safety verification is performed. In addition to collision detection, factors such as the forces acting on the robotic arm and the limitations of joint range of motion are considered. Using a robot dynamics model, the force distribution on the robotic arm during path execution is calculated to ensure that it does not exceed the robotic arm's load-bearing capacity. At the same time, the motion trajectory of each joint is checked to ensure that it is within its allowable range of motion, avoiding mechanical failure or surgical risks due to excessive movement.

[0081] To this end, this invention innovatively fuses multi-scale geometric features (extracted through wavelet transform) and deep learning features (learned using 3D CNN) for the analysis and labeling of target surgical sites. This fusion approach combines the precise description of local structures by traditional geometric features with the powerful learning ability of deep learning features for global semantic information, enabling more comprehensive and accurate identification of complex anatomical structures and lesion features. Compared with single feature analysis methods, it significantly improves the accuracy and reliability of target surgical site labeling. For example, in brain tumor surgery, it can more accurately locate small tumors and their relationship with surrounding important neural structures, providing more precise guidance for the surgery.

[0082] Furthermore, by integrating an anatomical knowledge database and utilizing deep learning models to analyze three-dimensional physiological structure models, a collaborative approach combining knowledge-driven and data-driven methods is achieved. The anatomical knowledge database provides prior information on anatomical structures and disease patterns to guide feature analysis and target localization, while the deep learning model learns individual differences and potential characteristics from actual patient data. This collaborative approach can better handle anatomical variations and complex disease conditions in different patients, improving the adaptability and accuracy of surgical planning. For example, in treating rare congenital heart disease surgeries, anatomical knowledge can be used to quickly locate key structures, while deep learning models can be used to discover patient-specific disease features, leading to more personalized surgical plans.

[0083] Finally, regarding robotic arm path planning, the innovation of this technology lies in combining real-time environment modeling, dynamic factor consideration, and advanced path planning algorithms. A dynamic environment model is constructed using a vision system and LiDAR to achieve real-time perception and collision detection of the surgical environment, ensuring the safe movement of the robotic arm. The RRT* algorithm and its improved version are used for path planning, combined with spline curve interpolation and smoothing, enabling the rapid finding of the optimal path in complex dynamic environments. Simultaneously, the dynamic adjustment of the patient's physiological movements is considered, allowing the robotic arm to accurately track the target surgical site, improving the precision and efficiency of surgical operations. For example, in minimally invasive abdominal surgery, even if the patient's breathing causes changes in the position of abdominal organs, the robotic arm can adjust its path in real time to ensure that surgical instruments stably act on the target site.

[0084] Optionally, when establishing the mapping relationship between the robotic arm path and the patient's location to generate the robotic arm's motion trajectory sequence, the generative surgical large model includes the following process;

[0085] (1) Establish the mapping relationship between the robotic arm path and the patient's position.

[0086] 1. Coordinate system establishment and registration

[0087] First, a unified coordinate system is established in the surgical space, including a robot coordinate system (with the robot base or a fixed point as the origin), a patient coordinate system (usually with specific anatomical landmarks on the patient's body or preoperative positioning marks as the origin), and an image coordinate system (based on image data acquired preoperatively or intraoperatively). Using high-precision positioning equipment (such as optical trackers and electromagnetic locators), the position and posture information of the robot and patient in space are measured, and the coordinate systems of the three are registered. For example, in neurosurgery, an optical tracker is used to track markers mounted on the patient's head and markers on the robot's robotic arm, obtaining their coordinates in space, thereby determining the position and orientation of the patient's head in the robot coordinate system, achieving precise coordinate system registration.

[0088] For the registration of the patient coordinate system and the image coordinate system, a feature point matching method is adopted. Specific anatomical feature points (such as skeletal landmarks, key points of neural structures, etc.) are identified in the preoperative or intraoperative imaging data (such as CT, MRI, etc.). At the same time, the corresponding actual location points are obtained on the patient's body through physical marking or real-time scanning. By calculating the correspondence between these points in the two coordinate systems, the transformation from the image coordinate system to the patient coordinate system is realized, ensuring that the anatomical structure information in the imaging data accurately corresponds to the actual body position of the patient.

[0089] 2. Construction of Spatial Transformation Model

[0090] Based on the registered coordinate system, a spatial transformation model is constructed to describe the mapping relationship between the robotic arm path and the patient's position. A homogeneous transformation matrix is ​​used to represent the transformation from the patient coordinate system to the robot coordinate system. This matrix contains translation and rotation information; by multiplying the coordinates of a point in the patient coordinate system by this homogeneous transformation matrix, it can be transformed into the robot coordinate system.

[0091] To update the spatial transformation relationship in real time, the system continuously acquires position and pose data of the robot and patient from the positioning device. A Kalman filter algorithm is used to process this data. This algorithm can fuse measurements from multiple sensors to make an optimal estimate of the system's state (including the robot and patient's position and pose). By continuously updating the spatial transformation matrix, the mapping relationship between the robotic arm path and the patient's position is ensured to remain accurate throughout the surgery. Even if the patient moves slightly during the surgery due to breathing, heartbeat, or positional adjustments, the mapping relationship can be adjusted promptly.

[0092] (2) Generate the motion trajectory sequence of the robotic arm

[0093] 1. Trajectory Planning Algorithm

[0094] A fifth-order polynomial interpolation algorithm is used to plan the motion trajectory sequence of the robotic arm from the starting position to the target position (determined according to the robotic arm path).

[0095] The quintic polynomial interpolation algorithm ensures the continuity of position, velocity, and acceleration at the start and end points of the trajectory, making the robotic arm's movement smoother and reducing impact on surgical instruments and patient tissues. For example, in articulated robotic arms, each joint moves according to a planned quintic polynomial trajectory, and multiple joints work together to enable the robotic arm's end effector (such as surgical instruments) to move accurately and smoothly in space along a predetermined path.

[0096] 2. Kinematic and dynamic constraints considerations

[0097] When generating motion trajectory sequences, the kinematic and dynamic constraints of the robotic arm are fully considered. Using the robotic arm's geometric model and kinematic equations, the reachable space and joint angle range for each joint at different positions are calculated to ensure that the planned trajectory lies within the robotic arm's kinematically feasible domain. For example, for a robotic arm with rotary joints, the joint angles in the trajectory are checked to ensure they do not exceed the allowable range based on the joint's rotation range limitations.

[0098] Simultaneously, using the robotic arm's dynamic model, the torque and force required for each joint during trajectory movement are calculated. Employing an inverse dynamics algorithm, based on parameters such as the robotic arm's mass distribution, moment of inertia, joint friction, and the planned trajectory and speed, the control torque required for each joint to achieve the trajectory is calculated. By limiting and optimizing the torque, it is ensured that the robotic arm does not exceed its torque capacity during movement, preventing damage or loss of control due to overload. Simultaneously, energy consumption is optimized, improving the robotic arm's efficiency and stability.

[0099] 3. Real-time trajectory adjustment and optimization

[0100] The system monitors changes in the surgical environment and patient condition in real time, such as unexpected patient movement, tissue deformation, and force feedback from surgical instruments. When these changes are detected, the mapping between the robotic arm path and the patient's position is immediately reassessed, and the motion trajectory sequence is adjusted according to the new situation. For example, if the patient suddenly moves due to pain, causing a change in the target surgical site, the robotic arm's motion trajectory is quickly replanned based on the new patient position information to ensure that the surgical instruments continue to accurately target the site.

[0101] A model predictive control (MPC) approach is employed to optimize the trajectory. The MPC algorithm solves a finite-time optimization problem within each control cycle based on the current system state (including the robotic arm's position, velocity, and patient position) and the predictive model to determine the optimal control input (i.e., the motion commands for the robotic arm joints) over a future period. Through continuous optimization, the robotic arm can adapt to dynamically changing environments, improving the system's response speed and robustness while maintaining surgical precision. For example, in laparoscopic surgery, when the contact force between surgical instruments and tissue changes, the MPC algorithm can promptly adjust the robotic arm trajectory to maintain appropriate operating force and avoid tissue damage.

[0102] To this end, this invention achieves a real-time and accurate mapping relationship between the robotic arm path and the patient's position through high-precision positioning equipment and advanced coordinate system registration and spatial transformation model construction methods. By utilizing the Kalman filter algorithm to process and update the position and posture data in real time, the robotic arm can dynamically adapt to the patient's movements during surgery, ensuring that it remains accurately aligned with the target surgical site. This real-time dynamic adjustment capability is rare in traditional surgical robots, greatly improving the safety and accuracy of surgery, especially in cases of prolonged surgery or when the patient's condition is unstable.

[0103] Furthermore, the use of a fifth-order polynomial interpolation algorithm for trajectory planning, combined with comprehensive consideration of the robotic arm's kinematics and dynamics constraints, results in a smooth, accurate, and efficient sequence of motion trajectories. The fifth-order polynomial ensures high-order continuity of the trajectory at the start and end points, reducing shock and vibration during robotic arm movement and facilitating stable operation of surgical instruments. Simultaneously, by precisely calculating kinematic and dynamic parameters, the trajectory is ensured to remain within the robotic arm's feasible domain, and energy consumption and control torque are optimized, improving the robotic arm's performance and lifespan. This represents a significant improvement over traditional trajectory planning methods.

[0104] Finally, a model predictive control (MPC) approach was introduced for real-time trajectory adjustment and optimization, enabling the robotic arm to respond promptly to changes in the surgical environment and patient condition. The MPC algorithm makes optimization decisions based on the predictive model within each control cycle, considering the dynamic changes of the system over a future period, and effectively addressing uncertainties such as sudden patient movement and tissue deformation. This robust model-predictive control strategy improves the adaptability and reliability of the surgical robot system, providing strong support for the smooth execution of surgery and demonstrating an innovative application in the field of surgical robot control technology.

[0105] Optionally, when the generative surgical large model drives the moving mechanism to move toward the patient's position to a designated point according to the motion trajectory sequence, and controls the robotic arm to drive the surgical instruments to perform surgery on the target surgical site, the process includes the following:

[0106] (1) Drive the moving mechanism to move

[0107] 1. Path tracking control

[0108] Based on the generated motion trajectory sequence, the path of the mobile mechanism from its current position to the designated point is decomposed into a series of discrete path points. A vision-servoing-based path tracking algorithm is employed, utilizing the robot's vision system (such as a camera mounted on the mobile mechanism) to acquire real-time image information of the surrounding environment, identify feature points along the path (such as markings on the operating room floor, specific landmarks, etc.), and calculate the positional deviation of the mobile mechanism relative to these feature points. For example, an image feature matching algorithm is used to find pre-defined path marker features in the image, calculate their coordinates in the image plane, and then, combined with the camera's calibration parameters, convert them into the actual positional deviation in the robot's coordinate system.

[0109] Based on positional deviation, a proportional-integral-derivative (PID) controller is used to calculate the speed and steering control commands for the moving mechanism. The PID controller adjusts the positional deviation in real time according to the set proportional gain, integral gain, and derivative gain, enabling the moving mechanism to quickly and accurately track the predetermined path. For example, if the moving mechanism deviates to the left in the forward direction, the PID controller will increase the speed of the right drive wheel and decrease the speed of the left drive wheel according to the magnitude and rate of change of the deviation, gradually returning the moving mechanism to the correct path. To improve the accuracy and stability of path tracking, fuzzy logic control is introduced on top of the PID controller. This dynamically adjusts the PID parameters based on the complexity of the environment and the motion state of the moving mechanism, adapting to different surgical scenarios.

[0110] 2. Obstacle Avoidance Strategy

[0111] The mobile mechanism is equipped with various sensors (such as lidar and ultrasonic sensors) to monitor obstacles in the surrounding environment in real time. Lidar constructs a three-dimensional point cloud map of the surrounding environment by emitting laser beams and receiving reflected light, enabling precise detection of the location, shape, and size of obstacles. Ultrasonic sensors are used for short-range obstacle detection, complementing lidar. For example, in an operating room, lidar can detect large obstacles such as operating tables and equipment carts, while ultrasonic sensors can detect small obstacles on the ground or nearby objects such as a person's legs.

[0112] When an obstacle is detected, the Dynamic Window (DWA) method is used for obstacle avoidance decision-making. The DWA algorithm searches for feasible velocity combinations in the velocity space (composed of the linear and angular velocities of the mobile mechanism), considering factors such as the mobile mechanism's current velocity, acceleration limits, and the position and shape of the obstacle. By evaluating the collision risk between the mobile mechanism's trajectory and the obstacle over a certain time period under each velocity combination, the optimal velocity command is selected and sent to the mobile mechanism's drive system. For example, if there is an obstacle ahead, the DWA algorithm will eliminate potential collision-prone velocity combinations in the velocity space, selecting a velocity that avoids the obstacle while still approaching the target path as closely as possible, allowing the mobile mechanism to safely and efficiently bypass the obstacle and continue moving towards the designated point.

[0113] 3. Positioning and attitude adjustment of the mobile mechanism

[0114] As the mobile mechanism approaches a designated location, a high-precision indoor positioning system (such as a positioning system based on ultra-wideband (UWB) technology) is used to accurately locate it. The UWB positioning system calculates the precise position coordinates of the mobile mechanism by measuring the distances between the mobile mechanism and multiple fixed base stations using triangulation principles. Simultaneously, an inertial measurement unit (IMU) is used to acquire the mobile mechanism's attitude information (such as pitch, roll, and yaw angles) to ensure that the mobile mechanism has an accurate position and attitude upon reaching the designated location. For example, during brain surgery, the mobile mechanism needs to be precisely positioned next to the operating table and maintain a suitable angle with the patient's head so that the robotic arm can accurately manipulate surgical instruments to reach the target surgical site in the brain.

[0115] Based on positioning and attitude information, the attitude of the mobile mechanism is adjusted by controlling its chassis adjustment mechanism (such as hydraulic or electric lifting devices, steering mechanisms, etc.) to achieve the optimal relative position between the mobile mechanism and the patient's body and the surgical environment. During the adjustment process, positioning and attitude data are continuously monitored, and feedback control algorithms are used to ensure the accuracy and stability of the adjustment, providing a stable basic platform for the operation of the robotic arm.

[0116] (2) Control the robotic arm to drive the surgical instruments.

[0117] 1. Joint space control

[0118] The robotic arm's motion trajectory sequence is converted into target position, velocity, and acceleration commands in joint space. Based on the robotic arm's kinematic model, the target angle that each joint needs to reach at each moment is calculated using inverse kinematics algorithms. For example, for a robotic arm with 6 degrees of freedom, given the target trajectory of the end effector (surgical instrument) in space, the corresponding angle change sequence of the 6 joints is calculated using inverse kinematics algorithms (such as geometric or numerical iteration methods), enabling the end effector to move along the predetermined trajectory.

[0119] A model-based joint space controller (such as a computational torque controller) is used to control the movement of the robotic arm joints. Based on the robotic arm's dynamic model, the computational torque controller calculates the control torque required for each joint to achieve the target trajectory. This controller considers factors such as the robotic arm's inertia, gravitational load, and friction. By accurately calculating the control torque, the robotic arm joints can quickly and accurately track the target trajectory while ensuring smooth and stable movement. For example, during bone drilling surgery, the robotic arm needs to precisely control the position and orientation of surgical instruments. The computational torque controller can adjust the joint control torque in real time based on changes in resistance during the drilling process to ensure the accuracy and depth of the drilling.

[0120] 2. Force feedback control

[0121] The robotic arm is equipped with force sensors (such as a six-dimensional force sensor mounted on the end effector) to measure the contact force and torque between surgical instruments and tissue in real time. The force sensors convert the measured force signals into electrical signals and transmit them to the system. Based on the force feedback information, the robotic arm's motion control strategy is adjusted. For example, during soft tissue surgery, if the surgical instruments apply excessive force to the tissue, it may cause tissue damage. Through force feedback control algorithms, the driving torque of the robotic arm joints is reduced, keeping the force exerted by the surgical instruments on the tissue within a safe range.

[0122] Force feedback control is achieved using an impedance control algorithm. Impedance control converts force feedback information into position or velocity adjustment commands for the robotic arm joints based on a predefined target impedance model (including parameters such as stiffness and damping). For example, when surgical instruments contact tissue, a suitable target impedance is set based on the tissue's elastic properties, allowing the robotic arm to maintain contact with the tissue while automatically adjusting its position and posture according to tissue deformation. This simulates the feel of manual operation by a surgeon, improving the flexibility and safety of surgical procedures.

[0123] 3. Surgical instrument operation control

[0124] For different types of surgical instruments (such as scalpels, forceps, and suture needles), the robotic arm is controlled to perform corresponding operations based on surgical needs and instrument characteristics. For example, when using a scalpel for cutting, the robotic arm is controlled to drive the scalpel to perform a cutting action with a precise trajectory and speed according to the planned cutting path and speed. For clamping instruments such as forceps, the opening and closing angles of the robotic arm joints are controlled to achieve precise clamping and release operations on tissues.

[0125] It communicates and controls the drive system of surgical instruments (such as electric motors and pneumatic devices). Based on the needs of the surgical procedure, it sends corresponding control signals (such as motor speed and torque commands, pneumatic valve on / off signals, etc.) to the surgical instrument drive system to realize the functional operation of the surgical instruments (such as cutting, clamping, suturing, etc.). Simultaneously, it monitors the working status of the surgical instruments in real time (such as the wear level of the blades, the magnitude of the clamping force, etc.) and adjusts or prompts the surgeon to replace instruments as needed.

[0126] Therefore, the innovation of this invention in mobile mechanism navigation lies in combining visual servo-based path tracking with a dynamic window-based obstacle avoidance strategy. By acquiring environmental information in real time through a vision system for path tracking, the accuracy and adaptability of the mobile mechanism's navigation in complex surgical environments are improved. Compared to traditional navigation methods based on preset paths or simple sensors, this approach better handles the dynamic changes in personnel and equipment within the operating room. Simultaneously, the dynamic window-based obstacle avoidance strategy enables efficient obstacle avoidance while ensuring the safety of the mobile mechanism, allowing it to quickly and flexibly reach designated locations and providing reliable mobile platform support for the smooth conduct of surgery.

[0127] Furthermore, in the field of robotic arm control, the combination of force feedback control and model-based joint space control is a significant innovation. Force feedback control enables the robotic arm to sense the forces between surgical instruments and tissues in real time. By simulating the surgeon's feel through impedance control algorithms, it achieves more flexible and safer surgical operations, effectively avoiding tissue damage caused by improper operation. Simultaneously, the model-based computational torque controller utilizes the robotic arm's dynamic model for precise torque calculation, combined with inverse kinematics solutions in joint space, ensuring the robotic arm can accurately track the predetermined trajectory, improving the accuracy and stability of surgical operations. Moreover, the concept of model predictive control can also be applied to robotic arm control. By predicting future motion states and force conditions, control strategies can be adjusted in advance, further enhancing the adaptability and robustness of the robotic arm in complex surgical environments.

[0128] Furthermore, the entire surgical system integrates multiple sensors (such as vision systems, force sensors, and positioning systems) to achieve multimodal perception of the surgical environment, patient status, and robotic arm movements. It can fuse this sensor data in real time for comprehensive analysis and decision-making, enabling coordinated control of the moving mechanism and the robotic arm. This multimodal perception and coordinated control approach allows the surgical system to automatically adjust the position and posture of the moving mechanism and the robotic arm's movements at different surgical stages and under complex conditions, improving the overall performance and intelligence of the surgical system and providing surgeons with more efficient, precise, and safe surgical assistance.

[0129] Optionally, the humanoid robotic surgical system further includes a positioning module and a vision module. The positioning module is used to locate the target surgical site during the surgery, and the vision module is used to visually track the robotic arm and the target surgical site during the surgery. When the large surgical model performs feature analysis on the three-dimensional physiological structure model to mark the target surgical site and formulate the robotic arm path accordingly, it performs the following steps: establishing spatial transformation relationships between the lesion image data and the positioning module, and between the positioning module and the vision module; performing feature analysis on the three-dimensional physiological structure model to mark the target surgical site; and formulating the robotic arm path for the target surgical site based on the spatial transformation relationship.

[0130] The specific implementation of the above steps to determine the robotic arm path includes the following process:

[0131] (1) Establish spatial transformation relationships

[0132] 1. Coordinate system calibration and registration

[0133] First, the coordinate systems involved in the positioning module, vision module, and image data are calibrated. For the positioning module (such as an optical positioning system or an electromagnetic positioning system), multiple calibration markers are placed at known spatial locations, and the detection values ​​of the positioning module at these markers are measured. Algorithms such as the least squares method are then used to calculate the transformation matrix between the positioning module's coordinate system and the robot's coordinate system. For example, a calibration frame is set up in the operating room, with several optical marker spheres distributed on it. The positioning module detects the positions of these marker spheres, and based on the known spatial coordinates and detected coordinates of the marker spheres, the translation and rotation relationships of the positioning module's coordinate system relative to the robot's coordinate system are determined.

[0134] For the vision module, a checkerboard calibration method or a circular calibration board method is used. The calibration board is placed within the surgical field of view, and the vision module captures multiple images of the calibration board from different angles. By identifying feature points on the calibration board (such as the corners of the checkerboard or the centers of the circular markers), and based on the pixel coordinates of these feature points in the images and their known coordinates in the world coordinate system (associated with the robot coordinate system), the intrinsic parameter matrix (including focal length, principal point coordinates, etc.) and extrinsic parameter matrix (describing the position and orientation of the vision module relative to the world coordinate system) of the vision module are calculated. This establishes the transformation relationship between the vision module coordinate system and the robot coordinate system.

[0135] For lesion imaging data, positioning markers worn by the patient before surgery (such as head frames or body surface markers with special markings) are used to identify the positions of these markers in the imaging data. At the same time, the spatial coordinates of these markers are acquired in real time during surgery through the positioning module, thereby establishing a connection between the imaging data coordinate system and the positioning module coordinate system. Then, through the established transformation relationship between the positioning module and the robot coordinate system, the imaging data coordinate system and the robot coordinate system are registered, and the spatial transformation relationship among the three is established.

[0136] 2. Spatial Transformation Model Construction and Optimization

[0137] Based on the calibration and registration results, a spatial transformation model is constructed. A homogeneous transformation matrix is ​​used to represent the transformation relationship between different coordinate systems, converting coordinates in the positioning module coordinate system to coordinates in the robot coordinate system, and converting coordinates in the vision module coordinate system to coordinates in the robot coordinate system, to facilitate unified position calculation and path planning. For example, let the coordinates of a point in the positioning module coordinate system be ((x... l ,y l ,z l ,1) T Its homogeneous transformation matrix with the robot coordinate system is (T) l If ), then the corresponding coordinates in the robot coordinate system are ((x) r ,y r ,z r ,1) T =T l *(x l ,y l ,z l ,1) T .

[0138] To improve the accuracy of spatial transformation, a Kalman filter algorithm is used to fuse and optimize the measurement data from the positioning and vision modules. The Kalman filter algorithm weights and fuses measurement data from different times based on the system's dynamic model (describing the motion state changes of the robot, positioning module, and vision module) and measurement model (describing the relationship between measurement data and system state), updating the spatial transformation matrix in real time. For example, when the positioning and vision modules simultaneously measure a target point, the Kalman filter algorithm calculates an optimal estimated coordinate based on the measurement accuracy of both modules and the reliability of historical data, thereby continuously correcting the spatial transformation relationship and reducing the impact of measurement errors and system noise on spatial positioning.

[0139] (2) Perform feature analysis and target surgical site marking on the three-dimensional physiological structure model.

[0140] 1. Multimodal feature extraction and fusion

[0141] A three-dimensional wavelet transform algorithm is used to decompose a three-dimensional physiological structure model at multiple scales, extracting local geometric features at different scales, such as mean curvature, Gaussian curvature, and shape index. Simultaneously, a deep learning model based on a three-dimensional convolutional neural network (3DCNN) is used to learn features from the model. A 3D CNN architecture containing multiple convolutional and pooling layers is constructed. The input is the three-dimensional physiological structure model data. Convolutional layers extract local features, and pooling layers reduce the data dimensionality. As the network depth increases, more global feature relationships are gradually learned. For example, in brain surgery, wavelet transform can highlight the complex folded structural features of brain tissue, while 3D CNN can learn the semantic features and relative positional relationships between different brain regions.

[0142] Geometric features extracted by wavelet transform and features learned by 3D CNN are fused in the fully connected layer of the network to obtain a comprehensive feature representation. This multimodal feature fusion approach enables more comprehensive and accurate identification of complex anatomical structures and lesion features, providing rich information for marking target surgical sites. For example, in liver surgery, the fused features can clearly show the location, size, shape, and relationship of liver tumors with surrounding blood vessels and bile ducts.

[0143] 2. Target localization based on anatomical knowledge

[0144] Access the built-in database of standard models of human anatomy and disease feature patterns, built upon authoritative medical textbooks, clinical experience, and extensive case data. Register and compare the 3D physiological structure model with the standard models in the database, calculating the differences between them. For example, in cardiac surgery, register the patient's 3D heart model with a standard model of a normal heart. By analyzing the areas of difference and combining this with the stored cardiac disease feature patterns in the database (such as the typical morphology of myocardial infarction areas and the geometric features of valvular heart disease), areas where lesions may exist can be quickly located.

[0145] Based on the surgical type and specific requirements input by the doctor through the human-machine interface, relevant features are further filtered and focused. For example, in the case of coronary artery bypass surgery, the focus will be on the stenosis or blockage of the coronary artery, as well as the surrounding vascular branches that can be used for bypass, thereby accurately marking the boundaries and extent of the target surgical site.

[0146] (3) Formulate the robotic arm path based on spatial transformation relationship

[0147] 1. Target surgical site coordinate transformation

[0148] Using established spatial transformation relationships, the coordinates of the marked target surgical site in the image data coordinate system or vision module coordinate system are converted to coordinates in the robot coordinate system. For example, by multiplying the previously calculated homogeneous transformation matrix from the image data coordinate system to the robot coordinate system by the coordinates of the target surgical site in the image data, the accurate position coordinates of the target surgical site in the robot coordinate system are obtained. This step ensures that the robotic arm path planning is based on the target position in the robot's actual workspace.

[0149] 2. Robotic Arm Path Planning Algorithm

[0150] The Rapid Exploratory Random Tree (RRT) algorithm and its improved versions (such as RRT*) are employed for robotic arm path planning. First, a search space is constructed between the robotic arm's starting position and the transformed target surgical site coordinates, randomly sampling points from the point cloud. Then, starting from the starting position, the sampled points are connected to the nearest nodes in the tree by continuously expanding the tree structure, while simultaneously checking for collisions with obstacles in the surgical environment (perceived in real-time by the vision and positioning modules). During the tree expansion process, the RRT* algorithm optimizes the path by reconnecting nodes to find shorter and safer paths. For example, in orthopedic surgery, to accurately deliver surgical instruments to the bone lesion site, the RRT* algorithm can quickly find a collision-free robotic arm movement path in complex surgical environments (including the patient's body, surgical equipment, etc.), and after finding a feasible path, it continues to optimize the path, reducing the distance and time of robotic arm movement and improving surgical efficiency.

[0151] 3. Path optimization and security verification

[0152] For the initially planned robotic arm path, spline curve interpolation is used for smoothing. For example, cubic spline curves are used to interpolate the path points, making the robotic arm's movement smoother and more continuous, reducing joint impact and wear. Simultaneously, dynamic factors during the surgical process are considered, such as the patient's breathing movements and minute body displacements caused by heartbeat, and the path is dynamically adjusted. By acquiring real-time physiological monitoring data of the patient (such as respiratory rate and heart rate), the body's movement trends are predicted, and the robotic arm path is adjusted in advance to ensure that the surgical instruments are always accurately aligned with the target surgical site.

[0153] Before path execution, a comprehensive safety verification is performed. In addition to collision detection, factors such as the forces acting on the robotic arm and the limitations of joint range of motion are considered. Using a robot dynamics model, the force distribution on the robotic arm during path execution is calculated to ensure that it does not exceed the robotic arm's load-bearing capacity. At the same time, the motion trajectory of each joint is checked to ensure that it is within its allowable range of motion, avoiding mechanical failure or surgical risks due to excessive movement.

[0154] To this end, this invention innovatively achieves precise registration and real-time optimization between the coordinate systems of the positioning module, the vision module, and the lesion image data. Through a meticulously designed calibration and registration method, combined with the Kalman filter algorithm for fusion and optimization of measurement data, spatial transformation relationships can be established accurately and in real-time during surgery, effectively solving the problem of transformation errors between different coordinate systems. This enables the large surgical model to accurately map the target surgical site information from the image data to the robot's actual workspace, providing a solid foundation for the precise operation of the robotic arm and significantly improving positioning accuracy compared to traditional surgical robots.

[0155] Furthermore, the use of 3D wavelet transform and 3D CNN deep learning models for multimodal feature extraction and fusion, combined with a target localization method based on anatomical knowledge, represents a significant innovation in target surgical site marking. This fusion approach fully leverages the advantages of both traditional geometric features and deep learning features, enabling more accurate identification of complex anatomical structures and lesion features. Furthermore, the assistance of anatomical knowledge makes the localization of target surgical sites more intelligent and accurate, particularly suitable for handling complex cases and situations with significant anatomical variations, thereby improving the reliability and adaptability of surgical planning.

[0156] Furthermore, in terms of robotic arm path planning, innovative applications are demonstrated by combining spatial transformation relationships and an improved RRT algorithm with dynamic environment perception and real-time adjustment optimization mechanisms. Through real-time perception of obstacles and patient body changes in the surgical environment by the positioning and vision modules, the RRT* algorithm is used to quickly find and optimize the robotic arm path. Simultaneously, the dynamic adjustment of patient physiological movements and path smoothing are considered, enabling the robotic arm to accurately and efficiently reach the target surgical site in complex and ever-changing surgical environments. This reduces surgical risks and improves the precision and smoothness of surgical operations, which is difficult to achieve with traditional surgical robot path planning.

[0157] Optionally, when the surgical large model performs feature analysis on the three-dimensional physiological structure model to mark the target surgical site and formulate the robotic arm path accordingly, it also performs the following steps: the designated point for surgery on the target surgical site, the depth of surgery, the control torque of the robotic arm, and the control strategy for the robotic arm to drive the surgical instruments to the designated point and reach the depth during the surgery.

[0158] Optionally, the following key steps are used to implement a control strategy for determining the designated point for surgery on the target surgical site, the depth of surgery, the control torque of the robotic arm, and the robotic arm's ability to move surgical instruments to the designated point and reach the specified depth during the surgery:

[0159] (1) Determine the specified location

[0160] 1. Site planning based on anatomical structure and surgical requirements

[0161] First, preliminary planning of designated points is conducted based on the built-in human anatomy database and surgical type knowledge base. For different surgeries, such as brain surgery and orthopedic surgery, the key locations that surgical instruments need to reach are determined according to the surgical objective and common operating procedures. For example, in brain tumor resection surgery, based on the location and size of the tumor and the distribution of surrounding important neural structures, combined with standard surgical approaches and operational requirements, several possible surgical instrument entry points and key operational points are determined. These points should facilitate access to the tumor while minimizing damage to normal brain tissue.

[0162] Then, a three-dimensional physiological structure model is used to determine the precise surgical sites. Through detailed analysis of the target surgical site and its surrounding tissues in the model, considering physical characteristics such as tissue hardness, elasticity, and vascular distribution, the initially planned sites are optimized and adjusted. For example, in bone surgery, if drilling or implantation is planned at a certain bone location, parameters such as bone density and strength are analyzed, combined with the course of surrounding blood vessels and nerves, to determine the safest and most effective operating site, avoiding damage to important structures.

[0163] Finite element analysis (FEA) was used to validate and optimize the selection of surgical sites. A three-dimensional physiological structure model was converted into a finite element model to simulate the stress distribution and deformation of surrounding tissues when surgical instruments were operated on at different sites. Based on the simulation results, the sites that minimized tissue stress and achieved the most reasonable deformation were selected as the final designated sites. For example, in soft tissue surgery, finite element analysis can predict the degree of traction and compression of surrounding soft tissues by surgical instruments at different sites, thereby determining the operating sites with the least impact on the tissue.

[0164] (2) Determine the surgical depth

[0165] 1. Image data and model analysis to assist in depth determination

[0166] Depth analysis is performed on lesion imaging data (such as CT and MRI), and image segmentation algorithms are used to accurately segment the target surgical site and its surrounding tissues from the images. Then, based on information such as the grayscale values ​​and texture features of different tissues in the images, the boundaries and extent of the lesion are determined, thereby determining the required surgical depth. For example, in tumor resection surgery, by analyzing the tumor's appearance in images, its depth and location within the tissue, as well as its boundary with surrounding normal tissue, are determined, providing a preliminary basis for the surgical depth.

[0167] By combining three-dimensional physiological structural models, the three-dimensional morphology and spatial relationships of tissues are considered. For lesions with complex shapes or irregular distributions, the model can provide more comprehensive spatial information, helping to more accurately determine the surgical depth. For example, in cardiac surgery, the three-dimensional structure of the heart is complex. Through the model, the depth of the lesion within the heart wall and its relationship with the internal chambers of the heart can be intuitively understood, thereby determining the appropriate surgical depth and avoiding penetration of the heart wall or damage to the internal structures of the heart.

[0168] Deep learning algorithms are used for depth prediction. Based on a large amount of surgical case data, a deep neural network model is trained. The input includes the patient's preoperative imaging data and 3D physiological structure model features, and the output is the predicted surgical depth. This model can learn the potential relationship between lesion features and surgical depth in different cases, providing a reference for determining the surgical depth. For example, in ophthalmic surgery, based on the patient's eye imaging data and model information, the neural network model can predict the appropriate depth to be achieved during cataract or retinal surgery.

[0169] (3) Calculate the control torque of the robotic arm

[0170] 1. Dynamic model construction and parameter identification

[0171] A precise dynamic model of the robotic arm is constructed, taking into account factors such as joint structure, link mass, moment of inertia, friction, and gravitational load. The parameters in the model are determined through a combination of theoretical calculations and experimental measurements. For example, the link mass and moment of inertia of the robotic arm can be theoretically obtained through CAD modeling and material density calculations. Then, by actually measuring the forces and accelerations acting on the robotic arm under different motion states, the theoretical parameters are corrected using algorithms such as the least squares method to obtain accurate dynamic model parameters.

[0172] During surgery, the robotic arm's joint position, velocity, and acceleration information, as well as the contact force information between the surgical instruments and tissue (measured by force sensors), are acquired in real time. This information is input into a dynamic model, and using dynamic principles such as the Newton-Euler equations or the Lagrange equations, the required control torque for each joint is calculated. For example, during tissue cutting surgery, when the surgical instruments encounter tissue resistance, the dynamic model calculates the additional joint control torque required to ensure that the surgical instruments operate according to the predetermined trajectory and force.

[0173] An adaptive control algorithm is employed to dynamically adjust the control torque. Because tissue characteristics and environmental conditions may change during surgery, the adaptive control algorithm automatically adjusts the parameters or control gain in the dynamic model based on real-time measured error signals (such as the deviation between the actual trajectory and the target trajectory, or the deviation between the actual contact force and the desired contact force). This allows the robotic arm to adapt to different surgical situations and maintain stable and accurate operation. For example, in soft tissue surgery, as tissue is removed, the tissue's hardness and elasticity may change; the adaptive control algorithm can adjust the control torque in a timely manner to ensure the operational precision of the surgical instruments.

[0174] (4) Develop a control strategy for the robotic arm to move surgical instruments to designated points and achieve the required depth.

[0175] 1. Integration of trajectory planning and control algorithms

[0176] A segmented trajectory planning method is employed, dividing the entire motion process of the robotic arm from its initial position to a designated point and reaching a predetermined depth into multiple stages. During the approach to the designated point stage, a vision-servoing-based trajectory planning algorithm is used. A vision module monitors the relative position of the robotic arm and the target surgical site in real time, adjusting the robotic arm's trajectory based on visual feedback to ensure accurate arrival at the designated point. For example, as the robotic arm approaches the target surgical site, the vision module continuously measures the deviation between the robotic arm's end effector and the target point. Based on this deviation information, a proportional-integral-derivative (PID) controller calculates the joint motion speed command, enabling the robotic arm to quickly and accurately approach the target point.

[0177] During the depth-reaching phase, force feedback control and position control algorithms are combined. Force sensors monitor the contact force between the surgical instruments and the tissue in real time. When the contact force reaches a certain threshold, it indicates that the surgical instruments have begun to contact the target tissue. At this point, the system switches to force control mode, controlling the advancement speed and force of the robotic arm according to the preset surgical depth and tissue characteristics, allowing the surgical instruments to slowly and steadily reach the predetermined depth. For example, during bone drilling surgery, when the drill bit contacts the bone surface, force feedback control adjusts the drill bit's feed speed and rotational torque based on the bone's hardness and the required drilling depth, ensuring accurate drilling depth without damaging surrounding tissues.

[0178] Model predictive control (MPC) algorithms are used to optimize and coordinate the entire motion process. Based on the robotic arm's dynamic model, current state, and future motion goals, the MPC algorithm solves a finite-time optimization problem in each control cycle, predicting the robotic arm's trajectory and control torque over a future period, and adjusting the current control strategy based on the prediction results. For example, the MPC algorithm can predict changes in tissue resistance and stress on the robotic arm joints that may be encountered during reaching a specified point and depth, adjusting the trajectory and control torque in advance to improve the accuracy and stability of the robotic arm's motion, while simultaneously optimizing energy consumption and surgical time.

[0179] Therefore, the innovation of this invention in determining the specified location and surgical depth lies in the comprehensive utilization of multi-source information, including anatomical structure databases, surgical type knowledge, imaging data, three-dimensional physiological structure models, and deep learning algorithms. This fusion of multi-source information allows for a more comprehensive and accurate consideration of various surgical factors, such as anatomical complexity, tissue characteristics, and lesion features. Compared to traditional surgical robots that rely on a single information source or simple experience-based judgment, this significantly improves the accuracy of determining the specified location and surgical depth, providing a more reliable guarantee for the successful implementation of the surgery.

[0180] Furthermore, a significant innovation lies in the calculation of control torque for robotic arms, which involves constructing an accurate dynamic model and combining it with adaptive control algorithms. By meticulously considering various physical characteristics of the robotic arm and acquiring real-time motion and contact force information, the control torque is calculated using dynamic principles, making the operation of the robotic arm more consistent with actual physical laws. The adaptive control algorithm can automatically adjust the torque according to dynamic changes during the surgical process, ensuring that the robotic arm can work stably and accurately under different tissue conditions. This improves the adaptability and operational precision of the robotic arm to complex surgical environments, which is difficult to achieve with traditional torque control methods based on experience or simple models.

[0181] Furthermore, the developed robotic arm control strategy integrates multiple advanced technologies such as visual servoing, force feedback control, and model predictive control, achieving optimized and precise control of the entire robotic arm movement process. Visual servoing provides high-precision position guidance during the approach to the target point, force feedback control ensures the safety and accuracy of the operation during the depth reaching stage, and model predictive control coordinates and optimizes the entire process. This enables the robotic arm to accurately reach the designated point and achieve the required depth according to the predetermined trajectory in complex and changing surgical environments, while improving the system's response speed and robustness, providing strong support for the refinement and intelligence of surgical operations.

[0182] Optionally, the humanoid robotic surgical system further includes a wireless module for receiving real-time physiological monitoring data and real-time medical imaging data from the patient via a constructed wireless channel, so that the surgical big model can analyze the real-time physiological monitoring data and real-time medical imaging data and generate surgical suggestion plans based on the analysis results.

[0183] Specifically, the key technologies involved in the operation of the aforementioned wireless module are as follows:

[0184] (1) Wireless module data reception and transmission

[0185] 1. Wireless communication protocols and security mechanisms

[0186] The wireless module employs high-speed, stable, and secure wireless communication protocols, such as Wi-Fi 6 (802.11ax) or 5G technology, ensuring rapid and reliable reception of real-time physiological monitoring data (such as heart rate, blood pressure, and blood oxygen saturation) and real-time medical imaging data (such as intraoperative ultrasound and endoscopic images). During communication, strict data encryption algorithms, such as Advanced Encryption Standard (AES), are implemented to guarantee the security and privacy of data transmission. For example, the physiological monitoring device worn by the patient transmits encrypted data to the wireless module via a Wi-Fi 6 network, preventing data theft or tampering during transmission.

[0187] To ensure data integrity and accuracy, the wireless module employs a data checksum and retransmission mechanism. Upon receiving data, it calculates the checksum and compares it with the checksum provided by the sender. If the checksums do not match, it immediately requests retransmission from the sender. Simultaneously, a data buffer is set up to temporarily store and sort the received data, ensuring that the data is transmitted in the correct order for subsequent processing. For example, when receiving intraoperative ultrasound image data, the wireless module checks each frame of data. If an erroneous frame is detected, it promptly requests the ultrasound equipment to retransmit, ensuring the continuity and integrity of the image data.

[0188] 2. Signal strength optimization and interference suppression

[0189] The wireless module is equipped with smart antenna technology, which can automatically adjust the antenna's direction and gain to optimize wireless signal strength. By monitoring signal quality indicators in real time (such as Received Signal Strength Indication (RSSI) and Signal-to-Noise Ratio (SNR), it automatically adjusts antenna parameters when the signal strength is weak or interfered with, ensuring a stable communication connection. For example, in an operating room where multiple electronic devices may generate electromagnetic interference, the smart antenna can adjust its direction to avoid interference sources while simultaneously increasing its gain to enhance the reception of weak signals.

[0190] Interference suppression algorithms, such as frequency domain interference cancellation or adaptive filtering, are employed to reduce interference from other wireless devices in the surrounding environment. Frequency domain interference cancellation analyzes the spectral characteristics of interfering signals and subtracts them from the received signal at the receiver, improving the quality of the useful signal. Adaptive filtering dynamically adjusts filter parameters based on real-time changes in the interfering signal, effectively filtering out interference components. For example, when other Wi-Fi or Bluetooth devices in the operating room cause interference, the wireless module utilizes these interference suppression algorithms to ensure stable reception of patient data.

[0191] (2) Data analysis and surgical suggestion planning generation of the large surgical model

[0192] 1. Real-time physiological monitoring data analysis

[0193] The received real-time physiological monitoring data is analyzed in real time. First, a dynamic model of the physiological parameters is established. For example, an autoregressive moving average (ARMA) model or a Kalman filter model is used to model data such as heart rate and blood pressure to predict the changing trends of physiological parameters. By comparing the actual measured values ​​with the predicted values, abnormal fluctuations in physiological parameters can be detected in a timely manner. For example, if a patient's heart rate suddenly deviates from the normal range and does not match the predicted trend, this abnormality can be detected quickly.

[0194] Machine learning algorithms are used to classify and assess the risks of physiological data. Based on a large amount of clinical case data, classification models such as Support Vector Machines (SVM), decision trees, or neural networks are trained to categorize physiological data into normal, mildly abnormal, moderately abnormal, and severely abnormal categories. Simultaneously, based on different abnormality categories and degrees, combined with the type of surgery and the patient's underlying condition, surgical risks are assessed. For example, in cardiac surgery, if blood pressure continues to drop and is classified as severely abnormal, the surgical model, considering the surgical progress and the patient's cardiac function, may determine that the patient is at risk of insufficient blood supply to the heart, requiring appropriate measures.

[0195] 2. Real-time medical image data analysis

[0196] For real-time medical image data, image segmentation and feature extraction algorithms are used for analysis. For example, deep learning-based image segmentation algorithms (such as the U-Net architecture) are used to segment different tissues and structures (such as tumors, blood vessels, and organs) in the image, and then the shape, size, texture, and other features of the segmented regions are calculated. In brain surgery, brain tumors and their surrounding important neural structures can be accurately segmented, and features such as tumor morphological changes and adhesion to surrounding tissues can be analyzed.

[0197] By comparing and analyzing real-time imaging features with preoperative imaging data and standard anatomical models, the progress of the surgery and the presence of any abnormalities can be assessed. For example, in orthopedic surgery, comparing intraoperative real-time X-ray images with the preoperatively planned bone and implant positions can detect whether bone repositioning is accurate and whether implant displacement has occurred. If abnormalities are found, the surgical strategy can be adjusted promptly.

[0198] 3. Surgical suggestion planning generation algorithm

[0199] Based on the analysis of real-time physiological monitoring data and real-time medical imaging data, the surgical big data model generates surgical suggestion plans using a rule-based reasoning system and a case-based reasoning system. The rule-based reasoning system directly generates response measures based on pre-set medical rules and expert experience, such as "when blood pressure falls below a certain threshold and persists for a certain duration, suspend the surgical procedure and take appropriate measures." The case-based reasoning system searches for similar cases in a historical surgical case database, drawing on past successful or unsuccessful experiences to provide reference suggestions for the current surgery. For example, if the current surgery encounters physiological data anomalies and imaging characteristics similar to a previous case, the surgical big data model can refer to the handling methods in that case and, combined with the current situation, formulate a personalized surgical suggestion plan, such as adjusting the operating parameters of surgical instruments, changing the surgical approach, or taking emergency hemostasis measures.

[0200] Therefore, the innovation of this invention in terms of the wireless module lies in the adoption of advanced wireless communication technology and security mechanisms, enabling high-speed, stable, and secure transmission of real-time patient physiological monitoring data and medical imaging data. The high-speed communication protocol ensures timely data delivery, meeting the real-time requirements during surgery; strict data encryption and verification mechanisms ensure data accuracy and privacy, which is crucial in the field of medical data transmission. Simultaneously, smart antennas and interference suppression technology improve the adaptability and reliability of the wireless module in complex surgical environments, effectively solving the problems of signal interference and instability, and demonstrating significant advantages over traditional wireless transmission methods.

[0201] Furthermore, the fusion analysis of real-time physiological monitoring data and medical imaging data in the large-scale surgical model is a significant innovation. By establishing dynamic models of physiological parameters and utilizing machine learning algorithms to classify and evaluate physiological data, combined with image segmentation and feature extraction techniques to analyze imaging data, a comprehensive and real-time assessment of the patient's surgical status is achieved. This multimodal data fusion analysis can more timely and accurately identify potential risks during surgery, providing doctors with more comprehensive decision-making support and helping to improve surgical safety and success rates, especially suitable for complex surgeries and the surgical management of high-risk patients.

[0202] Furthermore, in terms of generating surgical suggestion plans, the combination of rule-based reasoning and case-based reasoning demonstrates innovative applications. Rule-based reasoning leverages medical expert knowledge to ensure the standardization and accuracy in addressing common problems, while case-based reasoning can extract personalized experience from a large number of historical cases, providing reference for solving complex and rare surgical situations. This intelligent reasoning system can generate targeted surgical suggestion plans based on the real-time situation of the surgery, helping doctors make reasonable decisions quickly, improving the flexibility and adaptability of surgery, and representing an innovative development direction in the field of surgical decision support.

[0203] Optionally, the number of robotic arms is at least two, each robotic arm consists of multiple joints connected in series with more than 5 axes, and the end of each robotic arm forms a wrist, wherein the wrist of one robotic arm is equipped with a main surgical instrument, and the wrist of at least one of the remaining robotic arms is equipped with an auxiliary surgical instrument.

[0204] Specifically, regarding the structure and joint control of the robotic arm:

[0205] 1. Joint drive and kinematic modeling

[0206] Each joint of the robotic arm is equipped with a high-performance motor drive system, such as a brushless DC motor (BLDCM) or an AC servo motor, which provides precise torque control and fast response. The motor driver communicates with the motor, sending speed, position, and torque control commands. For each joint, a precise kinematic model is established, including forward and inverse kinematic models. The forward kinematic model is used to calculate the position and orientation of the robotic arm's end effector (wrist) in space based on joint angles, achieved through a series of coordinate transformation matrix multiplications. For example, for a 5-axis robotic arm, starting from the base joint, the coordinate transformations of each joint are calculated sequentially to obtain the position and orientation of the wrist in the robot coordinate system. The inverse kinematic model is used to calculate the joint angles based on the desired wrist position and orientation, typically using numerical iterative methods (such as the Newton-Raphson method) or analytical methods (such as geometric methods, for robotic arms with certain specific structures). For example, when moving the main surgical instrument to the target surgical site, the inverse kinematic model is used to calculate the required rotation angle of each joint, and then the motor driver controls the motor to rotate the corresponding angle.

[0207] To improve the precision and stability of joint control, a closed-loop control strategy is employed. Position sensors (such as encoders) and speed sensors (such as tachogenerators or speed calculated from encoder signals) are installed on each joint to provide real-time feedback of the joint's actual position and speed. This feedback information is compared with the target position and speed commands, and an error signal is calculated. Then, a PID (proportional-integral-derivative) controller or a more advanced controller (such as a model predictive controller) calculates the motor's control input and adjusts the motor's output torque, enabling the joint to quickly and accurately track the target motion trajectory. For example, during delicate surgical procedures such as suturing, the PID controller adjusts the motor torque based on the position error, ensuring the robotic arm can precisely control the position and orientation of surgical instruments, achieving minute and precise movements.

[0208] 2. Cooperative motion planning for robotic arms

[0209] When two or more robotic arms work together, complex motion planning is required. First, a 3D model of the entire surgical scene is created, including a model of the patient's body, surgical instruments, and other objects in the surgical environment. Based on the surgical task requirements, such as simultaneous tissue cutting and assisted traction, the task objectives and sequence of actions for each robotic arm are determined. For example, the robotic arm equipped with the main surgical instruments is responsible for precise tissue cutting, while the robotic arm equipped with auxiliary surgical instruments is responsible for traction of tissue at appropriate locations and times to expose the surgical field.

[0210] A priority-based task allocation and path planning algorithm is employed. Each robotic arm is assigned a priority based on the urgency and importance of the surgical task. During path planning, paths are first planned for high-priority robotic arms to ensure their successful completion of the main surgical procedures. Simultaneously, the movement space and operational requirements of lower-priority robotic arms are considered to avoid collisions. For example, in brain surgery, if the primary surgical instrument needs to quickly reach and remove a tumor deep within the brain, its path planning has the highest priority. The paths of the auxiliary robotic arms are planned without interfering with the operation of the primary surgical instrument, and may require waiting for the primary surgical instrument to reach a certain position before taking action to avoid the risk of collision.

[0211] Collision detection and avoidance algorithms ensure safety between robotic arms and between the robotic arms and the surgical environment. By monitoring the position and motion of the robotic arms and surrounding objects in real time, collision detection algorithms based on bounding boxes (such as axis-aligned bounding boxes (AABB) or oriented bounding boxes (OBB)) are used to quickly determine the existence of collision risks. If a potential collision is detected, the movement path or speed of the robotic arms is adjusted in a timely manner. For example, when two robotic arms are operating in a confined space, once a potential collision is detected, a pre-set collision avoidance strategy will be implemented, such as reducing the movement speed, adjusting the movement direction, or pausing the movement of one robotic arm, to ensure the safe conduct of the surgery.

[0212] Specifically, regarding the installation and operation control of surgical instruments:

[0213] 1. Surgical instrument quick change and interface design

[0214] The robotic arm's wrist is designed with a standardized surgical instrument interface for quick and easy switching between different types of primary and auxiliary surgical instruments. The interface employs a combination of mechanical locking and electrical connection to ensure secure instrument mounting and stable electrical signal transmission. For example, the interface features precise positioning pins and bayonet structures that automatically lock and align electrical contact points when an instrument is inserted, enabling the transmission of motor drive signals and sensor signals (such as force sensor signals). By recognizing specific codes or sensor signals on the interface, the type of surgical instrument installed is automatically identified, and the corresponding control program and parameters are loaded. For instance, when a scalpel is installed, a control algorithm suitable for scalpel operation is invoked, adjusting motor torque and speed limits to meet the needs of tissue cutting; when forceps are used, the control mode switches to one suitable for gripping operations.

[0215] 2. Main surgical instrument operation control algorithm

[0216] For the operation of main surgical instruments, specialized control algorithms are employed based on the type of surgery and the characteristics of the instruments. Taking electrosurgical instruments as an example, such as high-frequency electrosurgical units, precise control of the output power, frequency, and waveform is required to achieve tissue cutting, coagulation, or mixed-mode operation. By real-time monitoring of parameters such as the contact force between the surgical instrument and the tissue (using a force sensor on the wrist) and tissue impedance (by measuring the current and voltage relationship between the electrosurgical electrode and the tissue), feedback control algorithms adjust the output parameters of the electrosurgical unit. For example, when cutting tissue, if the contact force increases, indicating that the tissue is harder or the cutting depth is increased, the electrosurgical power will be appropriately increased to ensure the cutting effect. At the same time, based on changes in tissue impedance, the tissue type (such as muscle, fat, blood vessels, etc.) is determined, and the electrosurgical frequency is automatically adjusted to optimize the cutting and coagulation effects and reduce thermal damage to surrounding tissues.

[0217] In minimally invasive surgeries, such as laparoscopic surgery, the main surgical instruments need to be operated precisely within a confined space, while overcoming movement limitations when passing through the trocar opening. Utilizing the flexibility and joint control precision of the robotic arm, combined with visual feedback (obtained through a laparoscopic camera capturing the surgical field of view), a visual servo-based control algorithm is employed. By identifying surgical targets and anatomical landmarks in the image, the robotic arm's movement deviation is calculated, and then the joint angles are adjusted to ensure the main surgical instruments accurately reach the target position and are operated upon. For example, when suturing intra-abdominal tissue, based on the relative position of the suture needle and tissue in the laparoscopic image, the robotic arm is controlled to move the surgical instruments with precise suturing movements, ensuring suture quality.

[0218] 3. Strategies for controlling the operation of surgical instruments

[0219] The operation and control of auxiliary surgical instruments aims to assist the primary surgical instrument in completing surgical tasks, improving surgical efficiency and safety. For example, when the auxiliary surgical instrument is a traction forceps, the opening and closing angle and traction force of the forceps are controlled according to the surgical progress and the operational requirements of the primary surgical instrument. By real-time monitoring of the surgical field of view (e.g., through a visual module) and the position of the primary surgical instrument, the appropriate traction position and direction are determined. For instance, in orthopedic surgery, when the primary surgical instrument is drilling into the bone, the auxiliary surgical instrument is responsible for tractioning the surrounding soft tissue to prevent it from interfering with the drilling operation. Based on the drilling location and the distribution of soft tissue, the position and force of the traction forceps are automatically adjusted to ensure the smooth progress of the surgery.

[0220] For some specialized surgical instruments, such as suction devices or irrigators, their operation is controlled according to surgical needs. For example, if bleeding occurs during surgery, the suction device is automatically activated based on signals from blood detection sensors (such as optical sensors that detect changes in blood color and concentration), adjusting its suction power to promptly remove blood from the surgical field and ensure a clear surgical view. Simultaneously, the irrigator's flow rate and direction are controlled according to the cleaning needs of the surgical site and tissues to maintain the cleanliness of the surgical area, which facilitates surgical procedures and reduces the risk of infection.

[0221] Therefore, the innovation of this invention in robotic arm control lies in achieving high-precision motion control of multi-joint robotic arms with 5 or more axes. By employing high-performance motor drives, precise kinematic modeling, and closed-loop control strategies, combined with advanced controller algorithms, precise position and posture control of the robotic arm's end effector within complex surgical spaces can be achieved. Compared to traditional robotic arms, this high-precision control allows surgical instruments to reach the target surgical site more accurately, enabling more delicate surgical operations, such as manipulation of minute neural structures in neurosurgery, thus improving the success rate and safety of the surgery.

[0222] Furthermore, the collaborative operation of multiple robotic arms is one of the key innovations of this technology. By establishing a surgical scenario model, priority-based task allocation and path planning algorithms, and collision detection and avoidance algorithms, efficient and safe collaborative operation of multiple robotic arms within a confined surgical space is achieved. This collaborative mode can simulate the cooperation between different members of a surgical team, fully utilize the function of each robotic arm, and improve surgical efficiency. For example, in complex abdominal surgeries, multiple operations can be performed simultaneously, reducing surgical time and patient risk, which is difficult to achieve with traditional single-robotic-arm surgical systems.

[0223] Furthermore, the standardized surgical instrument interface design and adaptive control algorithm of the robotic arm wrist embody innovative applications. The quick-change interface facilitates the timely replacement of appropriate surgical instruments according to different surgical stages and needs, improving the flexibility of the surgical system. Simultaneously, the adaptive control algorithm for different types of surgical instruments can automatically adjust control parameters based on instrument characteristics and surgical operation requirements to achieve optimal surgical outcomes. Whether it's the precise energy control of electrosurgical instruments or the visual servo operation of minimally invasive surgical instruments, it provides surgeons with more powerful and convenient surgical tools, contributing to the advancement of surgical techniques.

[0224] Optionally, the torso is a humanoid torso with at least two degrees of freedom to perform at least one of the following during the surgery: forward and backward bending, left and right lateral bending, and height self-adjustment, in accordance with the surgical operation.

[0225] Therefore, regarding degrees of freedom and kinematic models:

[0226] 1. Joint Actuation and Sensor Configuration

[0227] Each degree of freedom of the humanoid torso is achieved by a corresponding drive device, such as an electric motor or a hydraulic / pneumatic actuator. These drives communicate with the motor driver or control valve, sending control signals to achieve precise motion control. At each joint of freedom, position sensors (such as rotary encoders or linear displacement sensors) and angle sensors (such as gyroscopes or tilt sensors) are installed to provide real-time feedback on the torso's position, angle, and motion status. For example, at the joints that allow for forward and backward bending, a rotary encoder can accurately measure the joint's rotation angle, while a gyroscope can detect angular velocity and posture changes during bending. This sensor data provides accurate feedback for closed-loop control.

[0228] For the degrees of freedom driven by electric motors, high-performance servo motors are used, which possess high-precision position control and rapid response capabilities. The target position or angle to be achieved for each degree of freedom is calculated based on the surgical operation requirements. Then, a PID (Proportional-Integral-Derivative) controller or Model Predictive Controller (MPC) is used to calculate the control input (such as voltage or current) to the motor, enabling the motor-driven joint to move accurately to the target position. For example, during brain surgery, when the surgeon needs to adjust the height and angle of surgical instruments, the height adjustment and bending degrees of freedom of the human torso are controlled according to the surgical plan and the surgeon's instructions. The motor output is precisely adjusted through a PID controller to ensure rapid and stable adjustment of the torso posture, allowing the surgical instruments to accurately reach the target surgical site in the brain.

[0229] 2. Kinematic Model Establishment and Solution

[0230] To accurately control the movement of the humanoid torso, a kinematic model is established. For rotational degrees of freedom such as forward and backward bending and left and right lateral bending, a rotation matrix method based on joint space is used to establish the kinematic model. For example, for a simple humanoid torso model with two rotational degrees of freedom (forward and backward bending and left and right lateral bending), by multiplying the rotation matrices of each degree of freedom, the position and posture transformation relationship of the torso end effector (such as the mounting position of the robotic arm) in space can be obtained. Let R... x (θ x Let θ be the rotation about the x-axis (the axis of forward and backward bending). x The rotation matrix of the angle, R y (θ y ) is a rotation θ about the y-axis (left and right lateral bending axis). y Given the rotation matrix of the angle, the total transformation matrix from the initial torso pose to the current pose is T = R. x (θ x )R y (θ y This model allows us to solve for the joint angles required for each degree of freedom based on the target position and orientation, thus achieving inverse kinematics solution.

[0231] For height self-adjustment degrees of freedom (if it is linear motion), a model based on linear kinematics is established. Depending on the type of drive device (e.g., lead screw and nut mechanism or hydraulic / pneumatic telescopic rod), the relationship between height and the drive device's motion parameters (e.g., motor speed or hydraulic rod stroke) is determined. For example, for a lead screw and nut driven height adjustment mechanism, given the lead screw pitch p, the relationship between the motor speed n and the height change h is h = pn. The target height is calculated based on surgical requirements, and then precise height adjustment is achieved by controlling the motor speed. In practical applications, to improve the smoothness and accuracy of motion, it may be necessary to optimize the kinematic model, considering the effects of gravity, friction, etc. For example, a gravity compensation term can be added when calculating the motor control input to ensure stable height adjustment under different load conditions.

[0232] Specifically, regarding the matching algorithm with surgical procedures:

[0233] 1. Pose planning based on surgical scenarios

[0234] Surgical scene information, including patient position, instrument position, and operating direction, is acquired through visual modules (such as cameras installed in the operating room or the robot's own vision system) and position sensors for surgical instruments. Based on the type of surgery and the operational requirements of the current stage, optimal posture planning for the humanoid torso is developed. For example, in abdominal surgery, when the surgeon needs to operate on organs deep within the abdominal cavity, the required forward and backward bending angles and height adjustments for the humanoid torso are calculated based on the length of the surgical instruments, the depth of the surgical site, and the patient's position. This allows the robotic arm to approach the surgical site at the optimal angle and distance, while avoiding collisions with the patient or surgical equipment.

[0235] Artificial intelligence-based path planning algorithms (such as reinforcement learning) are employed to optimize the motion trajectory of the humanoid torso. The surgical scene is treated as a dynamic environment, and the torso's posture adjustments are considered a series of actions. Through reinforcement learning, the robot learns the optimal posture adjustment strategy in numerous simulated surgical scenarios. For example, a reward function is defined: a positive reward is given when the robotic arm accurately and quickly reaches the target surgical site while remaining safe throughout the process (e.g., without collisions); conversely, a negative reward is given. After repeated training, the robot can automatically select the optimal torso posture adjustment path according to different surgical scenarios, improving surgical efficiency and safety.

[0236] 2. Real-time feedback and dynamic adjustment

[0237] During surgery, the movement of the human torso and the feedback from surgical instruments are monitored in real time. Sensor data (such as joint position sensors and force sensors) and instrument status information (such as instrument force and cutting depth) are used to determine if the current torso posture meets the surgical requirements. If deviations or adjustments are detected, dynamic adjustments are made immediately. For example, in delicate ophthalmic surgery, if the surgeon needs to slightly change the angle of the surgical instruments, the required minute posture adjustment of the human torso is quickly calculated based on the surgeon's fine-tuning instructions input through the human-machine interface or the detected operational intent based on changes in instrument force. By controlling the corresponding degrees of freedom, real-time dynamic adjustments are achieved to ensure that the surgical instruments are always maintained in the optimal operating position and angle.

[0238] Dynamic factors during surgery, such as the patient's breathing and subtle body movements, are considered, and the posture of the humanoid torso is adjusted accordingly. By acquiring real-time physiological monitoring data of the patient (such as respiratory rate and heart rate) and monitoring the patient's body position through a visual module, the patient's movement trends are predicted, and the posture of the humanoid torso is adjusted in advance to ensure the relative stability of the surgical instruments and the surgical site. For example, in chest surgery, the chest rises and falls when the patient breathes. Based on the respiratory cycle and amplitude, the height and angle of the humanoid torso are dynamically adjusted so that the robotic arm can adjust accordingly with the patient's breathing movements, ensuring that the surgical instruments always accurately target the surgical site.

[0239] Therefore, the innovation of this invention in humanoid torso control lies in achieving precise motion control of at least two degrees of freedom. Through advanced drive devices, high-precision sensors, and accurate kinematic models, the humanoid torso can accurately perform movements such as forward and backward bending, left and right lateral bending, and height self-adjustment during surgery. This precise control capability allows the robot to better adapt to different surgical sites and operational needs. Compared with traditional surgical aids with fixed structures or simple motion mechanisms, it significantly improves the accessibility and operational flexibility of surgical instruments, providing surgeons with a wider operating space and a more comfortable operating angle, thus contributing to improved surgical accuracy and efficiency.

[0240] Furthermore, utilizing visual modules and surgical instrument sensor information to achieve posture planning and real-time dynamic adjustment based on the surgical scenario is a significant innovation. By acquiring comprehensive information about the surgical scenario and combining it with artificial intelligence algorithms (such as reinforcement learning), the robot can automatically plan the optimal torso posture according to the actual surgical situation and adapt to various changes in real time during the operation, including changes in patient position, changes in surgical instrument operation requirements, and dynamic movements of the patient's body. This intelligent posture adjustment capability improves the robot's adaptability and autonomy in complex surgical environments, enabling it to better cooperate with surgeons in surgical operations, reduce the physical exertion and operational difficulty for surgeons during the operation, and improve the safety and success rate of the surgery.

[0241] Furthermore, the technology demonstrates innovative application in compensating for dynamic factors during surgery (such as patient breathing) by adjusting the posture of the human torso. By monitoring the patient's physiological data and changes in body position in real time, it predicts and compensates for surgical site displacement caused by the patient's dynamic movements, ensuring that the relative position of surgical instruments and the surgical site remains stable. This technology is significant in improving surgical precision, especially in surgeries requiring extremely high precision (such as neurosurgery and ophthalmology), effectively reducing surgical errors caused by patient movement and improving surgical quality—a function that traditional surgical aids struggle to achieve.

[0242] Optionally, the mobile mechanism is at least one of a humanoid legged, wheeled, or tracked mobile mechanism.

[0243] Specifically, for humanoid legged mobility mechanisms:

[0244] 1. Gait planning and control

[0245] The gait of the humanoid leg-foot-based mobility mechanism is planned based on the surgical scenario and patient location information. First, a kinematic model of the leg is established, including the position and angle of each joint (such as the hip, knee, and ankle joints) and their relationship to the overall leg movement. Using inverse kinematics algorithms, the required angle change sequence for each joint is calculated based on the target movement position and posture. For example, using a geometry-based inverse kinematics method, for a simple planar leg model, the angles of the hip, knee, and ankle joints are calculated based on the leg length and the target foot placement position.

[0246] A gait planning algorithm based on Model Predictive Control (MPC) is employed. The MPC algorithm predicts the leg trajectory for the next few steps within each control cycle based on the current leg state (including joint angles, velocity, acceleration, etc.) and environmental information (such as ground flatness, obstacle positions, etc.). It then selects the gait sequence that optimizes certain performance indicators (such as minimum energy consumption and maximum movement stability) through an optimization algorithm (such as quadratic programming). For example, in an operating room with a relatively flat floor but containing small obstacles such as equipment cables, the MPC algorithm considers avoiding these obstacles during gait planning, while adjusting stride length and stride frequency to minimize energy consumption and ensure stable movement.

[0247] During gait execution, sensors (such as encoders, gyroscopes, and force sensors) installed in the leg joints provide real-time feedback on the joint's actual movement and force conditions. This feedback information is compared with the planned gait, and a PID (proportional-integral-derivative) controller or adaptive controller adjusts the output torque of the joint motors to ensure the leg moves accurately according to the planned gait. For example, if there is a deviation between the actual joint angle and the target angle, the PID controller adjusts the motor torque based on the magnitude and rate of change of the deviation, allowing the joint to quickly return to the target angle. When encountering uneven ground that causes changes in leg force, the adaptive controller adjusts the motor parameters based on force sensor feedback to ensure stable leg support and movement.

[0248] 2. Balance control and environmental adaptation

[0249] To ensure the humanoid legged mobility mechanism maintains balance during movement, a sensor fusion-based balance control algorithm is employed. This algorithm integrates data from multiple sensors, including gyroscopes, accelerometers, and a vision module, to calculate the mechanism's posture and center of gravity changes in real time. For example, an extended Kalman filter algorithm is used to fuse gyroscope and accelerometer data for more accurate posture information. Simultaneously, the vision module's perception of the surrounding environment and ground helps determine if any tilting or unevenness exists.

[0250] The compensation movements for each leg joint are calculated based on the balance state. When forward leaning is detected, the center of gravity is shifted backward by adjusting the leg joint angles to maintain balance. A hierarchical control structure is employed: the upper layer calculates the desired angle changes of the joints based on overall balance requirements, while the lower layer uses joint controllers (such as computational torque controllers) to precisely control the motors, moving the joints to the desired angles. For example, when crossing small obstacles, the upper layer control plans the sequence of leg lifting and crossing movements based on the obstacle's height and position, while the lower layer control ensures precise movement of the leg joints during execution, while maintaining body balance, enabling the mobile mechanism to smoothly pass through obstacles and reach the designated surgical position.

[0251] Specifically, for wheeled mobile mechanisms:

[0252] 1. Path tracking and speed control

[0253] The target path of the wheeled mobile mechanism is determined based on the surgical plan, and the path is decomposed into a series of discrete path points. A path tracking algorithm based on visual servoing is employed, using a camera mounted on the mobile mechanism to acquire images of the surrounding environment, identify feature points in the path (such as ground markings, specific landmarks, etc.), and calculate the positional deviation of the mobile mechanism relative to these feature points. For example, an image feature matching algorithm is used to find pre-defined path marker features in the image, calculate their coordinates in the image plane, and then combine them with the camera calibration parameters to convert them into the actual positional deviation in the robot coordinate system.

[0254] Based on positional deviation, a proportional-integral-derivative (PID) controller is used to calculate the speed and steering control commands for the wheeled traverse mechanism. The PID controller adjusts the positional deviation in real time according to the set proportional gain, integral gain, and derivative gain, enabling the traverse mechanism to quickly and accurately track the predetermined path. For example, if the traverse mechanism deviates to the left in the forward direction, the PID controller increases the speed of the right drive wheel while decreasing the speed of the left drive wheel based on the magnitude and rate of change of the deviation, gradually returning the traverse mechanism to the correct path. To improve the accuracy and stability of path tracking, fuzzy logic control is introduced on top of the PID controller. This dynamically adjusts the PID parameters according to the complexity of the environment and the motion state of the traverse mechanism, adapting to different surgical scenarios.

[0255] 2. Obstacle Avoidance Strategies and Precise Positioning

[0256] Wheeled mobile mechanisms are equipped with various sensors (such as lidar and ultrasonic sensors) for real-time monitoring of obstacles in the surrounding environment. Lidar constructs a three-dimensional point cloud map of the surrounding environment by emitting a laser beam and receiving the reflected light, enabling precise detection of the location, shape, and size of obstacles. Ultrasonic sensors are used for short-range obstacle detection, complementing lidar. For example, in an operating room, lidar can detect large obstacles such as operating tables and equipment carts, while ultrasonic sensors can detect small obstacles on the ground or nearby objects such as a person's legs.

[0257] When an obstacle is detected, the Dynamic Window (DWA) method is used for obstacle avoidance decision-making. The DWA algorithm searches for feasible velocity combinations in the velocity space (composed of the linear and angular velocities of the mobile mechanism), considering factors such as the mobile mechanism's current velocity, acceleration limits, and the position and shape of the obstacle. By evaluating the collision risk between the mobile mechanism's trajectory and the obstacle over a certain time period under each velocity combination, the optimal velocity command is selected and sent to the mobile mechanism's drive system. For example, if there is an obstacle ahead, the DWA algorithm will eliminate potential collision-prone velocity combinations in the velocity space, selecting a velocity that avoids the obstacle while still approaching the target path as closely as possible, allowing the mobile mechanism to safely and efficiently bypass the obstacle and continue moving towards the designated point.

[0258] As the robot approaches the designated surgical location, a high-precision indoor positioning system (such as a positioning system based on ultra-wideband (UWB) technology) is used to accurately position the wheeled mobile mechanism. The UWB positioning system calculates the precise position coordinates of the mobile mechanism by measuring the distances between the mobile mechanism and multiple fixed base stations using triangulation principles. Simultaneously, an inertial measurement unit (IMU) acquires the mobile mechanism's attitude information (such as pitch, roll, and yaw angles) to ensure that the mobile mechanism has an accurate position and attitude upon reaching the designated point, enabling the robotic arm to accurately manipulate surgical instruments to reach the target surgical site.

[0259] For tracked mobile mechanisms:

[0260] 1. Terrain adaptation and dynamic distribution

[0261] Based on the perception of the surgical environment terrain by the vision module and other sensors (such as a terrain scanner), the parameters of the tracked mobile mechanism are adjusted. For different terrains (such as flat ground, slopes, obstacles, etc.), the appropriate tension, speed, and steering angle of the tracks are calculated. For example, when climbing a slope, the driving force of the tracks is increased, and the front and rear tension distribution of the tracks is adjusted according to the slope angle to ensure that the mobile mechanism can climb stably without slipping. A terrain adaptation algorithm based on fuzzy control is adopted to fuzzily classify different terrain types according to the characteristics of terrain roughness, slope, etc., and to correspond to different control strategies. For example, when a relatively rugged terrain is detected, the fuzzy controller will appropriately reduce the moving speed and increase the grip of the tracks to prevent the mobile mechanism from bumping or losing control during movement.

[0262] In complex terrain, agile movement is achieved by controlling the differential steering of the tracks. Based on the target path and current position, the speed difference between the two tracks is calculated, allowing the mobile mechanism to smoothly turn in narrow spaces or around obstacles. For example, in an operating room where large pieces of equipment obstruct the view, the tracked mobile mechanism needs to navigate around these devices to reach the surgical position. By precisely controlling the speed difference between the two tracks, a compact turning radius is achieved, enabling rapid arrival at the designated location.

[0263] 2. Stability control and movement accuracy

[0264] To ensure the stability of the tracked mobile mechanism during movement, an inertial measurement unit (IMU) and pressure sensors mounted on the chassis monitor the mechanism's attitude and the contact pressure between the tracks and the ground. Based on this information, the track motion parameters are adjusted to prevent the mobile mechanism from tipping over or tilting. For example, if excessive pressure on one track is detected, potentially causing a rollover, the speed of that track is reduced while the driving force of the other track is increased, restoring the mobile mechanism to balance.

[0265] Upon reaching the vicinity of the surgical site, a vision-based precise positioning method is employed. A camera mounted on the mobile mechanism captures images of the surrounding environment, which are then matched against a pre-stored surgical scene map to determine the precise location of the mobile mechanism. Image feature extraction and matching algorithms (such as Scale Invariant Feature Transform (SIFT)) are used to identify key feature points in the images, which are then compared with feature points on the map to calculate the positional deviation of the mobile mechanism. Finally, by adjusting the movement of the tracks, the mobile mechanism accurately reaches the designated surgical point, providing a stable base platform for the robotic arm's surgical operations.

[0266] Therefore, the innovation of this invention in humanoid legged mobility mechanisms lies in the adoption of model predictive control-based gait planning and sensor fusion-based balance control algorithms. The MPC algorithm can plan the optimal gait sequence according to complex surgical environments and mobility requirements, improving mobility efficiency and adaptability. It also considers factors such as energy consumption, making the robot's movement within the operating room more intelligent. The sensor fusion-based balance control algorithm can accurately perceive the posture and center of gravity changes of the mobility mechanism, adjusting leg movements in real time to ensure stable balance under various terrains and operating conditions. This is a relatively advanced technology in traditional humanoid robot mobility control, providing a guarantee for the flexible movement of surgical robots in complex surgical environments.

[0267] Furthermore, for wheeled mobile mechanisms, the combination of visual servo-based path tracking and dynamic window-based obstacle avoidance strategies with a high-precision positioning system is a significant innovation. Visual servo-based path tracking improves the accuracy and adaptability of path tracking, enabling flexible adjustment of the movement path based on different visual markers within the operating room. The dynamic window-based obstacle avoidance strategy can quickly and safely avoid obstacles in complex environments, ensuring the mobile mechanism successfully reaches the surgical position. High-precision positioning systems (such as UWB positioning) further improve the positioning accuracy of the mobile mechanism in surgical scenarios, laying the foundation for precise surgical operations by the robotic arm, and representing a significant improvement in navigation and positioning compared to traditional wheeled mobile mechanisms.

[0268] Furthermore, the innovation of the tracked mobile mechanism lies in its fuzzy control-based terrain adaptation algorithm and vision-based precise positioning method. Fuzzy control automatically adjusts the track parameters and movement patterns according to different terrain features, giving the mobile mechanism strong terrain adaptability and enabling stable movement in various complex surgical environments (such as slopes and obstacles). The vision-based precise positioning method provides high-precision positioning upon reaching the surgical position, ensuring accurate docking of the mobile mechanism and providing a stable platform for robotic arm operation. This combination of terrain adaptation and precise movement is a relatively novel application of traditional tracked mobile mechanisms in surgical scenarios, improving the overall reliability and practicality of the surgical robot.

[0269] Optionally, the host computer is also equipped with a multilingual translation model, which enables the large surgical model to interact with the doctor in multiple languages ​​through the human-computer interaction interface.

[0270] Specifically, regarding the architecture and training of multilingual translation models:

[0271] 1. Neural Network Architecture Selection

[0272] The host computer employs a Transformer-based neural network as the core of its multilingual translation model. The Transformer architecture, through a multi-head attention mechanism, can process different parts of the input text in parallel, effectively capturing semantic and grammatical relationships within the text. For example, when processing surgical-related medical terms and sentences, the multi-head attention mechanism can simultaneously consider the technical meaning of the terms, the structure of the sentences, and contextual information, thereby improving translation accuracy. The model consists of an encoder and a decoder. The encoder converts the input source language text into a series of semantic vector representations, while the decoder generates the target language translation text based on these vectors.

[0273] To address the specific needs of the medical field, the Transformer architecture was adaptively modified. An embedding layer containing a medical terminology database and a medical knowledge graph was added, enabling the model to better understand and process medical terminology during translation. For example, when encountering medical terms like "cardiac catheterization," the embedding layer can directly access the corresponding medical concepts and related knowledge, rather than simply translating according to general language rules, thus ensuring the professionalism and accuracy of the translation.

[0274] 2. Training Data and Methods

[0275] A large amount of multilingual medical text data was collected as the training set, including medical literature, medical records, surgical reports, etc., covering multiple language pairs (such as English-Chinese, English-French, English-German, and other common medical communication language pairs). In the data preprocessing stage, text cleaning, word segmentation, and stemming were performed to improve data quality. For example, for Chinese text, a professional medical Chinese word segmentation tool was used to accurately segment sentences into medical terms and common words, facilitating model learning.

[0276] A supervised training method is employed, with the objective function being minimizing the translation loss. The loss function uses cross-entropy loss to measure the difference between the model-generated translated text and the real target language text. During training, stochastic gradient descent (SGD) or its variants (such as Adagrad, Adadelta, etc.) are used to optimize the model parameters. Simultaneously, regularization techniques such as L1 and L2 regularization and Dropout are employed to prevent overfitting. For example, in training the English-Chinese medical translation model, a large amount of English medical text and its corresponding Chinese translations are used for training. The model continuously adjusts its parameters to ensure that the generated Chinese translations are as close as possible to the real translated text in terms of semantics, syntax, and medical professionalism. After multiple iterations of training, the model gradually converges and acquires the ability to accurately translate medical texts.

[0277] Specifically, regarding integration with large surgical models and human-computer interaction interfaces:

[0278] 1. Data interaction and transformation

[0279] When a doctor inputs text instructions or questions through a human-computer interaction interface (HCI), the HCI first performs language recognition on the input text to determine its source language. Then, the text data is transmitted to a multilingual translation model. The multilingual translation model converts the source language text into a unified internal representation (such as a semantic representation based on word vectors or character vectors) for translation processing. For example, if the doctor inputs a surgical instruction in French, after the language recognition module identifies it as French, the translation model converts the French instruction into a vector representation, and then translates it according to pre-trained French-target language (such as English or Chinese, languages ​​supported by the large surgical model). The translated text is then converted into a format that the large surgical model can understand before being transmitted to the large surgical model for processing.

[0280] After the surgical model processes the relevant information, the generated responses or suggestions may need to be translated into multiple languages ​​to be presented in the doctor's native language. For example, the surgical model generates an English surgical risk assessment report based on the patient's condition and surgical plan. The multilingual translation model translates this report into the doctor's native language (such as Chinese) and then displays it to the doctor through a human-computer interaction interface, achieving seamless multilingual communication between the surgical model and the doctor.

[0281] 2. Real-time performance and accuracy guaranteed

[0282] To ensure real-time multilingual interaction, efficient model inference algorithms and hardware acceleration technologies were employed. During model inference, caching mechanisms and parallel computing techniques were used to reduce translation time. For example, translation results for common medical phrases and sentences were cached and directly invoked when encountered again, avoiding redundant translation calculations. Simultaneously, graphics processing units (GPUs) or dedicated AI chips (such as TPUs) were used to accelerate the neural network's computation process, improving translation speed and meeting the demands of rapid communication during surgery.

[0283] To ensure translation accuracy, a quality assessment module is included in the translation model. This module evaluates the translation results based on metrics such as language model score and semantic similarity. If the evaluation result falls below a set threshold, the translation model will attempt alternative translation strategies or perform secondary translation optimization. For example, when translating a complex medical case description, if the quality assessment module detects semantic ambiguity or inaccuracies in the translated text, the model will re-analyze the sentence structure, adjust translation parameters, or refer to more medical knowledge graph information to correct the translation result, ensuring that doctors can accurately understand the information provided by the surgical model.

[0284] Therefore, the innovation of this invention in multilingual translation models lies in its adoption of a Transformer architecture optimized for the medical field. By introducing a medical terminology database and a knowledge graph embedding layer, the model can better understand and process medical professional texts, improving the accuracy and professionalism of medical multilingual translation. Compared with traditional general multilingual translation models, this optimized architecture can more accurately translate medical instructions, medical records, surgical reports, etc., in surgical scenarios, avoiding misunderstandings caused by errors in the translation of medical terminology, and providing reliable language support for international medical communication and multilingual surgical collaboration.

[0285] Furthermore, the seamless integration of the multilingual translation model with the large surgical model and the human-computer interaction interface is a significant innovation. It enables natural multilingual communication between doctors and the large surgical model, allowing doctors to interact smoothly with the surgical system and obtain surgery-related information and suggestions regardless of their language. Simultaneously, through real-time assurance measures (such as caching and hardware acceleration) and accuracy optimization mechanisms (such as quality assessment and secondary translation), the efficiency and quality of multilingual interaction are improved. This is a relatively novel feature in traditional surgical assistance systems, greatly expanding the application scope of surgical robots and facilitating the use of the system for surgical operations and communication by doctors from different language backgrounds.

[0286] Optionally, the humanoid robotic surgical system further includes a holographic projection module for projecting the three-dimensional physiological structure model into a holographic image through spatial mapping.

[0287] Specifically, regarding holographic projection data processing and conversion:

[0288] 1. First, the 3D physiological structure model data is optimized to meet the requirements of holographic projection. Since holographic projection demands high data accuracy and resolution, a voxel-based oversampling algorithm is used to process the model data, increasing its detail and clarity. For example, for the organ surfaces in the original 3D physiological structure model, the oversampling algorithm inserts more voxels into the existing voxel mesh, making the surface smoother and more refined, thus improving the visual effect after holographic projection.

[0289] The optimized 3D physiological structure model data is converted into a format suitable for holographic projection devices, such as holographic data format (supported by devices like HoloLens) or light field data format. During the conversion process, the light field distribution information corresponding to the model data is calculated according to the principles of holographic projection. For example, using mathematical tools such as Fourier transform, the spatial domain data of the 3D model is converted into frequency domain data. Then, based on the diffraction theory of holographic projection, the amplitude and phase distribution of the light field on the holographic projection plane are calculated. This information is encoded into a format that the holographic projection device can recognize for subsequent projection operations.

[0290] 2. Spatial mapping and coordinate calibration

[0291] To achieve accurate spatial mapping of holographic images, a positioning module and a vision module are used to acquire three-dimensional coordinate information of the surgical space. The positioning module determines the position and orientation of the holographic projection device in the surgical space, while the vision module scans the surgical environment and the patient's body to obtain spatial position information of surrounding objects. Then, the coordinate system of the three-dimensional physiological structure model is calibrated with the surgical space coordinate system. For example, by identifying the positions of specific markers or anatomical landmarks in the operating room in the model coordinate system and the actual spatial coordinate system, the coordinate transformation matrix between the two is calculated to ensure that the holographic image can be accurately projected onto the spatial position corresponding to the patient's body, achieving precise fusion of the virtual model and the real surgical scene.

[0292] Specifically, when the positioning module determines the position and orientation of the holographic projection device, the following are the key steps in the accurate spatial mapping process of the aforementioned holographic image:

[0293] (1) The positioning module determines the position and orientation of the holographic projection device.

[0294] The position and orientation of the holographic projection device in the surgical space are described by coordinate transformation, which is generally represented by a homogeneous transformation matrix.

[0295] A right-handed Cartesian coordinate system O was established in the surgical space (world coordinate system). w -X w Y w Z w Holographic projection devices themselves have a device coordinate system O. d -X d Y d Z d The positioning module can obtain the translation vector t = [t] of the holographic projection device's coordinate system relative to the world coordinate system through measurement and other means. x ,t y ,t z ] T (indicates X) w Y w Z w Translations in three directions and rotation information (usually represented by a rotation matrix R).

[0296] The rotation matrix R is a 3×3 orthogonal matrix that describes the angles of rotation of the device coordinate system relative to the world coordinate system about each coordinate axis. For example, about (X... w The rotation matrix R of the axis rotation angle α x (α) is:

[0297]

[0298] Around Y w The rotation matrix R of the axis rotation angle β y (β) is:

[0299]

[0300] Around Z w The rotation matrix R of the axis rotation angle γ z (γ) is:

[0301]

[0302] If the equipment circles X sequentially w Y w Z w The axes are rotated by angles α, β, and γ respectively (the order is important; different orders yield different results). The total rotation matrix R can be obtained by multiplying them sequentially, such as when rotating in the order of (Z), (Y), and (X):

[0303] R = R x (α)R y(β)R z (γ)

[0304] The homogeneous transformation matrix from the device coordinate system to the world coordinate system. It can be represented as:

[0305]

[0306] Among them 0 T = [0,0,0]. Thus, any point p in the device coordinate system... d =[x d ,y d ,z d ,1] T coordinate p in the world coordinate system w =[x w ,y w ,z w ,1] T It can be obtained by converting using the following formula:

[0307]

[0308] The parameters (t and R) obtained by the positioning module determine the position and orientation of the holographic projection device in the surgical space, which is the basis for subsequent spatial mapping.

[0309] (2) The vision module acquires spatial location information of surrounding objects.

[0310] When a visual module scans the surgical environment and the patient's body to obtain spatial location information of objects, it often involves technologies such as 3D reconstruction in computer vision. A common method is based on the principle of multi-view geometry, which involves processing images taken from different perspectives.

[0311] The vision module has two cameras (or takes pictures from different positions at different times, equivalent to a dual-view situation), with their optical centers at positions O1 and O2 in the world coordinate system, and corresponding to camera coordinate systems O1 and O2 respectively. c1 -X c1 Y c1 Z c1 and O c2 -X c2 Y c2 Z c2 .

[0312] For a point P in space, its coordinates in the two camera coordinate systems are p1, p2, p3, p4, p5, p6, p7, p8, p9, p1, p1, p2, p1, p2, p1, p2, p3, p4 ...1, p2 c1 =[x c1 ,y c1 ,z c1 ] T and p c2 =[x c2 ,yc2 ,z c2 ] T The coordinates of the given coordinates in the world coordinate system are p. w =[x w ,y w ,z w ] T .

[0313] First, the coordinates in the camera coordinate system need to be transformed to the world coordinate system. This involves the extrinsic parameter matrices of each camera (similar to the homogeneous transformation matrix of the holographic projection device mentioned earlier, which contains rotation and translation information). Let the extrinsic parameter matrix of camera 1 be... The extrinsic parameter matrix of camera 2 is Then we have:

[0314]

[0315] At the same time, there exists an essential matrix E and a fundamental matrix F between the two camera coordinate systems, satisfying the following relationship (based on the principle of epipolar geometry):

[0316]

[0317] The fundamental matrix F and the essential matrix E are related (assuming the camera intrinsic matrix is ​​K1 and K2 respectively):

[0318]

[0319] By matching feature points in the image (e.g., using feature extraction algorithms such as SIFT and SURF to find the pixel coordinates of the same feature points in two camera images (m1 = [u1, v1, 1)). T () and (m2=[u2,v2,1]) T By combining the camera's intrinsic parameters, we can first calculate the fundamental matrix (F), then obtain the essential matrix (E), and then solve for the coordinates of the spatial point (P) in the world coordinate system through methods such as singular value decomposition, thus obtaining the spatial position information of the surrounding objects.

[0320] (3) Coordinate system calibration: Calculate the coordinate transformation matrix.

[0321] Assume the three-dimensional physiological structure model has its own model coordinate system (O). m -X m Y m Z m It needs to be aligned with the surgical space coordinate system (world coordinate system (O)). w -X w Y w Z w Calibration is performed by identifying specific markers or anatomical landmarks within the operating room to establish connections.

[0322] Suppose there are (n) corresponding marker points (or anatomical landmarks), whose coordinates in the model coordinate system are (p mi =[x mi ,y mi ,z mi ] T ((i=1,2,…,n)), whose coordinates in the world coordinate system are (p wi =[x wi ,y wi ,z wi ] T (i = 1, 2, ..., n)).

[0323] The coordinate transformation matrix from the model coordinate system to the world coordinate system can be solved using the least squares method. (It is also a homogeneous transformation matrix form that includes rotation and translation information).

[0324] First, the coordinate transformation relationship is written in the following form (for each corresponding point):

[0325] p wi =Rp mi +t

[0326] Where (R) is the rotation matrix and (t) is the translation vector, we want to find suitable (R) and (t) that minimize the error. Construct the error function:

[0327]

[0328] By minimizing the aforementioned error function using optimization algorithms such as Singular Value Decomposition (SVD), the rotation matrix R and the translation vector (t) can be obtained, and the coordinate transformation matrix can then be determined. for:

[0329]

[0330] Thus, any point (p) in the three-dimensional physiological structure model m =[x m ,y m ,z m ,1] T The coordinates (p) in the world coordinate system (surgical space coordinate system) w =[x w ,y w ,z w ,1] T The following formula can be used for conversion:

[0331]

[0332] Through the above series of formula calculations based on coordinate transformation, multi-view geometry, and least squares method, accurate spatial mapping of holographic images is achieved, ensuring that holographic images can be accurately projected onto the spatial position corresponding to the patient's body, and completing the precise fusion of virtual model and real surgical scene.

[0333] Specifically, regarding the control and image projection of holographic projection equipment:

[0334] 1. Laser source and optical system control

[0335] The laser source in the holographic projection module is the key component for generating holographic images. Based on the light field information in the hologram data, the intensity, frequency, and phase of the laser source are precisely controlled. For example, the laser intensity is increased for bright areas in the hologram and decreased for dark areas. This precise adjustment of the laser source is achieved by controlling the laser driver with digital signals. Simultaneously, the position and angle of components such as mirrors and lenses in the optical system are controlled to adjust the propagation path and focusing effect of the laser beam, ensuring that the laser beam accurately illuminates the holographic projection medium (such as holographic film or light-scattering areas in the air) to form a clear holographic image.

[0336] Adaptive optics technology is employed to compensate for optical distortions caused by environmental factors (such as airflow and temperature changes). A wavefront sensor is installed in the optical system to monitor the wavefront distortion of the laser beam in real time. Adjustable optical elements, such as deformable mirrors, are then used to correct the wavefront. For example, when airflow in the operating room causes a slight bending of the laser beam, the wavefront sensor detects this change and calculates the adjustment parameters for the deformable mirror based on the detection result. This causes the deformable mirror to deform accordingly, thereby correcting the wavefront of the laser beam and ensuring the stability of the holographic image quality.

[0337] 2. Dynamic Holographic Image Update and Interaction

[0338] During surgery, as the three-dimensional physiological structure model is updated in real time (e.g., due to changes in the model caused by patient movement or surgical procedures), the updated model data is promptly converted into holographic projection data and projected as a holographic image. An incremental data update algorithm is employed, transmitting only the changed data within the model, reducing data transmission volume and processing time, thus enabling rapid updates to the holographic image. For example, when surgical instruments cut or move tissue, the corresponding tissue portion in the model changes; only the data of this change is calculated and transmitted, updating the corresponding area in the holographic image, allowing surgeons to see the impact of surgical procedures on the patient's body structure in real time.

[0339] To enable interaction between doctors and holographic images, the holographic projection module is equipped with interactive sensors (such as gesture recognition sensors and voice recognition sensors). These sensors acquire the doctor's interactive commands, such as rotating or zooming the holographic image with gestures, or querying information about specific tissues via voice. Then, the holographic image is manipulated accordingly based on the interactive commands, such as changing the viewing angle or displaying detailed information about specific tissues. For example, when a doctor makes a rotation gesture, the gesture recognition sensor transmits the gesture information, calculates the rotation angle and speed of the holographic image based on the direction and speed of the gesture, and then updates the holographic projection data so that the holographic image rotates according to the doctor's intention. This allows the doctor to observe the patient's physiological structure from different angles, improving the accuracy of surgical decisions and the convenience of surgical procedures.

[0340] Therefore, the innovation of this invention in holographic projection data processing lies in the use of voxel oversampling and light field data conversion algorithms, combined with precise spatial mapping and coordinate calibration techniques. Voxel oversampling improves the visual quality of holographic projection, enabling the three-dimensional physiological structure model to present richer details and higher clarity in the holographic image, which is crucial for doctors to accurately observe the patient's body structure. Precise spatial mapping and coordinate calibration achieve accurate fusion of the holographic image and the surgical space, accurately projecting the virtual physiological structure model onto the corresponding position on the patient's body, providing intuitive and accurate visual assistance for surgical operations. Compared with traditional two-dimensional images or simple three-dimensional display methods, this greatly improves the visualization and precision of surgery.

[0341] Furthermore, in the control of holographic projection equipment, adaptive optics technology, used to compensate for optical distortions caused by environmental factors, is a significant innovation. It ensures the stability of holographic images in complex surgical environments, unaffected by factors such as airflow and temperature changes, guaranteeing that doctors can always clearly see the holographic images. Dynamic holographic image updating and interaction technology further enhances the interactivity and real-time nature of holographic images. Doctors can interact with the holographic images through natural methods such as gestures and voice, adjusting the displayed content and viewing angle in real time, and observing changes in the model during surgery. This interactivity and real-time capability provide more flexible and efficient auxiliary means for surgical operations, representing an innovative development of traditional holographic projection technology in surgical applications, and contributing to improved surgical efficiency and success rates.

[0342] Optionally, the host is also equipped with an auxiliary decision-making and control component to respond to surgical query commands captured through the human-computer interaction interface and return response results, the response results including at least one of the following: surgical operation suggestions, surgical step planning suggestions, hand speed risk assessment, postoperative suggestions, and intraoperative suggestions.

[0343] Specifically, regarding data collection and preprocessing:

[0344] 1. Multi-source data integration

[0345] First, surgery-related data is collected from multiple data sources. This includes surgical query commands input by doctors via a human-computer interaction interface, basic patient medical record information, preoperative examination results, lesion imaging data, and real-time physiological monitoring data (such as heart rate, blood pressure, and blood oxygen saturation) and real-time medical imaging data (such as intraoperative ultrasound and endoscopic images) from the surgical procedure. This data is then integrated and standardized to ensure its effective use by decision support and control components. For example, image data in different formats is converted to a unified image format, and text data in medical records is structured to extract key information such as disease name, duration of medical history, and allergy history.

[0346] Data cleaning algorithms are used to remove noise and errors from data. For image data, image filtering algorithms remove speckle noise, salt-and-pepper noise, and other unwanted noise, improving image quality. For text data, syntax and semantic checking algorithms correct typos and incorrect medical terminology. For example, in medical records, if an incorrect term like "high snow pressure" (which should be "hypertension") appears, the data cleaning algorithm can identify and correct it, ensuring the accuracy and reliability of the data and providing a solid data foundation for subsequent analysis and decision-making.

[0347] Specifically, regarding analysis based on machine learning and knowledge rules:

[0348] 1. Machine learning model building and training

[0349] Multiple machine learning models were constructed for different types of decision analysis. For example, a random forest algorithm was used to build a classification model for surgical risk assessment. A large amount of historical surgical case data was collected, including patients' physiological indicators, disease characteristics, surgical types, and corresponding surgical outcomes (such as whether complications occurred, whether the surgery was successful, etc.). This data was divided into training and testing sets. The random forest model was trained using the training set, and its performance was optimized by adjusting parameters such as the number of decision trees and the depth of the trees. During training, the model learned the complex relationships between different features (such as patient age, blood pressure, tumor size, etc.) and surgical risk. For example, the model might find that older patients with multiple underlying diseases have a higher surgical risk when undergoing complex surgeries.

[0350] For surgical procedure planning suggestions, a sequence-to-sequence (Seq2Seq) model is employed, such as one based on a recurrent neural network (RNN) or long short-term memory network (LSTM) architecture. Training data is provided using surgical procedure records from numerous successful surgical cases. Initial surgical conditions (such as patient condition and surgical type) are used as the input sequence, and the sequence of surgical steps is used as the output sequence. The model learns the appropriate sequence of surgical steps and key operational points for different surgical scenarios. For example, in coronary artery bypass surgery, the Seq2Seq model can generate planning suggestions for specific surgical steps, including vessel acquisition and anastomosis, based on input information such as the patient's coronary artery occlusion and cardiac function.

[0351] 2. Knowledge Rule Base Construction and Application

[0352] A medical knowledge rule base is established to transform the experience of medical experts and the knowledge in medical guidelines into computer-recognizable rules. For example, rules regarding surgical contraindications can be formulated, such as "Major surgery is prohibited if a patient's platelet count is below 50 × 10⁹ / L, unless special platelet supplementation measures are taken." During the decision support process, when a surgical query is received, the system first performs preliminary judgment and screening based on the knowledge rule base. For instance, if a doctor asks whether a patient with a low platelet count is suitable for surgery, the system directly provides a preliminary contraindication suggestion by querying the knowledge rule base, and then combines the analysis results of the machine learning model to provide more comprehensive decision recommendations.

[0353] This involves fusing the output of machine learning models with a knowledge rule base. For example, in surgical risk assessment, a machine learning model might provide a data-driven risk probability value, which can be modified or supplemented by rules from the knowledge rule base. If the model predicts a moderate risk for a surgery, but the knowledge rule base indicates that the risk increases significantly under certain circumstances (such as a patient with a specific allergy), the final risk assessment will consider both factors, providing doctors with more accurate and detailed risk warnings and improving the scientific rigor and reliability of their decisions.

[0354] Specifically, regarding the generation and feedback of response results:

[0355] 1. Personalized response generation

[0356] Based on the doctor's surgical query instructions and the patient's specific situation, personalized response results are generated. For example, when a doctor inquires about a patient's surgical risk assessment, the decision support and control component comprehensively considers factors such as the patient's age, gender, disease type, physical condition, and type of surgery. If an elderly diabetic patient is undergoing orthopedic surgery, the system will focus on analyzing factors such as the impact of diabetes on wound healing and the cardiovascular risk of elderly patients. Combining machine learning model predictions and knowledge rule base specifications, a detailed surgical risk assessment report is generated, including potential complications (such as infection, cardiovascular accidents, etc.), the probability of these risks occurring, and corresponding preventive measures recommendations.

[0357] Surgical procedure planning suggestions are also customized based on individual patient differences. If a patient has variations in anatomical structure (such as vascular malformations, abnormal organ positions, etc.), the system will take these special circumstances into account when generating surgical procedures, adjusting the conventional surgical path and operation methods. For example, in liver surgery, if the patient's liver blood vessel distribution differs from normal, the decision support system will advise the surgeon to pay special attention to the identification and handling of blood vessels during surgery to avoid errors and improve the safety and success rate of the surgery.

[0358] 2. Real-time feedback and updates

[0359] During surgery, as new data (such as real-time physiological monitoring data and imaging data) continuously floods in, the decision support and control components update their response results in real time. For example, if a patient's blood pressure suddenly drops during surgery, the system immediately reassesses the surgical risks, considering the impact of the blood pressure drop on the surgical process, such as whether surgery needs to be paused or vasopressors should be implemented, and provides timely intraoperative advice to the surgeon. Simultaneously, based on the actual progress of the surgery, such as completed surgical steps and new findings (such as unexpected tissue adhesions or bleeding points), the system dynamically adjusts the surgical procedure planning and postoperative recommendations. For example, if the tumor is found to be more aggressive than expected during surgery, the system updates the postoperative follow-up plan and possible adjuvant therapy recommendations, ensuring that the surgeon receives timely decision support information most relevant to the actual surgical situation.

[0360] Therefore, the innovation of this invention in decision support lies in the integration and analysis of multi-source data. By collecting comprehensive patient information, including medical records, examination results, and real-time intraoperative data, a more comprehensive and in-depth understanding of the patient's surgical condition can be achieved. Based on this rich data, machine learning models are used to uncover the complex relationships behind the data, combined with expert experience and medical guidelines from a knowledge rule base, enabling precise analysis of decisions such as surgical risk assessment and surgical procedure planning. Compared with traditional decision-making methods that rely solely on physician experience or a single data source, this multi-source data-driven approach can provide more scientific and accurate decision recommendations, reduce surgical risks, and improve surgical quality.

[0361] 2. Intelligent decision-making through the fusion of machine learning and knowledge rules

[0362] The integration of machine learning models and knowledge rule bases represents a significant innovation. Machine learning models can automatically learn patterns and rules from massive amounts of data, but they may suffer from data bias or difficulty in interpretation. Knowledge rule bases, on the other hand, provide explicit medical knowledge and expert experience, offering reliability and interpretability. This integration allows the decision support system to leverage the powerful data processing capabilities of machine learning while adhering to medical expertise and standards, complementing and correcting each other to generate more rational and reliable decisions. This integrated intelligent decision-making mechanism is a relatively novel application in the field of surgical decision support, helping to increase doctors' trust in and acceptance of decision recommendations, and promoting the intelligent development of surgical decision-making.

[0363] 3. Personalized and real-time dynamic decision support

[0364] Personalized response generation and real-time feedback updates are the innovative highlights of this technology. Based on individual patient differences and the real-time progress of the surgery, the system can provide customized decision-making suggestions, closely aligning with the actual needs of the surgery, from pre-operative planning to intraoperative adjustments. Real-time dynamic updates of decision results can promptly address unexpected situations and changes during surgery, providing timely and effective decision support for doctors. This is difficult to achieve in traditional surgical decision support systems, greatly improving the flexibility and adaptability of surgery, ensuring the smooth progress of the surgery and patient safety.

[0365] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A humanoid robotic surgical system, characterized by, The humanoid robot surgery system comprises a mechanical arm, a trunk, a moving mechanism, a host computer, a human-computer interface, a positioning module, a visual module and a wireless module, the mechanical arm is assembled on the trunk and in communication connection with the host computer, the trunk is assembled on the moving mechanism, a host computer accommodating space is formed in the trunk to accommodate the host computer therein, the trunk is a humanoid trunk, the humanoid trunk has at least two degrees of freedom to match at least one of forward and backward bending, left and right side bending and height self-adjustment during surgery, the human-computer interface is assembled on the trunk and in communication connection with the host computer, a generative surgery large model is arranged on the host computer, the positioning module is used to position a target surgery site during surgery, the visual module is used to visually track the mechanical arm and the target surgery site during surgery, and the wireless module is used to receive real-time physiological monitoring data and real-time medical image data of a patient through a constructed wireless channel, so that the surgery large model analyzes the real-time physiological monitoring data and real-time medical image data and generates a surgery suggestion plan according to an analysis result. The human-computer interface receives basic medical record information, preoperative examination results and lesion image data of a patient to construct a three-dimensional physiological structure model; at this time, the following steps are specifically performed: a spatial conversion relationship among the lesion image data, the positioning module and the visual module is established; and the three-dimensional physiological structure model is analyzed to mark a target surgery site. Based on the spatial conversion relationship, a mechanical arm path is formulated for the target surgery site. The three-dimensional physiological structure model is analyzed to mark a target surgery site and formulate a mechanical arm path accordingly; at this time, the following steps are specifically performed: a point is determined, a surgery depth is determined, a control torque of the mechanical arm is calculated, and a control strategy of the mechanical arm driving a surgery instrument to reach the point and the depth is formulated. A mapping relationship between the mechanical arm path and a position of the patient is established to generate a motion trajectory sequence of the mechanical arm. According to the motion trajectory sequence, the moving mechanism is driven to move towards the position of the patient to reach the point, and the mechanical arm is controlled to drive the surgery instrument to perform surgery on the target surgery site. The number of the mechanical arms is at least two, each mechanical arm is connected in series by a plurality of joints of more than 5 axes, and a wrist is formed at the end of each mechanical arm, wherein the wrist of one mechanical arm is provided with a main surgery instrument, and the wrist of at least one of the remaining mechanical arms is provided with an auxiliary surgery instrument.

2. The anthropomorphic robotic surgical system of claim 1, wherein, The moving mechanism is at least one of a humanoid leg-foot type, a wheel type or a track type moving mechanism.

3. The anthropomorphic robotic surgical system of claim 1, wherein, A multi-language translation model is further arranged on the host computer to enable the surgery large model to interact with a doctor in multiple languages through the human-computer interface.

4. The anthropomorphic robotic surgical system of claim 1, wherein, The humanoid robot surgery system further comprises a holographic projection module for projecting the three-dimensional physiological structure model as a holographic image through spatial mapping.

5. The anthropomorphic robotic surgical system of claim 1, wherein, ​ 6. The anthropomorphic robotic surgical system of claim 1, wherein, The host computer is further deployed with an auxiliary decision and control component to respond to the surgical query instruction captured through the human-computer interface and return a response result, the response result including at least one of: a surgical operation suggestion, a surgical step planning suggestion, a hand speed risk assessment, a postoperative suggestion, and an intraoperative suggestion.

Images