Humanoid robotic surgical system

Abstract

In the humanoid robot operation system provided by the embodiment of the invention, the mechanical arm is assembled on the trunk and is in communication connection with the host, the trunk is assembled on the moving mechanism, the trunk is internally provided with the host accommodating space so as to assemble the host in the host accommodating space, and the man-machine interface is assembled on the trunk and is in communication connection with the host. A generation type operation large model is deployed on the host, and the operation large model executes the following steps that basic medical record information, preoperative examination results and focus image data of a patient are received through a man-machine interface to construct a three-dimensional physiological structure model; performing feature analysis on the three-dimensional physiological structure model to mark a target surgical site and formulate a mechanical arm path according to the target surgical site; establishing a mapping relation between the path of the mechanical arm and the position of the patient to generate an action track sequence of the mechanical arm; according to the action track sequence, the moving mechanism is driven to move towards the position of the patient to reach the designated point position, and the mechanical arm is controlled to drive the surgical instrument to act so as to perform an operation on the target surgical site.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of robotics technology, and in particular to a humanoid robotic surgery system. Background Art

[0002] Surgery plays a vital role in modern medicine, and the precision and safety of its operation have always been the goal pursued in the medical field. In the past 30 years, surgical navigation and positioning surgical robot technology has continued to develop and has been increasingly widely used in many fields such as neurosurgery, orthopedics, oral implants, abdominal puncture, bronchial navigation, etc.

[0003] At present, existing surgical robots have many limitations in clinical applications. The main functions of most surgical robots are concentrated in puncture navigation and positioning. Their automation and intelligence are relatively low, and the interaction between them and doctors is extremely limited. In addition, these robots are usually fixed in a specific position in the operating room, occupying a large surgical space, and can only perform simple positioning operations according to pre-set programs during the operation. It is difficult to flexibly adjust the surgical strategy according to the actual situation during the operation, and the overall help to clinical surgery is limited. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a humanoid robotic surgery system to at least partially solve the above problems.

[0005] According to a first aspect of an embodiment of the present invention, a humanoid robot surgical system is provided, which specifically comprises: a robotic arm, a trunk, a mobile mechanism, a host, and a human-machine interface, wherein the robotic arm is mounted on the trunk and is communicatively connected to the host, the trunk is mounted on the mobile mechanism, a host accommodating space is provided in the trunk so that the host can be mounted in the host accommodating space, the human-machine interface is mounted on the trunk and is communicatively connected to the host, a generative surgical large model is deployed on the host, and the surgical large model performs the following steps to implement the surgical process:

[0006] The three-dimensional physiological structure model is constructed by receiving the patient's basic medical history information, preoperative examination results and lesion imaging data through the human-machine interface;

[0007] Performing feature analysis on the three-dimensional physiological structure model to mark the target surgical site and formulate a robotic arm path accordingly;

[0008] Establishing a mapping relationship between the path of the robotic arm and the position of the patient to generate a motion trajectory sequence of the robotic arm;

[0009] According to the motion trajectory sequence, the moving mechanism is driven to move toward the patient's position to reach a specified point, and the robotic arm is controlled to drive the surgical instrument to move so as to perform surgery on the target surgical site.

[0010] The solution of the embodiment of the present invention has at least the following technical advantages:

[0011] 1. Accurate surgical planning and execution

[0012] The system obtains detailed patient information through the human-machine interface to build a three-dimensional physiological structure model. Compared with existing surgical robots that can only perform simple positioning, it can more accurately mark the target surgical site and develop the path of the robotic arm. This helps doctors understand the patient's condition more intuitively and accurately before surgery, plan the surgical plan, and thus improve the accuracy of the surgery. For example, in complex abdominal surgery, accurate identification of the relationship between the lesion and the surrounding tissue structure can avoid misoperation and reduce surgical risks.

[0013] Generating the robot arm motion trajectory sequence based on precise model analysis can ensure that surgical instruments move along the predetermined optimal path during surgery, achieve precise operation, and overcome the problem that existing robots are difficult to cope with intraoperative changes based on preset programs.

[0014] 2. Improve surgical efficiency

[0015] The mobile mechanism enables the robot to move freely and quickly reach the patient's designated position. There is no need for complex layout adjustments before surgery like traditional fixed robots, which saves surgical preparation time and improves the overall efficiency of the surgery.

[0016] The generative surgical model on the host coordinates the work of various components, making the movements of the robotic arm, torso and mobile mechanism more efficient and smooth, avoiding the time waste caused by the relatively independent operation and loose connection of various parts of the traditional robot.

[0017] 3. Enhance human-machine collaboration

[0018] The human-machine interface facilitates the interaction between doctors and robots. Doctors can input commands and obtain system feedback in a timely manner. Compared with the limited interaction methods of existing robots, this efficient communication method enables doctors to better control the surgical process and improve the coordination of the surgery.

[0019] The system can provide decision support to doctors based on their input and intraoperative conditions. For example, it can consider the doctor's operating habits and experience when formulating the path of the robotic arm, making the robot's assistance more in line with the doctor's needs and improving the tacit understanding of human-machine collaboration.

[0020] 4. Optimize surgical adaptability

[0021] The system can adjust the path and trajectory of the robotic arm according to the actual situation during the operation, adapting to dynamic factors during the operation, such as changes in patient position and tissue deformation, while existing robots are less flexible in this regard. This allows the operation to proceed more smoothly and reduces the risk of interruption or failure of the operation due to the inability to adjust the surgical strategy in time.

[0022] By establishing a mapping relationship between the robotic arm path and the patient's position, the system can quickly and accurately adjust the robotic arm movement under different surgical scenarios and individual patient differences, thereby improving the adaptability of the surgical system to various surgical situations.

[0023] Here, it should be noted that the above technical benefits do not need to be achieved simultaneously in the solution of one embodiment, but a certain embodiment may only have some of the technical benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 The figure is a schematic structural diagram of a humanoid robot surgical system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] Figure 1 FIG. 1 is a schematic diagram of the structure of a humanoid robot surgical system according to an embodiment of the present invention. Figure 1 As shown, it specifically includes: a robot arm 101, a trunk 102, a mobile mechanism 103, a host 104, and a human-machine interface 105. The robot arm is mounted on the trunk and is connected to the host in communication. The trunk is mounted on the mobile mechanism. A host accommodating space is provided in the trunk so that the host can be mounted in the host accommodating space. The human-machine interface is mounted on the trunk and is connected to the host in communication. A generative surgery model is deployed on the host. The surgery model performs the following steps to implement the surgical process:

[0026] The three-dimensional physiological structure model is constructed by receiving the patient's basic medical history information, preoperative examination results and lesion imaging data through the human-machine interface;

[0027] Performing feature analysis on the three-dimensional physiological structure model to mark the target surgical site and formulate a robotic arm path accordingly;

[0028] Establishing a mapping relationship between the path of the robotic arm and the position of the patient to generate a motion trajectory sequence of the robotic arm;

[0029] According to the motion trajectory sequence, the moving mechanism is driven to move toward the patient's position to reach a specified point, and the robotic arm is controlled to drive the surgical instrument to move so as to perform surgery on the target surgical site.

[0030] The humanoid robotic surgical system of the embodiment of the present invention has many significant advantages: in precise surgical planning and execution, with the help of the human-machine interface to build a three-dimensional physiological structure model, the target area can be accurately marked, the robot arm path can be formulated, and a motion trajectory sequence can be generated, thereby improving surgical accuracy and overcoming the limitations of existing robots; in terms of improving surgical efficiency, the mobile mechanism can be quickly put into place, and the generative surgical large model makes the coordination of various components more efficient, saving preparation time; enhanced human-machine collaboration is reflected in the convenient interaction of the human-machine interface and the decision support provided by the system, thereby improving surgical coordination and human-machine tacit understanding; optimized surgical adaptability is manifested in the ability to adjust the robot arm path and movements according to the actual situation during the operation and patient differences, reducing the risk of surgical interruption or failure, and making the operation go more smoothly.

[0031] When the generative surgery model receives the patient's basic medical history information, preoperative examination results and lesion image data through the human-machine interface to build a three-dimensional physiological structure model, the specific processing process includes the following:

[0032] (1) Data collection and preprocessing

[0033] 1. Interface data reception

[0034] The high-speed data bus is used to connect to the human-machine interface, which is equipped with a variety of communication protocol stacks to ensure seamless connection with different external devices (such as hospital information systems (HIS), laboratory information systems (LIS), and picture archiving and communication systems (PACS). For example, for imaging devices that follow the DICOM standard, the human-machine interface establishes a connection through the DICOM network communication protocol to achieve high-speed transmission of imaging data. At the same time, for text data such as basic medical record information and preoperative examination results, the secure and reliable HL7 (Health Level-7) protocol is used for reception. This design ensures that data can be accurately, completely and efficiently transmitted from external devices to.

[0035] When receiving data, the human-machine interface will perform preliminary format verification and integrity checks on the data. For example, for DICOM image data, check whether the key information in the file header (such as patient ID, examination date, imaging modality, etc.) is complete, and for text data, verify whether the format of the data field meets the predetermined specifications. Only data that passes the preliminary verification will be further transmitted for subsequent processing.

[0036] 2. Data verification and cleaning

[0037] A strict set of data verification rule bases is built in, which is based on the standard specifications and common error patterns of medical data. For example, for each indicator in the blood routine test results, a reasonable value range is set. If the received data exceeds this range, it is marked as suspicious data. For image data, the statistical characteristics of the image (such as mean, variance, etc.) are calculated and compared with the reference values ​​of normal images to determine whether the data is abnormal.

[0038] In terms of data cleaning, an interpolation algorithm based on machine learning is used for missing data. For example, a regression model is trained using existing patient data to predict the value of missing data based on other relevant characteristics of the patient (such as age, gender, disease type, etc.). For noise interference, a wavelet transform algorithm is used for filtering. Wavelet transform can analyze image data at different scales and frequencies, effectively remove noise while retaining the detailed features of the image. For example, when processing Gaussian noise in MRI images, the signal-to-noise ratio of the image can be significantly improved by selecting appropriate wavelet basis functions and thresholds.

[0039] (2) Image data registration and fusion

[0040] 1. Multimodal Image Registration

[0041] A registration algorithm based on maximizing mutual information is used, which uses the information redundancy between images of different modalities to achieve accurate registration. Specifically, for CT and MRI images, the images are first normalized to make their grayscale value range consistent. Then, the MRI image is moved in space with a certain step size, and the mutual information between the CT and MRI images after each movement is calculated. Mutual information is a measure of the degree of mutual dependence between two random variables. When the mutual information reaches the maximum value, it is considered that the two images are optimally matched in space.

[0042] In order to improve the accuracy and efficiency of registration, the algorithm adopts a multi-resolution strategy. It starts with a low-resolution image for rough registration, quickly determines the approximate range of transformation parameters, and then gradually increases the resolution to make registration adjustments at a finer scale. At the same time, the image pyramid structure is used to sample and process images at different resolution levels, reducing the amount of calculation while ensuring the accuracy of registration. For example, in the initial low-resolution stage, the image is reduced to 1 / 8 of the original size, and the approximate registration position is found by quickly calculating the mutual information, and then the registration results are gradually refined in the high-resolution stage.

[0043] 2. Image Fusion

[0044] For image fusion, a weighted fusion algorithm based on principal component analysis (PCA) is used. First, the registered CT and MRI images are decomposed by PCA to obtain their respective principal component vectors and eigenvalues. Then, weights are assigned to each principal component according to the importance and contribution of different tissues in different modality images. For example, for bone structures, higher weights are assigned in CT images, while for soft tissues, higher weights are assigned in MRI images.

[0045] Finally, the weighted principal components are subjected to an inverse PCA transformation to obtain the fused image. This fusion method can fully retain the high-resolution information of the bone structure in the CT image and the rich detail information of the soft tissue in the MRI image. The generated fused image is clearer and more accurate in displaying the lesion area and its relationship with the surrounding tissues. For example, in the planning of brain tumor surgery, the fused image can clearly show the location and size of the tumor and its relationship with the surrounding blood vessels and nerve tissues, providing an important basis for surgical path planning.

[0046] (3) 3D model reconstruction

[0047] 1. Segmentation Algorithm

[0048] A deep learning segmentation algorithm based on the 3D U-Net architecture is used. This network structure is extended to three-dimensional space based on the 2D U-Net, and can better process the context information of volume data. The network consists of two parts: an encoder and a decoder. The encoder gradually extracts the features of the image data through a series of convolutional layers and pooling layers, gradually reduces the resolution of the image, and increases the number of feature channels, thereby obtaining more abstract and advanced semantic features.

[0049] The decoder maps the features back to the original image resolution through upsampling layers and convolutional layers, and fuses them with the feature maps of the corresponding levels in the encoder to restore the detailed information of the image. During the training process, a large amount of annotated medical imaging data is used to train the network, and the loss function combines cross entropy loss with Dice coefficient loss to enable the network to learn accurate tissue segmentation boundaries. For example, when training the lung CT image segmentation model, the network can accurately identify different structures such as lung tissue, trachea, blood vessels, and lung tumors, and the segmentation accuracy reaches sub-millimeter level.

[0050] 2. 3D modeling technology

[0051] After the segmentation is completed, the Marching Cubes algorithm is used to reconstruct the surface of the tissue and lesion area represented by voxels. The algorithm extracts the isosurface in the volume data and converts the voxel data into a three-dimensional surface model represented by a triangular patch. Specifically, the algorithm traverses each voxel in the volume data and determines whether the voxel intersects with the isosurface based on the grayscale value of the voxel vertex. If it intersects, the intersection of the isosurface and the voxel edge is calculated by interpolation, and these intersections are connected to form a triangular patch, thereby constructing the surface of the three-dimensional model.

[0052] For data represented by polygonal meshes, a modeling technique based on half-edge data structure is used. This data structure can efficiently represent and process polygonal mesh models, and facilitates model editing, modification, and optimization operations. After the model is built, the Laplace smoothing algorithm is used to smooth the model surface. By adjusting the position of the vertices, the model surface is made smoother and more natural. At the same time, the hole filling algorithm is used to repair the holes in the model to ensure the integrity and topological correctness of the model. For example, when constructing a three-dimensional heart model, the model after smoothing and hole filling can accurately present the shape and structure of the heart, which is convenient for doctors to make intuitive observations and surgical planning.

[0053] (4) Model verification and optimization

[0054] 1. Model Validation

[0055] A database containing a variety of anatomical structure standard models is stored in the . These standard models are constructed based on a large amount of normal human anatomical data. During model verification, the constructed three-dimensional physiological structure model is geometrically matched with the corresponding standard model. First, the corresponding anatomical feature points in the two models are determined by the feature point matching algorithm, such as the joint points of the bones, the center points of the organs, etc. Then, the Hausdorff distance between the models is calculated, which measures the maximum mismatch between the two point sets.

[0056] At the same time, manual review is conducted in combination with the professional knowledge and experience of doctors. Doctors observe the model through special visualization software to check whether the anatomical structure in the model is accurate and whether the lesion area is reasonable. If deviations are found in the model during the verification process, such as the shape or position of an organ is significantly different from the standard model, relevant information will be recorded and traced back to data collection, registration, segmentation and other links. By analyzing the intermediate result data, possible causes of the problem can be found, such as motion artifacts during image data collection, and incorrect judgments of the segmentation algorithm in specific areas.

[0057] 2. Model Optimization

[0058] Based on the verification results, the parameters and algorithms in the model construction process are optimized and adjusted using the reinforcement learning algorithm based on gradient descent. For the threshold parameters in the image segmentation algorithm, the gradient direction of the parameters is calculated according to the error feedback of the model verification, and then the threshold parameters are updated according to a certain learning rate to make the segmentation results closer to the real anatomical structure. For the weights of the deep learning model, the back propagation algorithm is used to calculate the gradient of the loss function to the weights, and the weights are updated by the gradient descent method to improve the prediction accuracy of the model.

[0059] As the system accumulates more surgical case data, the training data set is continuously expanded, and the deep learning model is retrained and iteratively updated. At the same time, according to the characteristics of different types of cases (such as different types of diseases, patients of different age groups, etc.) and complex surgical scenarios (such as multi-organ joint surgery, minimally invasive surgery, etc.), the model's structure and parameter settings are adjusted so that the model can better adapt to various practical application needs and always maintain a high level of surgical assistance performance. For example, when dealing with complex abdominal multi-organ surgical cases, the optimized model can more accurately construct a three-dimensional physiological structure model, providing more reliable planning and navigation support for surgery.

[0060] In the above scheme, the present invention integrates basic medical record information, preoperative examination results and lesion image data of multiple modalities, and this innovation is fully reflected in the data processing flow. Through carefully designed data acquisition and preprocessing links, it is ensured that different types of data can be accurately and completely collected and integrated. In the data verification and cleaning stage, a variety of advanced algorithms are used to address the characteristics of multi-source data, such as machine learning-based interpolation algorithms to process missing data, which not only considers the characteristics of single image data, but also integrates the patient's overall clinical information. This multi-source data fusion processing method breaks the limitation of traditional surgical robots that rely only on a single image or simple positioning information, and provides more comprehensive and in-depth information on the patient's physiological and pathological status for surgical planning, which helps doctors formulate more accurate and personalized surgical plans.

[0061] In addition, the use of a deep learning segmentation algorithm based on the 3D U-Net architecture for image segmentation is one of the key innovations of this technology. Compared with traditional manual feature extraction or simple threshold segmentation methods, 3D U-Net can automatically learn complex feature patterns in image data, especially when processing three-dimensional volume data. It has a stronger ability to extract contextual information, can accurately distinguish different tissue types and lesion areas, and build a more realistic and detailed three-dimensional physiological structure model. During the training process, the combination of cross entropy loss and Dice coefficient loss effectively improves the model's recognition accuracy of segmentation boundaries. This method based on big data learning greatly improves the accuracy and reliability of the model, providing a solid foundation for the precise implementation of surgery. For example, in neurosurgery, it can more accurately locate small brain tumors and important neural structures around them, reducing surgical risks.

[0062] Furthermore, the adaptive optimization mechanism in the model construction process is an important innovation of the present invention. In the model verification and optimization stage, the accuracy of the model is comprehensively evaluated by combining comparison with the standard model and manual review by doctors. The model parameters are adjusted in real time using a gradient descent-based reinforcement learning algorithm, which not only optimizes the model in the current surgical case, but also continuously iterates and updates the model construction algorithm as the surgical case data accumulates. This continuous learning ability enables the large surgical model to adapt to the ever-changing practical needs in the medical field, such as the emergence of new diseases and improvements in surgical techniques. In the face of individual differences and complex cases among different patients, the model can continuously optimize itself and always maintain a high level of surgical assistance performance, providing a strong guarantee for improving surgical quality and success rate.

[0063] When the generative surgery model performs feature analysis on the three-dimensional physiological structure model to mark the target surgical site and formulate the robot arm path accordingly, the specific processing process includes the following:

[0064] (1) Feature analysis and target surgical site marking

[0065] 1. Multi-scale feature extraction

[0066] The 3D wavelet transform algorithm is used to perform multi-scale decomposition on the 3D physiological structure model. The algorithm can decompose the model into sub-bands of different scales and directions in 3D space, which is similar to the wavelet decomposition of images, but takes into account the characteristics of volume data. At each scale, the local geometric features of the model are calculated, such as mean curvature, Gaussian curvature, and shape index. For example, for vascular structures, at a finer scale, the curvature features can highlight the curved parts and branch points of the blood vessels, which may require special attention during surgical operations.

[0067] At the same time, a feature learning method based on a three-dimensional convolutional neural network (3D CNN) is adopted. A 3D CNN architecture with multiple convolutional layers and pooling layers is constructed, and the input is the three-dimensional physiological structure model data. In the convolution layer, a small-sized three-dimensional convolution kernel (such as 3x3x3) is used to traverse the model data and extract local features. As the network depth increases, the receptive field gradually expands, and more global feature relationships can be learned. For example, in brain surgery, the network can learn the relative position relationship between different brain regions and their association with the lesion area. The geometric features extracted by wavelet transform and the features learned by 3D CNN are fused in the fully connected layer of the network to obtain a comprehensive feature representation, which provides rich information for the target surgical site marking.

[0068] 2. Feature analysis based on anatomical knowledge

[0069] A database containing a large number of standard models of human anatomical structures and pathological characteristic patterns is integrated. The database is built based on authoritative medical textbooks, clinical experience and a large amount of case data. During the analysis process, the three-dimensional physiological structure model is registered and compared with the standard model in the database. For example, for heart surgery, the patient's three-dimensional heart model is registered with the normal heart standard model, and the difference between the two is calculated. At the same time, the database stores the pathological characteristic patterns of various heart diseases, such as the typical morphology of myocardial infarction areas, the geometric characteristics of heart valve lesions, etc. By matching these patterns, the areas where lesions may exist can be quickly located.

[0070] Combined with the type of surgery and specific requirements entered by the doctor through the human-machine interface, the relevant features are further screened and focused. For example, if it is a coronary artery bypass surgery, the focus will be on the narrowed or blocked parts of the coronary artery and the surrounding vascular branches that can be used for bypass. In this way, the target surgical site can be accurately locked using anatomical knowledge and the doctor's intention.

[0071] 3. Target surgical site marking algorithm

[0072] Based on the previous feature analysis results, the conditional random field (CRF) model is used to mark the target surgical site. The CRF model can consider the spatial relationship and contextual information between features and optimize the marking results. First, the fused features are used as the input of the CRF model to define a suitable energy function, which includes a feature-based unary potential energy (reflecting the possibility that each voxel or patch belongs to the target surgical site, determined according to the previously calculated feature value) and a spatial relationship-based binary potential energy (encouraging adjacent voxels or patches to have similar labels, for example, adjacent tissues belonging to the same organ should have the same label). Then, by solving the maximum a posteriori probability problem of the CRF model, the optimal marking result is obtained, and the boundary and range of the target surgical site are accurately marked. For example, in lung surgery, the safe resection range of the lung tumor and its surroundings can be accurately marked.

[0073] (2) Robotic arm path planning

[0074] 1. Environment modeling and collision detection

[0075] The robot's visual system (such as binocular or trinocular vision cameras) is used to obtain environmental image information in the operating room, and combined with the depth information of the laser radar, a three-dimensional point cloud model of the surgical environment is constructed. At the same time, the robot's own geometric model (including the robotic arm, torso, mobile mechanism, etc.) is also converted into a point cloud form. By performing pose estimation and collision detection on the robot point cloud model in the environmental point cloud model, the feasible motion space of the robot in the current environment is determined. For example, an octree-based collision detection algorithm is used to divide the environment and robot point clouds into octree structures of different resolutions, and by quickly traversing the octree nodes, it is determined whether there is a collision risk between the robot and the environment.

[0076] 2. Path Planning Algorithm

[0077] The rapidly exploring random tree (RRT) algorithm and its improved versions (such as RRT*) are used for robot path planning. First, a search space is constructed between the starting position of the robot and the target surgical site (determined by the marking results), and points in the point cloud space are randomly sampled. Then, starting from the starting position, the sampled points are connected to the nearest node in the tree by continuously expanding the tree structure, and at the same time, the connection path is checked for collision with the environment. In the process of expanding the tree, the RRT* algorithm optimizes the path and finds a shorter and safer path by reconnecting the nodes. For example, in orthopedic surgery, in order to accurately deliver surgical instruments to the site of bone lesions, the RRT* algorithm can quickly find a collision-free robot arm movement path in a complex surgical environment (including the patient's body, surgical equipment, etc.), and after finding a feasible path, continue to optimize the path, reduce the distance and time of the robot arm movement, and improve surgical efficiency.

[0078] 3. Path optimization and safety verification

[0079] The spline curve interpolation method is used to smooth the initially planned robot arm path. For example, cubic spline curves are used to interpolate the path points to make the movement of the robot arm smoother and more continuous, reducing the impact and wear of the joints. At the same time, the path is dynamically adjusted by considering dynamic factors during the operation, such as the patient's breathing movement and the tiny body displacement caused by the heartbeat. By acquiring the patient's physiological monitoring data (such as breathing frequency, heart rate, etc.) in real time, the movement trend of the body is predicted, and the robot arm path is adjusted in advance to ensure that the surgical instrument is always accurately aimed at the target surgical site.

[0080] Before the path is executed, a comprehensive safety verification is performed. In addition to collision detection, factors such as the force of the robot arm and the limitation of the joint motion range are also considered. The robot dynamics model is used to calculate the force distribution of the robot arm during the execution of the path to ensure that the load capacity of the robot arm is not exceeded. At the same time, check whether the motion trajectory of each joint is within its allowable range of motion to avoid mechanical failure or surgical risks caused by excessive motion.

[0081] To this end, the present invention innovatively fuses multi-scale geometric features (extracted through wavelet transform) and deep learning features (learned using 3D CNN) for analysis and marking of target surgical sites. This fusion method combines the precise description of local structures by traditional geometric features with the powerful learning ability of deep learning features for global semantic information. It can more comprehensively and accurately identify complex anatomical structures and lesion features, and significantly improves the accuracy and reliability of target surgical site marking compared to single feature analysis methods. For example, in brain tumor surgery, it is possible to more accurately locate tiny tumors and their relationship with surrounding important neural structures, providing more precise guidance for surgery.

[0082] Furthermore, by integrating the anatomical knowledge database and using the deep learning model to analyze the three-dimensional physiological structure model, the collaborative work of knowledge-driven and data-driven is achieved. The anatomical knowledge database provides a priori anatomical structure and lesion pattern information to guide feature analysis and target positioning, while the deep learning model learns individual differences and potential features from actual patient data. This collaborative approach can better handle anatomical variations and complex lesions in different patients and improve the adaptability and accuracy of surgical planning. For example, when dealing with rare congenital heart disease surgeries, anatomical knowledge can be used to quickly locate key structures, and deep learning models can be used to discover patient-specific lesion characteristics and develop more personalized surgical plans.

[0083] Finally, in terms of robotic arm path planning, the innovation of this technology lies in the combination of real-time environment modeling, dynamic factor considerations and advanced path planning algorithms. The visual system and lidar are used to build a dynamic environment model to achieve real-time perception and collision detection of the surgical environment, ensuring the safety of the robotic arm movement. The RRT* algorithm and its improved version are used for path planning, combined with spline curve interpolation and smoothing, which can quickly find the optimal path in a complex dynamic environment. At the same time, the dynamic adjustment of the patient's physiological movement is considered, so that the robotic arm can accurately track the target surgical site and improve the accuracy and efficiency of surgical operations. For example, in minimally invasive abdominal surgery, even if the patient's breathing causes the position of the abdominal organs to change, the robotic arm can adjust the path in real time to ensure that the surgical instruments act stably on the target site.

[0084] Optionally, the generative surgery model includes the following process when establishing a mapping relationship between the robot arm path and the patient's position to generate a motion trajectory sequence of the robot arm;

[0085] (1) Establish the mapping relationship between the robot arm path and the patient position

[0086] 1. Coordinate system establishment and registration

[0087] First, a unified coordinate system is established in the surgical space, including the robot coordinate system (with the robot base or a fixed point as the origin), the patient coordinate system (usually with specific anatomical landmarks on the patient's body or preoperative positioning marks as the origin) and the image coordinate system (based on image data obtained before or during surgery). By using high-precision positioning equipment (such as optical trackers, electromagnetic locators, etc.), the position and posture information of the robot and the patient in space are measured, and the coordinate systems of the three are aligned. For example, in neurosurgery, an optical tracker is used to track the markers installed on the patient's head and the markers on the robot's mechanical arm to obtain their coordinates in space, thereby determining the position and orientation of the patient's head in the robot's coordinate system and achieving accurate coordinate system alignment.

[0088] For the registration of the patient coordinate system and the image coordinate system, a method based on feature point matching is used. Specific anatomical feature points (such as bone landmarks, neural structure key points, etc.) are identified in the image data (such as CT, MRI, etc.) obtained before or during surgery, and the corresponding actual position points are obtained on the patient's body through physical marking or real-time scanning. By calculating the correspondence between these points in the two coordinate systems, the conversion from the image coordinate system to the patient coordinate system is realized to ensure that the anatomical structure information in the image data accurately corresponds to the actual body position of the patient.

[0089] 2. Construction of spatial transformation model

[0090] Based on the registered coordinate system, a spatial transformation model is constructed to describe the mapping relationship between the robot path and the patient position. A homogeneous transformation matrix is ​​used to represent the transformation from the patient coordinate system to the robot coordinate system. This matrix contains translation and rotation information. By multiplying the point coordinates in the patient coordinate system by the homogeneous transformation matrix, it can be converted to the robot coordinate system.

[0091] In order to update the spatial transformation relationship in real time, the position and posture data of the robot and the patient fed back by the positioning device are continuously obtained. The data are processed using the Kalman filter algorithm, which can fuse the measurement values ​​of multiple sensors to make the best estimate of the system state (including the position and posture of the robot and the patient). By continuously updating the spatial transformation matrix, it is ensured that the mapping relationship between the robot arm path and the patient's position remains accurate during the operation. Even if the patient moves slightly during the operation due to breathing, heartbeat or body position adjustment, the mapping relationship can be adjusted in time.

[0092] (2) Generate the motion trajectory sequence of the robot arm

[0093] 1. Trajectory planning algorithm

[0094] A quintic polynomial interpolation algorithm is used to plan the motion trajectory sequence of the robot from the starting position to the target position (determined by the robot path).

[0095] The quintic polynomial interpolation algorithm can ensure the continuity of the position, velocity and acceleration of the trajectory at the starting and ending points, making the movement of the robot arm more stable and smooth, reducing the impact on surgical instruments and patient tissues. For example, in an articulated robot arm, each joint moves according to the planned quintic polynomial trajectory, and multiple joints work together to achieve accurate and smooth movement of the robot arm end effector (such as surgical instruments) in space along a predetermined path.

[0096] 2. Kinematic and dynamic constraints considerations

[0097] When generating the motion trajectory sequence, the kinematic and dynamic constraints of the robot are fully considered. Through the geometric model and kinematic equations of the robot, the accessible space and joint angle range of each joint at different positions are calculated to ensure that the planned trajectory is within the kinematic feasible domain of the robot. For example, for a robot with a rotating joint, check whether the joint angle in the trajectory exceeds the allowable range based on the rotation range limit of the joint.

[0098] At the same time, the dynamic model of the robot arm is used to calculate the torque and force required for each joint during the trajectory movement. The inverse dynamics algorithm is used to calculate the control torque required for each joint to achieve the trajectory based on the robot arm's mass distribution, moment of inertia, joint friction and other parameters, as well as the planned trajectory and movement speed. By limiting and optimizing the torque, it is ensured that the robot arm does not exceed its torque carrying capacity during movement, avoiding damage to the robot arm or loss of control of movement due to overload, while optimizing energy consumption and improving the work efficiency and stability of the robot arm.

[0099] 3. Real-time trajectory adjustment and optimization

[0100] Monitor changes in the surgical environment and patient status in real time, such as unexpected patient movement, tissue deformation, force feedback of surgical instruments, etc. When these changes are detected, immediately re-evaluate the mapping relationship between the robot arm path and the patient's position, and adjust the motion trajectory sequence according to the new situation. For example, if the patient suddenly moves his body due to pain, causing the position of the target surgical site to change, the robot arm's motion trajectory is quickly re-planned based on the new patient position information to ensure that the surgical instrument can continue to accurately act on the target site.

[0101] The trajectory is optimized using a method based on model predictive control (MPC). The MPC algorithm solves a finite time domain optimization problem in each control cycle based on the current system state (including robot position, speed, patient position, etc.) and the prediction model to determine the optimal control input (i.e., the motion instructions of the robot joints) for a period of time in the future. Through continuous rolling optimization, the robot can adapt to the dynamically changing environment, and improve the response speed and robustness of the system while ensuring surgical accuracy. For example, in laparoscopic surgery, when the contact force between the surgical instrument and the tissue changes, the MPC algorithm can adjust the trajectory of the robot in time to maintain the appropriate operating force and avoid tissue damage.

[0102] To this end, the present invention realizes a real-time and accurate mapping relationship between the robot arm path and the patient position through high-precision positioning equipment and advanced coordinate system registration and space transformation model construction methods. The Kalman filter algorithm is used to process and update the position and posture data in real time, which can dynamically adapt to the patient's movement during the operation and ensure that the robot arm is always accurately aligned with the target surgical site. This real-time dynamic adjustment capability is relatively rare in traditional surgical robots, greatly improving the safety and accuracy of the operation, especially in long-term operations or when the patient's condition is unstable.

[0103] In addition, the fifth-order polynomial interpolation algorithm is used for trajectory planning. Combined with comprehensive consideration of the kinematic and dynamic constraints of the robot arm, the generated motion trajectory sequence is smooth, accurate and efficient. The fifth-order polynomial can ensure the high-order continuity of the trajectory at the starting and ending points, reduce the impact and vibration of the robot arm movement, and facilitate the stable operation of surgical instruments. At the same time, by accurately calculating the kinematic and dynamic parameters, ensuring that the trajectory is within the feasible domain of the robot arm, and optimizing energy consumption and control torque, the working performance and service life of the robot arm are improved, which is an important improvement on the traditional trajectory planning method.

[0104] Finally, a model predictive control (MPC)-based method is introduced for real-time trajectory adjustment and optimization, so that the robot arm can respond in time to changes in the surgical environment and patient status. The MPC algorithm makes optimization decisions based on the prediction model in each control cycle, taking into account the dynamic changes of the system in the future, and can effectively deal with uncertain factors such as sudden movement of patients and tissue deformation. This robust control strategy based on model prediction improves the adaptability and reliability of the surgical robot system, provides a strong guarantee for the smooth progress of the operation, and reflects the innovative application in the field of surgical robot control technology.

[0105] Optionally, the generative surgery model includes the following process when driving the moving mechanism to move toward the patient's position to reach a specified point according to the action trajectory sequence, and controlling the robotic arm to drive the surgical instrument to perform surgery on the target surgical site:

[0106] (1) Drive the moving mechanism to move

[0107] 1. Path tracking control

[0108] According to the generated motion trajectory sequence, the path of the mobile mechanism from the current position to the specified point is decomposed into a series of discrete path points. A path tracking algorithm based on visual servoing is adopted, and the robot's visual system (such as a camera installed on the mobile mechanism) is used to obtain image information of the surrounding environment in real time, identify feature points in the path (such as marking lines on the floor of the operating room, specific landmarks, etc.), and calculate the position deviation of the mobile mechanism relative to these feature points. For example, the image feature matching algorithm is used to find the pre-set path identification features in the image, calculate its coordinates in the image plane, and then combine it with the calibration parameters of the camera to convert it into the actual position deviation in the robot coordinate system.

[0109] Based on the position deviation, a proportional-integral-differential (PID) controller is used to calculate the speed and steering control instructions of the mobile mechanism. The PID controller adjusts the position deviation in real time according to the set proportional gain, integral gain, and differential gain, so that the mobile mechanism can quickly and accurately track the predetermined path. For example, if the mobile mechanism deviates to the left in the forward direction, the PID controller will increase the speed of the right drive wheel and reduce the speed of the left drive wheel according to the size and rate of change of the deviation, so that the mobile mechanism gradually returns to the correct path. In order to improve the accuracy and stability of path tracking, fuzzy logic control is introduced on the basis of the PID controller. According to the complexity of the environment and the motion state of the mobile mechanism, the PID parameters are dynamically adjusted to meet the needs of different surgical scenarios.

[0110] 2. Obstacle Avoidance Strategy

[0111] The mobile mechanism is equipped with a variety of sensors (such as LiDAR, ultrasonic sensors, etc.) to monitor obstacles in the surrounding environment in real time. LiDAR constructs a three-dimensional point cloud map of the surrounding environment by emitting laser beams and receiving reflected light, and can accurately detect the position, shape and size of obstacles. Ultrasonic sensors are used for close-range obstacle detection as a supplement to LiDAR. For example, in the operating room, LiDAR can detect large obstacles such as operating tables and equipment carts, and ultrasonic sensors can detect small obstacles on the ground or close-range objects such as people's legs.

[0112] When an obstacle is detected, the dynamic window algorithm (DWA) is used to make obstacle avoidance decisions. The DWA algorithm searches for feasible speed combinations in the speed space (composed of the linear velocity and angular velocity of the mobile mechanism), taking into account factors such as the current speed of the mobile mechanism, acceleration limits, and the position and shape of the obstacle. By evaluating the collision risk between the motion trajectory of the mobile mechanism and the obstacle within a certain period of time under each speed combination, the optimal speed command is selected and sent to the drive system of the mobile mechanism. For example, if there is an obstacle ahead, the DWA algorithm will exclude speed combinations that may cause collisions in the speed space, and select a speed that can avoid the obstacle and get as close to the target path as possible, so that the mobile mechanism can safely and efficiently bypass the obstacle and continue to move to the designated point.

[0113] 3. Mobile mechanism positioning and posture adjustment

[0114] When the mobile mechanism approaches the designated point, a high-precision indoor positioning system (such as a positioning system based on ultra-wideband (UWB) technology) is used to accurately position the mobile mechanism. The UWB positioning system measures the distance between the mobile mechanism and multiple fixed base stations and uses the principle of triangulation to calculate the precise position coordinates of the mobile mechanism. At the same time, the inertial measurement unit (IMU) is combined to obtain the attitude information of the mobile mechanism (such as pitch angle, roll angle, and yaw angle) to ensure that the mobile mechanism has an accurate position and attitude when it reaches the designated point. For example, during brain surgery, the mobile mechanism needs to be accurately docked at a specific position next to the operating table and maintain a suitable angle with the patient's head so that the robotic arm can accurately operate the surgical instruments to reach the target surgical site in the brain.

[0115] Based on the positioning and posture information, the mobile mechanism's chassis adjustment mechanism (such as hydraulic or electric lifting device, steering mechanism, etc.) is controlled to adjust the posture so that the mobile mechanism, the patient's body and the surgical environment reach the best relative position relationship. During the adjustment process, the positioning and posture data are continuously monitored, and the feedback control algorithm is used to ensure the accuracy and stability of the adjustment, providing a stable basic platform for the operation of the robotic arm.

[0116] (2) Control the robotic arm to drive the surgical instrument

[0117] 1. Joint space control

[0118] The robot's motion trajectory sequence is converted into the target position, velocity and acceleration instructions in the joint space. According to the kinematic model of the robot, the target angle that each joint needs to reach at each moment is calculated by the inverse kinematics solution algorithm. For example, for a robot with 6 degrees of freedom, the target trajectory of the end effector (surgical instrument) in space is known, and the inverse kinematics algorithm (such as based on the geometric method or numerical iteration method) is used to calculate the corresponding angle change sequence of the 6 joints so that the end effector can move along the predetermined trajectory.

[0119] A model-based joint space controller (such as a computational torque controller) is used to control the motion of the robot joints. The computational torque controller calculates the control torque required for each joint to achieve the target trajectory based on the dynamic model of the robot. The controller takes into account factors such as the inertia, gravity load, and friction of the robot arm. By accurately calculating the control torque, the robot joints can quickly and accurately track the target trajectory while ensuring the smoothness and stability of the movement. For example, during bone drilling surgery, the robot arm needs to accurately control the position and posture of the surgical instrument. The computational torque controller can adjust the control torque of the joint in real time according to the changes in resistance during the drilling process to ensure the accuracy and depth of the drilling.

[0120] 2. Force feedback control

[0121] The robotic arm is equipped with a force sensor (such as a six-dimensional force sensor installed on the end effector) to measure the contact force and torque between the surgical instrument and the tissue in real time. The force sensor converts the measured force signal into an electrical signal and transmits it to the robot. According to the force feedback information, the motion control strategy of the robotic arm is adjusted. For example, during soft tissue surgery, if the force applied by the surgical instrument to the tissue is too large, it may cause tissue damage. Through the force feedback control algorithm, the driving torque of the robotic arm joint is reduced to keep the force of the surgical instrument on the tissue within a safe range.

[0122] Impedance control algorithm is used to realize force feedback control. Impedance control converts force feedback information into position or speed adjustment instructions of the robot arm joint according to the set target impedance model (including stiffness, damping and other parameters). For example, when the surgical instrument contacts the tissue, the appropriate target impedance is set according to the elastic characteristics of the tissue, so that the robot arm can automatically adjust its position and posture according to the deformation of the tissue while maintaining contact with the tissue, simulating the doctor's manual feel and improving the flexibility and safety of the surgical operation.

[0123] 3. Surgical instrument operation control

[0124] For different types of surgical instruments (such as scalpels, tweezers, suture needles, etc.), the robot arm is controlled to perform corresponding operations according to the surgical requirements and instrument characteristics. For example, when using a scalpel for cutting, the robot arm is controlled to drive the scalpel to perform cutting actions at a precise trajectory and speed according to the cutting path and speed planned for the operation. For clamping instruments such as tweezers, precise clamping and release operations of tissues are achieved by controlling the opening and closing angles of the robot arm joints.

[0125] Communicate and control the surgical instrument drive system (such as electric motors, pneumatic devices, etc.). According to the needs of the surgical operation, send corresponding control signals (such as motor speed, torque instructions, pneumatic valve switch signals, etc.) to the surgical instrument drive system to realize the functional operation of the surgical instrument (such as cutting, clamping, suturing, etc.). At the same time, monitor the working status of the surgical instrument in real time (such as the degree of wear of the tool, the size of the clamping force, etc.), and make adjustments or prompt the doctor to replace the instrument as needed.

[0126] To this end, the innovative point of the present invention in the mobile mechanism navigation is the combination of path tracking based on visual servoing and dynamic window method obstacle avoidance strategy. By acquiring environmental information in real time through the visual system for path tracking, the accuracy and adaptability of the mobile mechanism's navigation in a complex surgical environment are improved. Compared with the traditional navigation method based on preset paths or simple sensors, it can better cope with the dynamic changes of personnel, equipment, etc. in the operating room. At the same time, the dynamic window method obstacle avoidance strategy can achieve efficient obstacle avoidance actions while ensuring the safety of the mobile mechanism, so that the mobile mechanism can quickly and flexibly reach the designated point, providing a reliable mobile platform support for the smooth progress of the operation.

[0127] In addition, in terms of robot arm control, the combination of force feedback control and model-based joint space control is an important innovation. Force feedback control enables the robot arm to sense the force between surgical instruments and tissues in real time, and simulates the doctor's hand feel through the impedance control algorithm to achieve more flexible and safe surgical operations, effectively avoiding damage to tissues caused by improper operation. At the same time, the model-based computational torque controller uses the dynamic model of the robot arm to perform accurate torque calculations, combined with the inverse kinematics solution of the joint space, to ensure that the robot arm can track the predetermined trajectory with high precision, thereby improving the accuracy and stability of surgical operations. In addition, the idea of ​​model predictive control can also be applied to robot arm control. By predicting the future motion state and force conditions, the control strategy can be adjusted in advance, further enhancing the adaptability and robustness of the robot arm in complex surgical environments.

[0128] Furthermore, the entire surgical system integrates a variety of sensors (such as visual systems, force sensors, positioning systems, etc.) to achieve multimodal perception of the surgical environment, patient status, and robotic arm movements. These sensor data can be integrated in real time to conduct comprehensive analysis and decision-making to achieve coordinated control of the mobile mechanism and the robotic arm. This multimodal perception and collaborative control method enables the surgical system to automatically adjust the position and posture of the mobile mechanism and the operating movements of the robotic arm at different surgical stages and in complex situations, improving the overall performance and intelligence level of the surgical system and providing surgeons with more efficient, accurate, and safe surgical assistance.

[0129] Optionally, the humanoid robotic surgical system also includes a positioning module and a vision module, the positioning module is used to locate the target surgical site during the operation, and the vision module is used to visually track the robotic arm and the target surgical site during the operation: when the large surgical model performs feature analysis on the three-dimensional physiological structure model to mark the target surgical site and formulates a robotic arm path accordingly, the following steps are performed: establishing a spatial conversion relationship between the lesion image data and the positioning module, and between the positioning module and the vision module; performing feature analysis on the three-dimensional physiological structure model to mark the target surgical site; and formulating the robotic arm path for the target surgical site based on the spatial conversion relationship.

[0130] When the above steps are specifically implemented to formulate the robot arm path, the following process is included:

[0131] (1) Establishing spatial transformation relationships

[0132] 1. Coordinate system calibration and registration

[0133] First, the coordinate systems involved in the positioning module, visual module, and image data are calibrated. For the positioning module (such as an optical positioning system or an electromagnetic positioning system), multiple calibration markers are arranged at known spatial positions, the detection values ​​of these markers by the positioning module are measured, and the conversion matrix between the positioning module coordinate system and the robot coordinate system is calculated using algorithms such as the least squares method. For example, a calibration frame is set up in the operating room, on which several optical marker balls are distributed. The positioning module detects the positions of these marker balls, and determines the translation and rotation relationship of the positioning module coordinate system relative to the robot coordinate system based on the known spatial coordinates and detection coordinates of the marker balls.

[0134] For the visual module, the checkerboard calibration method or the circular calibration plate calibration method is used. The calibration plate is placed in the surgical field of view, and the visual module takes multiple images of the calibration plate at different angles. By identifying the feature points on the calibration plate (such as the corner points of the checkerboard or the center of the circular mark), the intrinsic parameter matrix (including focal length, principal point coordinates, etc.) and extrinsic parameter matrix (describing the position and posture of the visual module relative to the world coordinate system) of the visual module are calculated based on the pixel coordinates of these feature points in the image and their known coordinates in the world coordinate system (associated with the robot coordinate system). In this way, the conversion relationship between the visual module coordinate system and the robot coordinate system is established.

[0135] For lesion imaging data, the positioning markers worn by the patient before surgery (such as a head frame or body surface markers with special marks) are used to identify the positions of these markers in the imaging data. At the same time, the spatial coordinates of these markers are acquired in real time during surgery through the positioning module, thereby establishing a connection between the imaging data coordinate system and the positioning module coordinate system. Furthermore, through the established conversion relationship between the positioning module and the robot coordinate system, the imaging data coordinate system and the robot coordinate system are aligned, completing the establishment of the spatial conversion relationship between the three.

[0136] 2. Construction and optimization of spatial transformation model

[0137] Based on the calibration and registration results, a spatial transformation model is constructed. A homogeneous transformation matrix is ​​used to represent the transformation relationship between different coordinate systems. The coordinates in the positioning module coordinate system are transformed into the coordinates in the robot coordinate system, and the coordinates in the vision module coordinate system are transformed into the robot coordinate system, so as to perform unified position calculation and path planning. For example, the coordinates of a point in the positioning module coordinate system are set to ((x l ,y l ,z l ,1) T ), its homogeneous transformation matrix with the robot coordinate system is (T l ), then the corresponding coordinates in the robot coordinate system ((x r ,y r ,z r ,1) T =T l *(x l ,y l ,z l ,1) T .

[0138] In order to improve the accuracy of spatial conversion, the Kalman filter algorithm is used to fuse and optimize the measurement data of the positioning module and the visual module. The Kalman filter algorithm performs weighted fusion of the measurement data at different times based on the system's dynamic model (describing the changes in the motion state of the robot, positioning module, and visual module) and the measurement model (describing the relationship between the measurement data and the system state), and updates the spatial conversion matrix in real time. For example, when the positioning module and the visual module measure a target point at the same time, the Kalman filter algorithm calculates an optimal estimated coordinate based on the measurement accuracy of the two and the credibility of the historical data, thereby continuously correcting the spatial conversion relationship and reducing the impact of measurement errors and system noise on spatial positioning.

[0139] (2) Feature analysis of the 3D physiological structure model and marking of the target surgical site

[0140] 1. Multimodal feature extraction and fusion

[0141] The three-dimensional wavelet transform algorithm is used to perform multi-scale decomposition of the three-dimensional physiological structure model, and local geometric features at different scales are extracted, such as mean curvature, Gaussian curvature, and shape index. At the same time, a deep learning model based on a three-dimensional convolutional neural network (3DCNN) is used to learn the features of the model. A 3D CNN architecture containing multiple convolutional layers and pooling layers is constructed, and the input is the three-dimensional physiological structure model data. The convolution layer extracts local features, and the pooling layer reduces the data dimension. As the network depth increases, more global feature relationships are gradually learned. For example, in brain surgery, wavelet transform can highlight the complex folded structural features of brain tissue, and 3D CNN can learn the semantic features and relative position relationships between different brain regions.

[0142] The geometric features extracted by wavelet transform and the features learned by 3D CNN are fused in the fully connected layer of the network to obtain a comprehensive feature representation. This multimodal feature fusion method can more comprehensively and accurately identify complex anatomical structures and lesion features, providing rich information for target surgical site marking. For example, in liver surgery, the fused features can clearly show the location, size, shape of liver tumors and their relationship with surrounding blood vessels and bile ducts.

[0143] 2. Target localization based on anatomical knowledge

[0144] Access the built-in human anatomical structure standard model and lesion characteristic pattern database, which is built based on authoritative medical textbooks, clinical experience and a large amount of case data. The 3D physiological structure model is registered and compared with the standard model in the database to calculate the difference between the two. For example, for heart surgery, the patient's heart 3D model is registered with the normal heart standard model, and by analyzing the difference area, combined with the heart disease lesion characteristic patterns stored in the database (such as the typical morphology of the myocardial infarction area, the geometric characteristics of the heart valve lesions, etc.), the area where the lesion may exist can be quickly located.

[0145] According to the type of surgery and specific requirements input by the doctor through the human-machine interface, the relevant features are further screened and focused. For example, if it is a coronary artery bypass surgery, the focus will be on the narrowed or blocked part of the coronary artery and the surrounding vascular branches that can be used for bypass surgery, so as to accurately mark the boundaries and scope of the target surgical site.

[0146] (3) Formulate the robot arm path based on the spatial transformation relationship

[0147] 1. Target surgical site coordinate transformation

[0148] Using the established spatial transformation relationship, the coordinates of the marked target surgical site in the image data coordinate system or the visual module coordinate system are converted to the coordinates in the robot coordinate system. For example, the coordinates of the target surgical site in the image data are multiplied by the previously calculated homogeneous transformation matrix from the image data coordinate system to the robot coordinate system to obtain its accurate position coordinates in the robot coordinate system. This step ensures that the robot arm path planning is based on the target position in the actual workspace of the robot.

[0149] 2. Robotic Arm Path Planning Algorithm

[0150] The rapid exploration random tree (RRT) algorithm and its improved versions (such as RRT*) are used for robot path planning. First, a search space is constructed between the starting position of the robot and the converted coordinates of the target surgical site, and points in the point cloud space are randomly sampled. Then, starting from the starting position, the sampled points are connected to the nearest node in the tree by continuously expanding the tree structure, and at the same time, it is checked whether the connection path collides with obstacles in the surgical environment (real-time perception by the visual module and the positioning module). In the process of expanding the tree, the RRT* algorithm will optimize the path and find a shorter and safer path by reconnecting the nodes. For example, in orthopedic surgery, in order to accurately deliver surgical instruments to the bone lesion site, the RRT* algorithm can quickly find a collision-free robot arm movement path in a complex surgical environment (including the patient's body, surgical equipment, etc.), and after finding a feasible path, continue to optimize the path, reduce the distance and time of the robot arm movement, and improve surgical efficiency.

[0151] 3. Path optimization and safety verification

[0152] The spline curve interpolation method is used to smooth the initially planned robot arm path. For example, cubic spline curves are used to interpolate the path points to make the movement of the robot arm smoother and more continuous, reducing the impact and wear of the joints. At the same time, the path is dynamically adjusted by considering dynamic factors during the operation, such as the patient's breathing movement and the tiny body displacement caused by the heartbeat. By acquiring the patient's physiological monitoring data (such as breathing frequency, heart rate, etc.) in real time, the movement trend of the body is predicted, and the robot arm path is adjusted in advance to ensure that the surgical instrument is always accurately aimed at the target surgical site.

[0153] Before the path is executed, a comprehensive safety verification is performed. In addition to collision detection, factors such as the force of the robot arm and the limitation of the joint motion range are also considered. The robot dynamics model is used to calculate the force distribution of the robot arm during the execution of the path to ensure that the load capacity of the robot arm is not exceeded. At the same time, check whether the motion trajectory of each joint is within its allowable range of motion to avoid mechanical failure or surgical risks caused by excessive motion.

[0154] To this end, the present invention innovatively achieves precise registration and real-time optimization between the positioning module, the visual module and the lesion image data coordinate system. Through a carefully designed calibration and registration method, combined with the fusion optimization of the measurement data by the Kalman filter algorithm, it is possible to establish a spatial transformation relationship in real time and accurately during the operation, effectively solving the problem of conversion errors between different coordinate systems. This enables the large surgical model to accurately map the target surgical site information in the image data to the actual workspace of the robot, providing a solid foundation for the precise operation of the robotic arm, and significantly improving the positioning accuracy compared to traditional surgical robots.

[0155] In addition, the use of three-dimensional wavelet transform and 3D CNN deep learning model for multimodal feature extraction and fusion, combined with the target positioning method based on anatomical knowledge, is an important innovation of this technology in marking the target surgical site. This fusion method gives full play to the advantages of traditional geometric features and deep learning features, and can more accurately identify complex anatomical structures and lesion characteristics. With the assistance of anatomical knowledge, the positioning of the target surgical site is more intelligent and accurate, especially suitable for dealing with complex cases and situations with large anatomical structure variations, and improves the reliability and adaptability of surgical planning.

[0156] Furthermore, in terms of robot arm path planning, innovative applications are embodied based on spatial transformation relationships and improved RRT algorithms, combined with dynamic environment perception and real-time adjustment and optimization mechanisms. Through real-time perception of obstacles and patient body changes in the surgical environment by the positioning module and the visual module, the RRT* algorithm is used to quickly find and optimize the robot arm path, while considering the dynamic adjustment of the patient's physiological movements and the smoothing of the path, so that the robot arm can accurately and efficiently reach the target surgical site in a complex and changeable surgical environment, reduce surgical risks, and improve the accuracy and fluency of surgical operations, which is difficult to achieve with traditional surgical robot path planning.

[0157] Optionally, when the large surgical model performs feature analysis on the three-dimensional physiological structure model to mark the target surgical site and formulates the robotic arm path accordingly, the following steps are also performed: the designated point for surgery at the target surgical site, the depth of surgery, the control torque of the robotic arm, and the control strategy for the robotic arm to drive the surgical instrument to the designated point and reach the depth during the operation.

[0158] Optionally, the designated point for surgery at the target surgical site, the depth of surgery, the control torque of the robotic arm, and the control strategy of the robotic arm driving the surgical instrument to the designated point and reaching the depth during the surgery are implemented through the following key steps:

[0159] (1) Determine the designated point

[0160] 1. Point planning based on anatomical structure and surgical requirements

[0161] First, preliminary planning of designated points is carried out based on the built-in human anatomical structure database and surgical type knowledge base. For different surgeries, such as brain surgery, orthopedic surgery, etc., the key locations that surgical instruments need to reach are determined according to the purpose of the surgery and common operating specifications. For example, in brain tumor resection surgery, based on the location and size of the tumor and the distribution of important surrounding neural structures, combined with standard surgical approaches and operating requirements, several possible surgical instrument entry points and key operating points are determined. These points should be convenient for approaching the tumor while minimizing damage to normal brain tissue.

[0162] Then, the three-dimensional physiological structure model is used to determine the points more accurately. Through detailed analysis of the target surgical site and its surrounding tissues in the model, the physical properties of the tissues, such as hardness, elasticity, and vascular distribution, are considered to optimize and adjust the initially planned points. For example, in bone surgery, if drilling or implantation is planned at a certain bone location, the density, strength and other parameters of the bone at that location will be analyzed, combined with the direction of the surrounding blood vessels and nerves, to determine the safest and most effective operation point to avoid damage to important structures.

[0163] The point selection is verified and optimized using a finite element analysis (FEA)-based method. The three-dimensional physiological structure model is converted into a finite element model to simulate the stress distribution and deformation of the surrounding tissue when the surgical instrument is operated at different points. According to the simulation results, the point that minimizes tissue stress and has the most reasonable deformation is selected as the final designated point. For example, in soft tissue surgery, finite element analysis can predict the degree of traction and compression of the surrounding soft tissue when the surgical instrument is operated at different points, thereby determining the operation point with the least impact on the tissue.

[0164] (2) Determine the surgical depth

[0165] 1. Image data and model analysis to assist depth judgment

[0166] Perform in-depth analysis on lesion imaging data (such as CT, MRI, etc.), and use image segmentation algorithms to accurately segment the target surgical site and its surrounding tissues from the image. Then, based on the grayscale values, texture features, and other information of different tissues in the image, determine the boundaries and range of the lesion tissue, thereby determining the depth that the surgery needs to reach. For example, in tumor resection surgery, by analyzing the appearance of the tumor in the image, its depth position in the tissue and the boundary with the surrounding normal tissue are determined, providing a preliminary basis for the depth of the surgery.

[0167] Combined with the three-dimensional physiological structure model, the three-dimensional morphology and spatial relationship of the tissue are considered. For some lesions with complex shapes or irregular distribution, the model can provide more comprehensive spatial information to help more accurately determine the depth of surgery. For example, in cardiac surgery, the three-dimensional structure of the heart is complex. Through the model, we can intuitively understand the depth of the lesion in the heart wall and its relationship with the internal chambers of the heart, so as to determine the appropriate surgical depth and avoid penetrating the heart wall or damaging the internal structure of the heart.

[0168] Depth prediction using deep learning algorithms. Based on a large amount of surgical case data, a deep neural network model is trained, with the patient's preoperative imaging data and three-dimensional physiological structure model features as input, and the predicted surgical depth as output. The model can learn the potential relationship between lesion characteristics and surgical depth in different cases, providing a reference for determining surgical depth. For example, in ophthalmic surgery, based on the patient's eye imaging data and model information, the neural network model can predict the appropriate depth to be achieved during cataract surgery or retinal surgery.

[0169] (3) Calculate the control torque of the robot arm

[0170] 1. Kinetic model construction and parameter identification

[0171] Construct an accurate dynamic model of the robot arm, which takes into account factors such as the joint structure, connecting rod mass, moment of inertia, friction, gravity load, etc. The various parameters in the model are determined by combining theoretical calculations with experimental measurements. For example, for the connecting rod mass and moment of inertia of the robot arm, the theoretical values ​​can be obtained through CAD modeling and material density calculation, and then the theoretical parameters are corrected by actually measuring the force and acceleration of the robot arm in different motion states using algorithms such as the least squares method to obtain accurate dynamic model parameters.

[0172] During the operation, the joint position, velocity and acceleration information of the robot arm, as well as the contact force information between the surgical instrument and the tissue (measured by force sensors) are obtained in real time. This information is input into the dynamic model, and the control torque required for each joint is calculated using dynamic principles such as the Newton-Euler equation or the Lagrange equation. For example, during tissue cutting surgery, when the surgical instrument encounters tissue resistance, the joint control torque that needs to be increased is calculated based on the dynamic model to ensure that the surgical instrument can operate according to the predetermined trajectory and force.

[0173] Adaptive control algorithms are used to dynamically adjust the control torque. Since tissue characteristics and environmental conditions may change during surgery, the adaptive control algorithm automatically adjusts the parameters or control gains in the dynamic model based on real-time measured error signals (such as the deviation between the actual motion trajectory and the target trajectory, the deviation between the actual contact force and the expected contact force, etc.), so that the robot arm can adapt to different surgical situations and always maintain stable and accurate operation. For example, in soft tissue surgery, as the tissue is removed, the hardness and elasticity of the tissue may change. The adaptive control algorithm can adjust the control torque in time to ensure the operating accuracy of the surgical instrument.

[0174] (4) Develop a control strategy for the robotic arm to drive the surgical instrument to the specified point and depth

[0175] 1. Integration of trajectory planning and control algorithms

[0176] The segmented trajectory planning method is used to divide the entire movement process of the robot arm from the initial position to the designated point and reaching the predetermined depth into multiple stages. In the stage of approaching the designated point, a trajectory planning algorithm based on visual servoing is used, and the visual module is used to monitor the relative position of the robot arm and the target surgical site in real time. The movement trajectory of the robot arm is adjusted through visual feedback to ensure that it reaches the designated point accurately. For example, when the robot arm approaches the target surgical site, the visual module continuously measures the deviation between the end effector of the robot arm and the target point, and uses the proportional-integral-differential (PID) controller to calculate the joint movement speed command based on the deviation information, so that the robot arm can quickly and accurately approach the target point.

[0177] At the depth-reaching stage, force feedback control and position control algorithms are combined. The contact force between the surgical instrument and the tissue is monitored in real time through a force sensor. When the contact force reaches a certain threshold, it indicates that the surgical instrument begins to contact the target tissue. At this time, it switches to force control mode and controls the propulsion speed and strength of the robotic arm according to the preset surgical depth and tissue characteristics, so that the surgical instrument slowly and steadily reaches the predetermined depth. For example, during bone drilling surgery, when the drill bit contacts the bone surface, the feed speed and rotation torque of the drill bit are adjusted through force feedback control according to the hardness of the bone and the drilling depth requirements to ensure that the drilling depth is accurate and does not damage surrounding tissues.

[0178] The entire motion process is optimized and coordinated using the model predictive control (MPC) algorithm. Based on the dynamic model, current state, and future motion goals of the robot arm, the MPC algorithm solves a finite time domain optimization problem in each control cycle, predicts the motion trajectory and control torque of the robot arm in the future, and adjusts the current control strategy based on the prediction results. For example, the MPC algorithm can predict in advance the changes in tissue resistance and the force on the robot arm joints that may be encountered in the process of reaching the specified point and depth, adjust the trajectory and control torque in advance, improve the accuracy and stability of the robot arm movement, and optimize energy consumption and surgical time.

[0179] To this end, the innovation of the present invention in determining the designated points and surgical depth lies in the comprehensive use of multi-source information such as anatomical structure database, surgical type knowledge, image data, three-dimensional physiological structure model, and deep learning algorithm. Through the fusion of this multi-source information, various surgical factors such as anatomical structure complexity, tissue characteristics, and lesion characteristics can be considered more comprehensively and accurately. Compared with traditional surgical robots that rely only on a single information source or simple empirical judgment, the accuracy of the designated points and surgical depth determination is greatly improved, providing a more reliable guarantee for the successful implementation of the operation.

[0180] In addition, in terms of calculating the control torque of the robot arm, building an accurate dynamic model and combining it with an adaptive control algorithm is an important innovation. By considering in detail the various physical characteristics of the robot arm and the motion information and contact force information obtained in real time, the control torque is calculated using the principle of dynamics, making the operation of the robot arm more in line with the actual physical laws. The adaptive control algorithm can automatically adjust the torque according to the dynamic changes during the operation, ensuring that the robot arm can work stably and accurately under different tissue conditions, and improving the adaptability and operation accuracy of the robot arm to complex surgical environments. This is difficult to achieve with traditional torque control methods based on experience or simple models.

[0181] Furthermore, the developed robot arm control strategy integrates a variety of advanced technologies such as visual servo, force feedback control and model predictive control, achieving optimization and precise control of the entire robot arm movement process. Visual servo provides high-precision position guidance in the stage of approaching the target point, force feedback control ensures the safety and accuracy of the operation in the stage of reaching the depth, and model predictive control coordinates and optimizes the entire process, enabling the robot arm to accurately reach the designated point and reach the depth according to the predetermined trajectory and requirements in a complex and changeable surgical environment. At the same time, it improves the response speed and robustness of the system, providing strong support for the refinement and intelligence of surgical operations.

[0182] Optionally, the humanoid robotic surgical system also includes: a wireless module for receiving the patient's real-time physiological monitoring data and real-time medical imaging data through a constructed wireless channel, so that the large surgical model can analyze the real-time physiological monitoring data and real-time medical imaging data, and generate a surgical recommendation plan based on the analysis results.

[0183] Specifically, the key technologies in the working process of the above wireless module are as follows:

[0184] (1) Wireless module data reception and transmission

[0185] 1. Wireless communication protocols and security mechanisms

[0186] The wireless module uses high-speed, stable and secure wireless communication protocols, such as Wi-Fi 6 (802.11ax) or 5G communication technology, to ensure that the patient's real-time physiological monitoring data (such as heart rate, blood pressure, blood oxygen saturation, etc.) and real-time medical imaging data (such as intraoperative ultrasound, endoscopic images, etc.) can be received quickly and reliably. During the communication process, strict data encryption algorithms, such as Advanced Encryption Standard (AES) encryption, are implemented to ensure the security and privacy of data transmission. For example, the physiological monitoring device worn by the patient encrypts the data and transmits it to the wireless module through the Wi-Fi 6 network to prevent the data from being stolen or tampered with during transmission.

[0187] In order to ensure the integrity and accuracy of the data, the wireless module adopts a data checksum and retransmission mechanism. When receiving data, the checksum of the data is calculated and compared with the checksum provided by the sender. If the checksum is inconsistent, the sender is immediately requested to retransmit the data. At the same time, a data buffer is set to temporarily store and sort the received data to ensure that the data is transmitted in the correct order for subsequent processing. For example, when receiving intraoperative ultrasound image data, the wireless module will check each frame of data. If an error frame is found, the ultrasound device is promptly requested to retransmit to ensure the continuity and integrity of the image data.

[0188] 2. Signal strength optimization and interference suppression

[0189] The wireless module is equipped with smart antenna technology, which can automatically adjust the direction and gain of the antenna to optimize the wireless signal strength. By real-time monitoring of signal quality indicators (such as received signal strength indicator (RSSI), signal-to-noise ratio (SNR), etc.), when the signal strength is weak or interfered, the antenna parameters are automatically adjusted to ensure a stable communication connection. For example, in the case of a variety of electronic devices in the operating room that may generate electromagnetic interference, the smart antenna can avoid the interference source by adjusting the direction, while increasing the gain to enhance the reception of weak signals.

[0190] Interference suppression algorithms, such as frequency domain interference cancellation technology or adaptive filtering technology, are used to reduce the interference of other wireless devices in the surrounding environment on data transmission. Frequency domain interference cancellation technology analyzes the spectrum characteristics of the interference signal and subtracts it from the received signal at the receiving end to improve the quality of the useful signal. Adaptive filtering technology dynamically adjusts the filter parameters according to the real-time changes of the interference signal to effectively filter out the interference components. For example, when other Wi-Fi devices or Bluetooth devices in the operating room generate interference, the wireless module uses these interference suppression algorithms to ensure stable reception of patient data.

[0191] (2) Analysis of surgical data by large-scale model and generation of surgical suggestion plan

[0192] 1. Real-time physiological monitoring data analysis

[0193] Perform real-time analysis on the received real-time physiological monitoring data. First, establish a dynamic model of physiological parameters. For example, use the autoregressive moving average (ARMA) model or the Kalman filter model to model heart rate, blood pressure and other data to predict the changing trend of physiological parameters. By comparing the actual measured values ​​with the predicted values, abnormal fluctuations in physiological parameters can be discovered in time. For example, if the patient's heart rate suddenly deviates from the normal range and does not match the predicted trend, this abnormality can be quickly detected.

[0194] Use machine learning algorithms to classify physiological data and conduct risk assessment. Based on a large amount of clinical case data, classification models such as support vector machines (SVM), decision trees, or neural networks are trained to classify physiological data into categories such as normal, mildly abnormal, moderately abnormal, and severely abnormal. At the same time, surgical risks are assessed based on different abnormality categories and degrees, combined with the type of surgery and the patient's basic conditions. For example, in cardiac surgery, if blood pressure continues to drop and is classified as severely abnormal, combined with the surgical progress and the patient's heart function status, the large surgical model may determine that the patient is at risk of insufficient blood supply to the heart and needs to take corresponding measures.

[0195] 2. Real-time medical imaging data analysis

[0196] For real-time medical imaging data, image segmentation and feature extraction algorithms are used for analysis. For example, different tissues and structures (such as tumors, blood vessels, organs, etc.) in the image are segmented using deep learning-based image segmentation algorithms (such as the U-Net architecture), and then the shape, size, texture and other features of the segmented area are calculated. In brain surgery, brain tumors and important neural structures around them can be accurately segmented, and features such as morphological changes of the tumor and adhesion to surrounding tissues can be analyzed.

[0197] By comparing and analyzing real-time image features with preoperative image data and standard anatomical models, the progress of the operation and whether there are any abnormalities can be determined. For example, in orthopedic surgery, the real-time X-ray images during the operation are compared with the bone position and implant position planned before the operation to detect whether the bone reduction is accurate and whether the implant has shifted. If abnormalities are found, the surgical strategy can be adjusted in a timely manner.

[0198] 3. Surgical suggestion planning generation algorithm

[0199] Based on the analysis results of real-time physiological monitoring data and real-time medical imaging data, the surgical model uses a rule-based reasoning system and a case-based reasoning system to generate surgical advice plans. The rule-based reasoning system directly generates countermeasures based on pre-set medical rules and expert experience, such as "when blood pressure is below a certain threshold and lasts for more than a certain period of time, suspend the surgical operation and take corresponding measures." The case-based reasoning system searches for similar cases in the historical surgical case database, draws on past successes or failures, and provides reference suggestions for the current operation. For example, if the current operation encounters physiological data abnormalities and imaging features similar to those in a previous case, the surgical model can refer to the treatment methods in the case and combine them with the current actual situation to develop a personalized surgical advice plan, such as adjusting the operating parameters of surgical instruments, changing the surgical approach, and taking emergency hemostasis measures.

[0200] To this end, the innovation of the present invention in terms of wireless modules lies in the use of advanced wireless communication technology and security mechanisms to achieve high-speed, stable and secure transmission of patients' real-time physiological monitoring data and medical imaging data. High-speed communication protocols ensure timely data transmission and meet the real-time requirements during surgery; strict data encryption and verification mechanisms ensure data accuracy and privacy, which is crucial in the field of medical data transmission. At the same time, smart antennas and interference suppression technologies improve the adaptability and reliability of wireless modules in complex surgical environments, effectively solving the problems of signal interference and instability, and have obvious advantages over traditional wireless transmission methods.

[0201] In addition, the fusion analysis of real-time physiological monitoring data and medical imaging data by the surgical large model is an important innovation. By establishing a dynamic model of physiological parameters and using machine learning algorithms to classify and evaluate physiological data, and combining image segmentation and feature extraction technology to analyze imaging data, a comprehensive and real-time assessment of the patient's surgical status is achieved. This multimodal data fusion analysis can more promptly and accurately identify potential risks in surgery, provide doctors with a more comprehensive basis for decision-making, and help improve the safety and success rate of surgery. It is especially suitable for the surgical management of complex surgeries and high-risk patients.

[0202] Furthermore, in terms of generating surgical advice plans, the combination of rule-based reasoning and case-based reasoning reflects innovative applications. Rule-based reasoning uses the knowledge of medical experts to ensure the standardization and accuracy of responses to common problems, while case-based reasoning can mine personalized experience from a large number of historical cases to provide references for solving complex and rare surgical situations. This intelligent reasoning system can generate targeted surgical advice plans based on the real-time situation of the operation, helping doctors make reasonable decisions quickly and improving the flexibility and adaptability of the operation. It is an innovative development direction in the field of surgical decision-making assistance.

[0203] Optionally, the number of the robotic arms is at least two, each robotic arm is composed of multiple joints of more than 5 axes connected in series, and the end of each robotic arm forms a wrist, wherein the wrist of one robotic arm is installed with a main surgical instrument, and the wrist of at least one of the remaining robotic arms is installed with an auxiliary surgical instrument.

[0204] Specifically, in terms of robotic arm structure and joint control:

[0205] 1. Joint drive and kinematic modeling

[0206] Each joint of the robot arm is equipped with a high-performance motor drive system, such as a brushless DC motor (BLDCM) or an AC servo motor, which can provide precise torque control and fast response speed. The motor driver communicates with the motor to send speed, position and torque control instructions. For each joint, an accurate kinematic model is established, including forward kinematics and inverse kinematics models. The forward kinematics model is used to calculate the position and posture of the end of the robot arm (wrist) in space based on the joint angle, which is achieved by multiplying a series of coordinate transformation matrices. For example, for a 5-axis robot arm, starting from the base joint, the coordinate transformation of each joint is calculated in turn, and finally the position and posture of the wrist in the robot coordinate system are obtained. The inverse kinematics model is used to calculate the joint angle based on the desired wrist position and posture, which is usually solved by numerical iteration methods (such as the Newton-Raphson method) or analytical solutions (such as geometric methods, for certain specific structures of the robot arm). For example, when the main surgical instrument needs to be moved to the target surgical site, the inverse kinematics model is used to calculate the angle that each joint needs to rotate, and then the motor driver is used to control the motor to rotate the corresponding angle.

[0207] In order to improve the accuracy and stability of joint control, a closed-loop control strategy is adopted. Position sensors (such as encoders) and speed sensors (such as tachometer generators or speed calculations through encoder signals) are installed on each joint to provide real-time feedback of the actual position and speed information of the joint. The feedback information is compared with the target position and speed instructions, and the error signal is calculated. The control input of the motor is then calculated through a PID (proportional-integral-differential) controller or a more advanced controller (such as a model predictive controller), and the output torque of the motor is adjusted so that the joint can quickly and accurately track the target motion trajectory. For example, when performing delicate surgical operations, such as suturing, the PID controller adjusts the motor torque according to the position error to ensure that the robotic arm can accurately control the position and posture of the surgical instrument and achieve tiny and precise movements.

[0208] 2. Robotic Arm Collaborative Motion Planning

[0209] When two or more robotic arms work together, complex motion planning is required. First, a 3D model of the entire surgical scene is built, including a model of the patient's body, a model of surgical instruments, and models of other objects in the surgical environment. According to the requirements of the surgical task, such as simultaneous tissue cutting and auxiliary traction operations, the task objectives and action sequence of each robotic arm are determined. For example, the robotic arm with the main surgical instrument is responsible for precise tissue cutting, while the robotic arm with the auxiliary surgical instrument is responsible for pulling the tissue at the right position and time to expose the surgical field of view.

[0210] A priority-based task allocation and path planning algorithm is used. A priority is set for each robotic arm, which is determined according to the urgency and importance of the surgical task. When planning the path, the path is planned for the robotic arm with a high priority first to ensure that it can successfully complete the main surgical operation. At the same time, the movement space and operation requirements of the robotic arm with a low priority are considered to avoid collisions. For example, in brain surgery, if the main surgical instrument needs to quickly reach the tumor deep in the brain for resection, its path planning has the highest priority, and the path of the auxiliary robotic arm is planned without affecting the operation of the main surgical instrument. It may be necessary to wait for the main surgical instrument to reach a certain position before taking action to avoid the risk of collision.

[0211] Collision detection and avoidance algorithms are used to ensure safety between the robotic arms and between the robotic arms and the surgical environment. By real-time monitoring of the position and motion state of the robotic arms and surrounding objects, a collision detection algorithm based on bounding boxes (such as axis-aligned bounding boxes (AABB) or oriented bounding boxes (OBB)) is used to quickly determine whether there is a risk of collision. If a potential collision is detected, the movement path or speed of the robotic arms is adjusted in time. For example, when two robotic arms are operating in a small space, once a possible collision is detected, a pre-set collision avoidance strategy will be used, such as reducing the movement speed, adjusting the movement direction, or pausing the movement of a certain robotic arm to ensure the safety of the operation.

[0212] Specifically, in terms of surgical instrument installation and operation control:

[0213] 1. Rapid replacement of surgical instruments and interface design

[0214] A standardized surgical instrument interface is designed on the wrist of the robotic arm to quickly and easily replace different types of main and auxiliary surgical instruments. The interface uses a combination of mechanical locking and electrical connection to ensure that the surgical instruments are firmly installed and the electrical signal transmission is stable. For example, the interface has a precise positioning pin and bayonet structure. When the surgical instrument is inserted, it can automatically lock and align the electrical connection contacts to achieve the transmission of motor drive signals, sensor signals (such as force sensor signals), etc. By identifying the specific coding or sensor signal on the interface, the type of surgical instrument installed is automatically identified, and the corresponding control program and parameters are loaded. For example, when a scalpel is installed, a control algorithm suitable for scalpel operation will be called to adjust the motor torque and speed limit to meet the needs of cutting tissue; when replaced with tweezers, it will switch to a control mode suitable for clamping operations.

[0215] 2. Main surgical instrument operation control algorithm

[0216] For the operation of the main surgical instruments, a special control algorithm is used according to the type of surgery and the characteristics of the instrument. Taking electrosurgical instruments as an example, such as high-frequency electric knives, the output power, frequency and waveform of the electric knife need to be precisely controlled to achieve tissue cutting, coagulation or mixed mode operation. By real-time monitoring of parameters such as the contact force between the surgical instrument and the tissue (using the force sensor on the wrist) and tissue electrical impedance (by measuring the current and voltage relationship between the electric knife electrode and the tissue), the output parameters of the electric knife are adjusted using a feedback control algorithm. For example, when cutting tissue, if the contact force increases, indicating that the tissue is harder or the cutting depth increases, the electric knife power will be appropriately increased to ensure the cutting effect; at the same time, according to the change in tissue electrical impedance, the tissue type (such as muscle, fat, blood vessels, etc.) is judged, and the frequency of the electric knife is automatically adjusted to optimize the cutting and coagulation effects and reduce thermal damage to surrounding tissues.

[0217] When performing minimally invasive surgery, such as laparoscopic surgery, the main surgical instruments need to be precisely operated in a small space, while overcoming the movement restrictions of the instruments when passing through the Trocar hole. Taking advantage of the flexibility and joint control accuracy of the robotic arm, combined with visual feedback (obtaining the surgical field image through the laparoscope camera), a control algorithm based on visual servoing is adopted. By identifying the surgical target and anatomical landmarks in the image, the movement deviation of the robotic arm is calculated, and then the joint angle is adjusted so that the main surgical instrument can accurately reach the target position and operate. For example, when suturing intra-abdominal tissue, according to the relative position of the suture needle and the tissue in the laparoscopic image, the robotic arm is controlled to drive the surgical instrument to perform precise suturing movements to ensure the suturing quality.

[0218] 3. Assisted surgical instrument operation control strategy

[0219] The operation control of auxiliary surgical instruments is intended to cooperate with the main surgical instruments to complete surgical tasks and improve surgical efficiency and safety. For example, when the auxiliary surgical instrument is a traction forceps, the opening and closing angle and traction force of the traction forceps are controlled according to the progress of the operation and the operational requirements of the main surgical instrument. The appropriate traction position and direction are determined by real-time monitoring of the surgical field of view (such as through a visual module) and the position of the main surgical instrument. For example, in orthopedic surgery, when the main surgical instrument is drilling a bone, the auxiliary surgical instrument is responsible for pulling the surrounding soft tissue to avoid affecting the drilling operation. According to the drilling position and the distribution of the soft tissue, the position and strength of the traction forceps are automatically adjusted to ensure the smooth progress of the operation.

[0220] For some auxiliary surgical instruments with special functions, such as aspirators or irrigators, their working status is controlled according to the needs of the operation. For example, during the operation, if bleeding occurs, the aspirator is automatically started and the suction power is adjusted according to the signal of the blood detection sensor (such as an optical sensor to detect changes in blood color and concentration), and the blood in the surgical field is cleared in time to ensure a clear surgical field; at the same time, according to the surgical site and tissue cleaning requirements, the flushing flow and direction of the irrigator are controlled to keep the surgical area clean, which is conducive to surgical operations and reduces the risk of infection.

[0221] To this end, the innovation of the present invention in terms of robot arm control is to achieve high-precision motion control of multi-joint serial robot arms with more than 5 axes. By adopting high-performance motor drive, precise kinematic modeling and closed-loop control strategy, combined with advanced controller algorithms, it is possible to achieve precise position and posture control of the end of the robot arm in a complex surgical space. Compared with traditional robot arms, this high-precision control enables surgical instruments to reach the target surgical site more accurately and perform more delicate surgical operations, such as the operation of tiny neural structures in neurosurgery, which improves the success rate and safety of the operation.

[0222] In addition, the collaborative work of multiple robotic arms is one of the important innovations of this technology. By establishing a surgical scene model, priority-based task allocation and path planning algorithms, as well as collision detection and avoidance algorithms, efficient and safe collaborative work of multiple robotic arms in a small surgical space is achieved. This collaborative mode can simulate the collaboration between different members of the surgical team, give full play to the function of each robotic arm, and improve surgical efficiency. For example, in complex abdominal surgeries, multiple operations can be performed simultaneously, reducing surgical time and patient risks, which is difficult to achieve with traditional single-robotic-arm surgical systems.

[0223] Furthermore, the standardized surgical instrument interface design and adaptive control algorithm of the robotic arm wrist reflect innovative applications. The quick replacement interface facilitates the timely replacement of appropriate surgical instruments according to different surgical stages and needs during the operation, improving the flexibility of the surgical system. At the same time, the adaptive control algorithm for different types of surgical instruments can automatically adjust the control parameters according to the characteristics of the instrument and the requirements of the surgical operation to achieve the best surgical effect. Whether it is the precise energy control of electrosurgical instruments or the visual servo operation of minimally invasive surgical instruments, it provides surgeons with more powerful and convenient surgical operation tools, which helps to promote the development of surgical technology.

[0224] Optionally, the torso is a humanoid torso having at least two degrees of freedom so as to match the surgical operation during the operation to perform at least one of forward and backward bending, left and right side bending, and height self-adjustment.

[0225] To this end, in terms of degree of freedom drive and kinematic model:

[0226] 1. Joint drive and sensor configuration

[0227] Each degree of freedom of the humanoid torso is realized by a corresponding drive device, such as an electric motor or a hydraulic / pneumatic drive. The motor driver or control valve communicates with these drive devices and sends control signals to achieve precise motion control. At the joints of each degree of freedom, position sensors (such as rotary encoders or linear displacement sensors) and angle sensors (such as gyroscopes or inclination sensors) are installed to provide real-time feedback of the position, angle, and motion state information of the torso. For example, at the joints of the degrees of freedom that realize forward and backward bending, the rotary encoder can accurately measure the rotation angle of the joint, and the gyroscope can detect the angular velocity and posture changes during the bending process. These sensor data provide accurate feedback for closed-loop control.

[0228] For the degrees of freedom driven by electric motors, high-performance servo motors are used, which have high-precision position control and fast response capabilities. The target position or angle that each degree of freedom needs to reach is calculated according to the surgical operation requirements, and then the control input (such as voltage or current) of the motor is calculated through a PID (proportional-integral-differential) controller or a model predictive controller (MPC) to enable the motor-driven joint to move accurately to the target position. For example, during brain surgery, when the doctor needs to adjust the height and angle of the surgical instrument, the height adjustment degree of freedom and bending degree of freedom of the humanoid torso are controlled according to the surgical plan and the doctor's operating instructions, and the motor output is accurately adjusted through the PID controller to ensure rapid and stable adjustment of the torso posture, so that the surgical instrument can accurately reach the target surgical site in the brain.

[0229] 2. Kinematic model establishment and solution

[0230] In order to accurately control the movement of the humanoid torso, its kinematic model is established. For rotational degrees of freedom such as bending forward and backward and bending sideways, the kinematic model is established using the rotation matrix method based on the joint space. For example, for a simple humanoid torso model with two rotational degrees of freedom (bending forward and backward and bending sideways), by multiplying the rotation matrix of each degree of freedom, the position and posture transformation relationship of the torso end (such as the installation position of the robotic arm) in space can be obtained. Let R x (θ x ) is set to rotate θ around the x-axis (front and rear bending axis) x The rotation matrix of the angle, R y (θ y ) is the rotation θ around the y-axis (left and right bending axis) y The total transformation matrix from the initial posture of the torso to the current posture is T = R x (θ x )R y (θ y ). Through this model, the joint angle required for each degree of freedom can be inversely solved according to the target position and posture to achieve inverse kinematics solution.

[0231] For the self-adjustment degree of freedom of height (if it is linear motion), a model based on linear kinematics is established. According to the type of drive device (such as a screw-nut mechanism or a hydraulic / pneumatic telescopic rod), determine the relationship between the height and the motion parameters of the drive device (such as the number of motor revolutions or the stroke of the hydraulic rod). For example, for a height adjustment mechanism driven by a screw nut, the pitch p of the screw is known, and the relationship between the number of motor revolutions n and the height change h is h=pn. The target height is calculated according to the surgical requirements, and then the height is precisely adjusted by controlling the number of motor revolutions. In practical applications, in order to improve the smoothness and accuracy of the movement, it may be necessary to optimize the kinematic model and consider the influence of factors such as gravity and friction. For example, when calculating the motor control input, a gravity compensation term is added to ensure stable height adjustment under different load conditions.

[0232] Specifically, in terms of matching algorithms with surgical procedures:

[0233] 1. Posture planning based on surgical scenarios

[0234] The surgical scene information, including the patient's position, the position of the surgical instrument, and the direction of operation, is obtained through visual modules (such as cameras installed in the operating room or the robot's own visual system) and surgical instrument position sensors. The optimal posture plan for the humanoid torso is formulated in combination with the type of surgery and the operational requirements of the current surgical stage. For example, in abdominal surgery, when the doctor needs to operate on organs deep in the abdominal cavity, the forward and backward bending angles and height adjustments required by the humanoid torso are calculated based on the length of the surgical instrument, the depth of the surgical site, and the patient's position, so that the robotic arm can approach the surgical site at the optimal angle and distance while avoiding collisions with the patient's body or surgical equipment.

[0235] An AI-based path planning algorithm (such as a reinforcement learning algorithm) is used to optimize the motion trajectory of the humanoid torso. The surgical scene is regarded as a dynamic environment, and the posture adjustment of the humanoid torso is regarded as a series of actions. Through the reinforcement learning algorithm, the robot learns the optimal posture adjustment strategy in a large number of simulated surgical scenes. For example, a reward function is defined. When the robotic arm can reach the target surgical site accurately and quickly and remain safe throughout the process (such as no collision), a positive reward is given; otherwise, a negative reward is given. After multiple trainings, the robot can automatically select the best torso posture adjustment path according to different surgical scenes, improving surgical efficiency and safety.

[0236] 2. Real-time feedback and dynamic adjustment

[0237] During the operation, the motion state of the humanoid torso and the operation feedback of the surgical instruments are monitored in real time. Through sensor data (such as joint position sensors, force sensors, etc.) and the status information of the surgical instruments (such as the force of the instruments, cutting depth, etc.), it is judged whether the current torso posture meets the surgical operation requirements. If deviations or adjustments are found, dynamic adjustments are made immediately. For example, when performing delicate ophthalmic surgery, if the doctor needs to slightly change the angle of the surgical instrument during the operation, the small posture adjustment required by the humanoid torso is quickly calculated according to the fine-tuning instructions entered by the doctor through the human-machine interface or the operation intention detected according to the force change of the surgical instrument. By controlling the corresponding degrees of freedom, real-time dynamic adjustments are achieved to ensure that the surgical instruments are always kept in the best operating position and angle.

[0238] Taking into account dynamic factors during the operation, such as the patient's respiratory movement, slight body displacement, etc., the posture of the humanoid torso is compensated and adjusted. By acquiring the patient's physiological monitoring data (such as respiratory rate, heart rate, etc.) in real time and monitoring the patient's body position through the visual module, the patient's body movement trend is predicted, and the posture of the humanoid torso is adjusted in advance to ensure the relative position of the surgical instrument and the surgical site is stable. For example, in chest surgery, when the patient breathes, the chest will rise and fall. According to the breathing cycle and amplitude, the height and angle of the humanoid torso are dynamically adjusted, so that the robotic arm can be adjusted accordingly with the patient's respiratory movement, ensuring that the surgical instrument always acts accurately on the target surgical site.

[0239] To this end, the innovation of the present invention in the control of the humanoid torso lies in the realization of precise motion control of at least 2 degrees of freedom. Through advanced drive devices, high-precision sensors and precise kinematic models, the humanoid torso can accurately achieve forward and backward bending, left and right side bending, and height self-adjustment during surgery. This precise control capability enables the robot to better adapt to different surgical sites and operation requirements. Compared with traditional surgical auxiliary equipment with fixed structures or simple motion mechanisms, it significantly improves the accessibility and operational flexibility of surgical instruments, provides surgeons with a wider operating space and a more comfortable operating angle, and helps to improve the accuracy and efficiency of surgery.

[0240] In addition, the use of visual modules and surgical instrument sensor information to achieve posture planning and real-time dynamic adjustment based on the surgical scene is an important innovation. By obtaining comprehensive information about the surgical scene and combining artificial intelligence algorithms (such as reinforcement learning), the robot can automatically plan the best trunk posture according to the actual situation of the operation, and adapt to various changes in real time during the operation, including changes in patient position, changes in surgical instrument operation requirements, and dynamic movements of the patient's body. This intelligent posture adjustment capability improves the robot's adaptability and autonomy to complex surgical environments, can better cooperate with doctors in surgical operations, reduce doctors' physical exertion and difficulty in operation during the operation, and improve the safety and success rate of the operation.

[0241] Furthermore, innovative applications are reflected in the compensation and adjustment of the humanoid torso posture by considering dynamic factors during the operation (such as patient breathing, etc.). By real-time monitoring of the patient's physiological data and body position changes, the displacement of the surgical site caused by the patient's dynamic movement is predicted and compensated, ensuring that the relative position of the surgical instrument and the surgical site remains stable. This technology is of great significance in improving the accuracy of surgical operations, especially in operations that require extremely high surgical accuracy (such as neurosurgery, ophthalmic surgery, etc.). It can effectively reduce surgical errors caused by patient movement and improve surgical quality, which is a function that is difficult to achieve with traditional surgical auxiliary equipment.

[0242] Optionally, the moving mechanism is at least one of a humanoid leg-and-foot type, a wheel type or a track type moving mechanism.

[0243] Specifically, for the humanoid leg-foot mobile mechanism:

[0244] 1. Gait planning and control

[0245] The gait of the humanoid leg-foot mobile mechanism is planned according to the surgical scene and patient position information. First, a kinematic model of the leg is established, including the relationship between the position and angle of each joint (such as the hip joint, knee joint, and ankle joint) and the overall movement of the leg. Through the inverse kinematics algorithm, the required angle change sequence of each joint is calculated according to the target moving position and posture. For example, for a simple planar leg model, the inverse kinematics method based on geometric analysis is used to calculate the angles of the hip joint, knee joint, and ankle joint according to the leg length and the target foothold position.

[0246] A gait planning algorithm based on model predictive control (MPC) is used. The MPC algorithm predicts the leg motion trajectory of the next few steps in each control cycle based on the current leg state (including joint angle, speed, acceleration, etc.) and environmental information (such as ground flatness, obstacle position, etc.), and selects the gait sequence that optimizes a certain performance indicator (such as minimum energy consumption, maximum movement stability, etc.) through an optimization algorithm (such as quadratic programming). For example, if the floor of the operating room is relatively flat but there are some small obstacles such as equipment cables, the MPC algorithm will consider avoiding these obstacles when planning the gait, and adjust the step length and step frequency to minimize energy consumption and ensure stable movement.

[0247] During the gait execution process, the actual movement state and force conditions of the joints are fed back in real time through sensors installed in the leg joints (such as encoders, gyroscopes, force sensors, etc.). This feedback information is compared with the planned gait, and the output torque of the joint motor is adjusted using a PID (proportional-integral-differential) controller or an adaptive controller to ensure that the legs move accurately according to the planned gait. For example, if there is a deviation between the actual joint angle and the target angle, the PID controller adjusts the motor torque according to the deviation size and change rate to quickly return the joint to the target angle; when the ground is uneven and the leg force changes, the adaptive controller adjusts the motor parameters according to the force sensor feedback to ensure stable support and movement of the legs.

[0248] 2. Balance control and environmental adaptation

[0249] In order to keep the humanoid leg-foot mobile mechanism balanced during movement, a balance control algorithm based on sensor fusion is adopted. The data from multiple sensors such as gyroscopes, accelerometers, and vision modules are integrated to calculate the posture and center of gravity position changes of the mobile mechanism in real time. For example, the extended Kalman filter algorithm is used to fuse the data of the gyroscope and accelerometer to obtain more accurate posture information. At the same time, the visual module's perception of the surrounding environment and the ground is combined to determine whether there is a tilt or unevenness.

[0250] The compensation action of each leg joint is calculated according to the balance state. When the body is detected to be leaning forward, the center of gravity is moved backward to maintain balance by adjusting the leg joint angle. A hierarchical control structure is adopted. The upper layer calculates the expected angle change of the joint according to the overall balance requirements, and the lower layer realizes precise control of the motor through the joint controller (such as the calculated torque controller) to actually move the joint to the desired angle. For example, when crossing a small obstacle, the upper layer control plans the action sequence of leg lifting and crossing according to the height and position of the obstacle, and the lower layer control ensures that the leg joints move accurately during the execution process while maintaining the body balance, so that the mobile mechanism can smoothly pass the obstacle and reach the designated surgical position.

[0251] Specifically, for wheeled mobile mechanisms:

[0252] 1. Path tracking and speed control

[0253] The target path of the wheeled mobile mechanism is determined according to the surgical plan, and the path is decomposed into a series of discrete path points. A path tracking algorithm based on visual servoing is used to obtain the surrounding environment image using the camera installed on the mobile mechanism, identify the feature points in the path (such as ground marking lines, specific landmarks, etc.), and calculate the position deviation of the mobile mechanism relative to these feature points. For example, the image feature matching algorithm is used to find the pre-set path identification features in the image, calculate its coordinates in the image plane, and then combine it with the calibration parameters of the camera to convert it into the actual position deviation in the robot coordinate system.

[0254] Based on the position deviation, a proportional-integral-differential (PID) controller is used to calculate the speed and steering control instructions of the wheeled mobile mechanism. The PID controller adjusts the position deviation in real time according to the set proportional gain, integral gain, and differential gain, so that the mobile mechanism can quickly and accurately track the predetermined path. For example, if the mobile mechanism deviates to the left in the forward direction, the PID controller will increase the speed of the right drive wheel and reduce the speed of the left drive wheel according to the size and rate of change of the deviation, so that the mobile mechanism gradually returns to the correct path. In order to improve the accuracy and stability of path tracking, fuzzy logic control is introduced on the basis of the PID controller. According to the complexity of the environment and the motion state of the mobile mechanism, the PID parameters are dynamically adjusted to meet the needs of different surgical scenarios.

[0255] 2. Obstacle avoidance strategy and precise positioning

[0256] The wheeled mobile mechanism is equipped with a variety of sensors (such as LiDAR, ultrasonic sensors, etc.) for real-time monitoring of obstacles in the surrounding environment. LiDAR constructs a three-dimensional point cloud map of the surrounding environment by emitting laser beams and receiving reflected light, and can accurately detect the position, shape and size of obstacles. Ultrasonic sensors are used for close-range obstacle detection as a supplement to LiDAR. For example, in an operating room, LiDAR can detect large obstacles such as operating tables and equipment carts, while ultrasonic sensors can detect small obstacles on the ground or close-range objects such as people's legs.

[0257] When an obstacle is detected, the dynamic window algorithm (DWA) is used to make obstacle avoidance decisions. The DWA algorithm searches for feasible speed combinations in the speed space (composed of the linear velocity and angular velocity of the mobile mechanism), taking into account factors such as the current speed of the mobile mechanism, acceleration limits, and the position and shape of the obstacle. By evaluating the collision risk between the motion trajectory of the mobile mechanism and the obstacle within a certain period of time under each speed combination, the optimal speed command is selected and sent to the drive system of the mobile mechanism. For example, if there is an obstacle ahead, the DWA algorithm will exclude speed combinations that may cause collisions in the speed space, and select a speed that can avoid the obstacle and get as close to the target path as possible, so that the mobile mechanism can safely and efficiently bypass the obstacle and continue to move to the designated point.

[0258] When approaching the designated surgical site, a high-precision indoor positioning system (such as a positioning system based on ultra-wideband (UWB) technology) is used to accurately position the wheeled mobile mechanism. The UWB positioning system measures the distance between the mobile mechanism and multiple fixed base stations and uses the principle of triangulation to calculate the precise position coordinates of the mobile mechanism. At the same time, the inertial measurement unit (IMU) is combined to obtain the posture information of the mobile mechanism (such as pitch angle, roll angle, and yaw angle) to ensure that the mobile mechanism has an accurate position and posture when it reaches the designated point, so that the robotic arm can accurately operate the surgical instrument to reach the target surgical site.

[0259] For crawler mobile mechanisms:

[0260] 1. Terrain adaptation and power distribution

[0261] The parameters of the tracked mobile mechanism are adjusted based on the perception of the terrain of the surgical environment by the visual module and other sensors (such as terrain scanners). For different terrains (such as flat ground, slopes, obstacles, etc.), the appropriate tension, speed and steering angle of the track are calculated. For example, when climbing a slope, the driving force of the track is increased, and the front and rear tension distribution of the track is adjusted according to the slope angle to ensure that the mobile mechanism can climb stably without slipping. A terrain adaptation algorithm based on fuzzy control is used to fuzzily divide different terrain types according to the roughness, slope and other characteristics of the terrain, and correspond to different control strategies. For example, when it is detected that the terrain is more rugged, the fuzzy controller will appropriately reduce the moving speed and increase the grip of the track to prevent the mobile mechanism from bumping or losing control during movement.

[0262] In complex terrain, flexible movement is achieved by controlling the differential steering of the tracks. The speed difference of the tracks on both sides is calculated according to the target path and the current position, so that the mobile mechanism can turn smoothly in narrow spaces or around obstacles. For example, when there are some large equipment blocking the operating room and the tracked mobile mechanism needs to bypass these equipment to reach the operating location, the speed difference of the tracks on both sides can be precisely controlled to achieve a compact turning radius and quickly reach the designated location.

[0263] 2. Stability control and movement accuracy

[0264] In order to ensure the stability of the crawler mobile mechanism during movement, the inertial measurement unit (IMU) and pressure sensor installed on the chassis monitor the posture of the mobile mechanism and the contact pressure between the track and the ground. Based on this information, the movement parameters of the track are adjusted to prevent the mobile mechanism from rolling over or tilting. For example, when it is detected that the pressure on one side of the track is too high and may cause a rollover, the speed of the track on that side is reduced and the driving force of the track on the other side is increased to restore the balance of the mobile mechanism.

[0265] When it reaches the vicinity of the surgical site, a vision-based precise positioning method is used. The camera installed on the mobile mechanism is used to capture images of the surrounding environment, which are matched with the pre-stored surgical scene map to determine the precise position of the mobile mechanism. Through image feature extraction and matching algorithms (such as the scale-invariant feature transform (SIFT) algorithm), key feature points in the image are identified and compared with the feature points in the map to calculate the position deviation of the mobile mechanism. Then, by adjusting the movement of the crawler, the mobile mechanism can accurately reach the designated surgical site, providing a stable basic platform for the surgical operation of the robotic arm.

[0266] To this end, the innovation of the present invention in the humanoid leg-foot mobile mechanism lies in the use of gait planning based on model predictive control and balance control algorithm based on sensor fusion. The MPC algorithm can plan the optimal gait sequence according to the complex surgical environment and mobile requirements, improve the mobile efficiency and adaptability, and take into account factors such as energy consumption, making the robot's movement in the operating room more intelligent. The sensor fusion balance control algorithm can accurately sense the posture and center of gravity changes of the mobile mechanism, adjust the leg movements in real time, and ensure stable balance under various terrains and operating conditions. This is a relatively advanced technology in the traditional humanoid robot mobile control, and provides a guarantee for the flexible movement of the surgical robot in a complex surgical environment.

[0267] In addition, for wheeled mobile mechanisms, path tracking based on visual servoing and dynamic window method obstacle avoidance strategy combined with high-precision positioning system is an important innovation. Visual servo path tracking improves the accuracy and adaptability of path tracking, and can flexibly adjust the moving path according to different visual signs in the operating room. The dynamic window method obstacle avoidance strategy can quickly and safely avoid obstacles in complex environments, ensuring that the mobile mechanism reaches the surgical site smoothly. High-precision positioning systems (such as UWB positioning) further improve the positioning accuracy of mobile mechanisms in surgical scenarios, laying the foundation for precise surgical operations of robotic arms, and significantly improving navigation and positioning compared to traditional wheeled mobile mechanisms.

[0268] Furthermore, the innovation of the crawler mobile mechanism lies in the terrain adaptation algorithm based on fuzzy control and the precise positioning method based on vision. Fuzzy control can automatically adjust the parameters and movement mode of the crawler according to different terrain characteristics, so that the mobile mechanism has a strong ability to adapt to terrain and can move stably in various complex surgical environment terrains (such as slopes, obstacles, etc.). The precise positioning method based on vision can provide high-precision positioning when reaching the surgical site, ensuring that the mobile mechanism is accurately docked and providing a stable platform for the operation of the robotic arm. This combination of terrain adaptation and precise movement is a relatively novel application of traditional crawler mobile mechanisms in surgical scenarios, which improves the overall reliability and practicality of surgical robots.

[0269] Optionally, a multi-language translation model is also deployed on the host, so as to enable the large surgical model to interact with the doctor in multiple languages ​​through the human-computer interaction interface.

[0270] Specifically, for multilingual translation model architecture and training:

[0271] 1. Neural network architecture selection

[0272] The host uses a neural network based on the Transformer architecture as the core of the multi-language translation model. The Transformer architecture can process different parts of the input text in parallel through a multi-head attention mechanism, effectively capturing the semantic and grammatical relationships in the text. For example, when processing medical terms and sentences related to surgery, the multi-head attention mechanism can simultaneously focus on the professional meaning of the terms, the structure of the sentences, and contextual information, thereby improving the accuracy of the translation. The model consists of two parts: an encoder and a decoder. The encoder converts the input source language text into a series of semantic vector representations, and the decoder generates the translated text in the target language based on these vectors.

[0273] In view of the particularity of the medical field, the Transformer architecture has been adaptively adjusted. The embedding layer of the medical terminology library and medical knowledge graph has been added, so that the model can better understand and process medical professional vocabulary during the translation process. For example, when encountering a medical term such as "cardiac catheterization", the embedding layer can directly obtain its corresponding medical concepts and related knowledge, rather than simply translating according to general language rules, thereby ensuring the professionalism and accuracy of the translation.

[0274] 2. Training Data and Methods

[0275] A large amount of medical multilingual text data is collected as training sets, including medical literature, medical records, surgical reports, etc., covering multiple language pairs (such as English-Chinese, English-French, English-German and other common medical communication language pairs). In the data preprocessing stage, the text is cleaned, segmented, stemmed, and other operations are performed to improve data quality. For example, for Chinese text, a professional medical Chinese word segmentation tool is used to accurately segment sentences into medical terms and common words to facilitate model learning.

[0276] A supervised training method is used, with the objective function of minimizing the translation loss. The loss function uses cross entropy loss to measure the difference between the translation text generated by the model and the real target language text. During the training process, stochastic gradient descent (SGD) or its variant algorithms (such as Adagrad, Adadelta, etc.) are used to optimize the model parameters. At the same time, in order to prevent overfitting, regularization techniques such as L1 and L2 regularization, Dropout, etc. are used. For example, when training the English-Chinese medical translation model, a large number of English medical texts and their corresponding Chinese translation texts are used for training. The model continuously adjusts parameters so that the generated Chinese translation is as close as possible to the real translation text in terms of semantics, grammar, and medical expertise. After multiple iterations of training, the model gradually converges and has the ability to accurately translate medical texts.

[0277] Specifically, for integration with large surgical models and human-computer interaction interfaces:

[0278] 1. Data interaction and conversion

[0279] When the doctor inputs text instructions or questions through the human-computer interaction interface, the human-computer interaction interface first performs language recognition on the input text to determine its source language. Then, the text data is transmitted to the multilingual translation model. The multilingual translation model converts the source language text into a unified internal representation (such as a semantic representation based on word vectors or character vectors) for translation processing. For example, if the doctor enters a French surgical instruction, the language recognition module recognizes it as French, and the translation model converts the French instruction into a vector representation, and then translates it according to the trained French-target language (such as English or Chinese, etc., languages ​​supported by the surgical model) translation rules, and then converts the translated text into a format that the surgical model can understand, and transmits it to the surgical model for processing.

[0280] After the surgical model processes the relevant information, the generated responses or suggestions may also need to be translated into multiple languages ​​so that they can be presented in the doctor's native language. For example, the surgical model generates an English surgical risk assessment report based on the patient's condition and surgical plan. The multilingual translation model translates it into the doctor's native language (such as Chinese), and then displays it to the doctor through the human-computer interaction interface, achieving seamless multilingual communication between the surgical model and the doctor.

[0281] 2. Real-time and accuracy guarantee

[0282] In order to ensure the real-time nature of multi-language interaction, efficient model reasoning algorithms and hardware acceleration technologies are used. In the process of model reasoning, cache mechanisms and parallel computing technologies are used to reduce translation time. For example, for some common medical phrases and sentences, their translation results are cached and directly called when encountered again to avoid repeated translation calculations. At the same time, graphics (GPU) or dedicated artificial intelligence chips (such as TPU) are used to accelerate the calculation process of neural networks, improve translation speed, and meet the needs of rapid communication during surgery.

[0283] In order to ensure the accuracy of translation, a quality assessment module is set up in the translation model. This module evaluates the translation results based on indicators such as language model score and semantic similarity. If the evaluation result is lower than the set threshold, the translation model will try to adopt other translation strategies or perform secondary translation optimization. For example, when translating a complex medical case description, the quality assessment module finds that the translated text is semantically ambiguous or inaccurate. The model will re-analyze the sentence structure, adjust the translation parameters, or refer to more medical knowledge graph information to correct the translation results to ensure that doctors can accurately understand the information provided by the large surgical model.

[0284] To this end, the innovation of the present invention in the multilingual translation model lies in the use of a Transformer architecture optimized for the medical field. By introducing a medical terminology library and a knowledge graph embedding layer, the model can better understand and process medical professional texts, thereby improving the accuracy and professionalism of medical multilingual translation. Compared with traditional general multilingual translation models, this optimized architecture can more accurately translate medical instructions, medical records, surgical reports, etc. in surgical scenarios, avoiding misunderstandings caused by incorrect translation of medical terminology, and providing reliable language support for international medical exchanges and multilingual surgical collaboration.

[0285] In addition, the seamless integration of the multilingual translation model with the surgical large model and the human-computer interaction interface is an important innovation. It enables natural multilingual communication between doctors and the surgical large model. No matter what language the doctor uses, he can interact with the surgical system smoothly and obtain surgery-related information and suggestions. At the same time, through real-time guarantee measures (such as caching and hardware acceleration) and accuracy optimization mechanisms (such as quality assessment and secondary translation), the efficiency and quality of multilingual interaction are improved. This is a relatively new function in traditional surgical assistance systems, which greatly expands the application scope of surgical robots and facilitates doctors with different language backgrounds to use the system for surgical operations and communication.

[0286] Optionally, the humanoid robotic surgery system further includes: a holographic projection module, configured to project the three-dimensional physiological structure model into a holographic image through spatial mapping.

[0287] Specifically, for holographic projection data processing and conversion:

[0288] 1. First, optimize the 3D physiological structure model data to meet the requirements of holographic projection. Since holographic projection has high requirements for data accuracy and resolution, a voxel-based supersampling algorithm is used to process the model data to increase the details and clarity of the model. For example, for the organ surface in the original 3D physiological structure model, more voxels are inserted on the basis of the original voxel grid through the supersampling algorithm to make the surface smoother and more delicate, thereby improving the visual effect after holographic projection.

[0289] The optimized three-dimensional physiological structure model data is converted into a format suitable for holographic projection equipment, such as hologram data format (a format supported by devices such as HoloLens) or light field data format. During the conversion process, the light field distribution information corresponding to the model data is calculated according to the principle of holographic projection. For example, mathematical tools such as Fourier transform are used to convert the spatial domain data of the three-dimensional model into frequency domain data, and then the light field amplitude and phase distribution on the holographic projection plane are calculated based on the diffraction theory of holographic projection, and this information is encoded into a format that can be recognized by the holographic projection equipment for subsequent projection operations.

[0290] 2. Spatial mapping and coordinate calibration

[0291] In order to achieve accurate spatial mapping of holographic images, the positioning module and the visual module are used to obtain the three-dimensional coordinate information of the surgical space. The positioning module determines the position and posture of the holographic projection device in the surgical space, and the visual module is used to scan the surgical environment and the patient's body to obtain the spatial position information of the surrounding objects. Then, the coordinate system of the three-dimensional physiological structure model is calibrated with the surgical space coordinate system. For example, by identifying the position of specific marking points or anatomical landmarks in the operating room in the model coordinate system and the actual space coordinate system, the coordinate conversion matrix between the two is calculated to ensure that the holographic image can be accurately projected to the spatial position corresponding to the patient's body, realizing the precise integration of the virtual model and the real surgical scene.

[0292] Specifically, when the positioning module determines the position and posture of the holographic projection device, the following are the key links in the accurate spatial mapping process of the above holographic image:

[0293] (1) Positioning module determines the position and posture of the holographic projection device

[0294] The position and posture of the holographic projection device in the surgical space are described by coordinate transformation, which is generally represented by a homogeneous transformation matrix.

[0295] A right-handed Cartesian coordinate system O is established in the surgical space (world coordinate system) w -X w Y w Z w , and the holographic projection device itself has a device coordinate system O d -X d Y d Z d The positioning module can obtain the translation vector t=[t x ,t y ,t z ] T (Indicates that in X w , Y w , Z w The translation in three directions) and rotation information (usually represented by the rotation matrix R).

[0296] The rotation matrix R is a 3×3 orthogonal matrix that describes the rotation angle of the device coordinate system relative to the world coordinate system around each coordinate axis. w ) The rotation matrix R of the axis rotation angle α x (α) is:

[0297]

[0298] Around Y w The rotation matrix R of the axis rotation angle β y (β) is:

[0299]

[0300] Around Z w The rotation matrix R of the axis rotation angle γ z (γ) is:

[0301]

[0302] If the device goes around X w , Y w , Z w The axes are rotated by angles α, β, and γ respectively (the order is important, and different orders have different results). Then the total rotation matrix R can be obtained by multiplying them in sequence, such as when rotating in the order of (Z), (Y), and (X):

[0303] R=R x (α)R y(β)R z (γ)

[0304] Then the homogeneous transformation matrix from the device coordinate system to the world coordinate system is It can be expressed as:

[0305]

[0306] Where 0 T =[0,0,0]. Thus, any point p in the device coordinate system d =[x d ,y d ,z d ,1] T The coordinate p in the world coordinate system w =[x w ,y w ,z w ,1] T It can be converted by the following formula:

[0307]

[0308] These parameters (t and R) obtained by the positioning module determine the position and posture of the holographic projection device in the surgical space, which is the basis for subsequent spatial mapping.

[0309] (2) The visual module obtains the spatial position information of surrounding objects

[0310] When the visual module scans the surgical environment and the patient's body to obtain the spatial position information of objects, it often involves technologies such as three-dimensional reconstruction in computer vision. The commonly used method is based on the principle of multi-view geometry and processes images taken from different perspectives.

[0311] The visual module has two cameras (or it can be equivalent to dual-viewing by taking photos from different positions at different times). The optical center positions in the world coordinate system are O1 and O2, and the corresponding camera coordinate systems are O c1 -X c1 Y c1 Z c1 and O c2 -X c2 Y c2 Z c2 .

[0312] For a point P in space, the coordinates in the two camera coordinate systems are p c1 =[x c1 ,y c1 ,z c1 ] T and p c2 =[x c2 ,yc2 ,z c2 ] T , whose coordinates in the world coordinate system are p w =[x w ,y w ,z w ] T .

[0313] First, the coordinates in the camera coordinate system need to be converted to the world coordinate system, which involves the extrinsic matrix of each camera (similar to the homogeneous transformation matrix of the holographic projection device mentioned above, including rotation and translation information). Suppose the extrinsic matrix of camera 1 is The extrinsic parameter matrix of camera 2 is Then we have:

[0314]

[0315] At the same time, there is an essential matrix E and a basic matrix F between the two camera coordinate systems, which satisfy the following relationship (based on the principle of epipolar geometry):

[0316]

[0317] The basic matrix F has a certain relationship with the essential matrix E (assuming that the camera intrinsic parameter matrices are K1 and K2 respectively):

[0318]

[0319] By matching feature points in the image (for example, using feature extraction algorithms such as SIFT and SURF to find the pixel coordinates of the same feature points in the two camera images (m1 = [u1, v1, 1] T ) and (m2=[u2,v2,1] T )), combined with the camera intrinsic parameters, we can first calculate the basic matrix (F), and then get the essential matrix (E). Then, through the singular value decomposition and other methods, we can solve the coordinates of the spatial point (P) in the world coordinate system, and realize the acquisition of the spatial position information of the surrounding objects.

[0320] (3) Coordinate system calibration and calculation of coordinate transformation matrix

[0321] Assume that the three-dimensional physiological structure model has its own model coordinate system ( m -X m Y m Z m ), it is necessary to compare it with the surgical space coordinate system (world coordinate system (O w -X w Y w Z w )) is calibrated by identifying specific markers or anatomical landmarks in the operating room.

[0322] Assume that there are (n) corresponding marker points (or anatomical landmark points), whose coordinates in the model coordinate system are (p mi =[x mi ,y mi ,z mi ] T )((i=1,2,…,n)), the coordinates in the world coordinate system are (p wi =[x wi ,y wi ,z wi ] T )((i=1,2,…,n)).

[0323] The coordinate transformation matrix from the model coordinate system to the world coordinate system can be solved by the least squares method (It is also in the form of a homogeneous transformation matrix containing rotation and translation information).

[0324] First, write the coordinate transformation relationship as follows (for each corresponding point):

[0325] p wi =Rp mi +t

[0326] Where (R) is the rotation matrix and (t) is the translation vector. We need to find the appropriate (R) and (t) to minimize the error. Construct the error function:

[0327]

[0328] By minimizing the error function mentioned above through optimization algorithms such as singular value decomposition (SVD), we can obtain the rotation matrix R and translation vector (t), and then determine the coordinate transformation matrix for:

[0329]

[0330] In this way, any point (p m =[x m ,y m ,z m ,1] T ) in the world coordinate system (surgical space coordinate system) (p w =[x w ,y w ,z w ,1] T ) can be converted using the following formula:

[0331]

[0332] Through the above series of formula calculations based on coordinate transformation, multi-view geometry and least squares method, accurate spatial mapping of the holographic image is achieved, ensuring that the holographic image can be accurately projected to the spatial position corresponding to the patient's body, completing the precise integration of the virtual model and the actual surgical scene.

[0333] Specifically, for holographic projection equipment control and image projection:

[0334] 1. Laser light source and optical system control

[0335] The laser light source in the holographic projection module is a key component for generating holographic images. According to the light field information in the hologram data, the intensity, frequency and phase of the laser light source are precisely controlled. For example, for the bright area in the hologram, the intensity of the laser is increased, and for the dark area, the intensity is reduced. The laser light source is precisely adjusted by controlling the laser driver through digital signals. At the same time, the position and angle of the reflectors, lenses and other components in the optical system are controlled to adjust the propagation path and focusing effect of the laser beam to ensure that the laser beam can accurately irradiate the holographic projection medium (such as the light scattering area in the holographic film or air medium) to form a clear holographic image.

[0336] Adaptive optics technology is used to compensate for optical distortion caused by environmental factors (such as air flow, temperature changes, etc.). By installing a wavefront sensor in the optical system, the wavefront distortion of the laser beam is monitored in real time, and then the wavefront is corrected using adjustable optical elements such as deformable mirrors. For example, when the air flow in the operating room causes the laser beam to bend slightly, the wavefront sensor detects the change in the wavefront, calculates the adjustment parameters of the deformable mirror based on the detection results, and causes the deformable mirror to deform accordingly, thereby correcting the wavefront of the laser beam and ensuring the stability of the quality of the holographic image.

[0337] 2. Dynamic holographic image updates and interactions

[0338] During the operation, as the three-dimensional physiological structure model is updated in real time (such as changes in the model due to patient movement or surgical operations), the updated model data is promptly converted into holographic projection data and projected as a holographic image. An incremental data update algorithm is used to transmit only the part of the data that has changed in the model, reducing the amount of data transmission and processing time, and achieving rapid updates of holographic images. For example, when the surgical instrument cuts or moves the tissue, the corresponding part of the tissue in the model changes. Only the data of this part of the change is calculated and transmitted, and the corresponding area in the holographic image is updated, allowing doctors to see the impact of the surgical operation on the patient's body structure in real time.

[0339] In order to realize the interaction between doctors and holographic images, the holographic projection module is equipped with interactive sensors (such as gesture recognition sensors, voice recognition sensors, etc.). The doctor's interactive instructions are obtained through these sensors, such as the doctor rotating and scaling the holographic image through gestures, or querying the information of specific tissues through voice. Then, the holographic image is operated accordingly according to the interactive instructions, such as changing the viewing angle of the holographic image, displaying detailed information of specific tissues, etc. For example, when the doctor makes a rotation gesture, the gesture recognition sensor transmits the gesture information to the holographic image, calculates the rotation angle and speed of the holographic image according to the direction and speed of the gesture, and then updates the holographic projection data, so that the holographic image rotates according to the doctor's intention, which is convenient for doctors to observe the patient's physiological structure from different angles, and improves the accuracy of surgical decision-making and the convenience of surgical operations.

[0340] To this end, the innovation of the present invention in holographic projection data processing lies in the use of voxel supersampling and light field data conversion algorithms, combined with precise spatial mapping and coordinate calibration technology. Voxel supersampling improves the visual quality of holographic projection, allowing the three-dimensional physiological structure model to present richer details and higher clarity in the holographic image, which is very important for doctors to accurately observe the patient's body structure. Accurate spatial mapping and coordinate calibration realize the precise fusion of holographic images and surgical space, accurately projecting the virtual physiological structure model to the corresponding position of the patient's body, providing intuitive and accurate visual assistance for surgical operations, and greatly improving the visualization and accuracy of surgery compared to traditional two-dimensional images or simple three-dimensional display methods.

[0341] In addition, in terms of holographic projection equipment control, adaptive optical technology is an important innovation for compensating for optical distortion caused by environmental factors. It can ensure the stability of the quality of holographic images in complex surgical environments, without being overly affected by factors such as air flow and temperature changes, ensuring that doctors can always see the holographic images clearly. Dynamic holographic image update and interaction technology makes holographic images more interactive and real-time. Doctors can interact with holographic images through natural interactive methods such as gestures and voice, adjust the display content and viewing angle of the image in real time, and can see the changes in the model in time during the operation. This interactivity and real-time nature provide more flexible and efficient auxiliary means for surgical operations. It is an innovative development of traditional holographic projection technology in surgical applications, which helps to improve the efficiency and success rate of surgery.

[0342] Optionally, the host is also deployed with an auxiliary decision and control component to respond to surgical query instructions captured through the human-computer interaction interface and return a response result, wherein the response result includes at least one of: surgical operation suggestions, surgical step planning suggestions, hand speed risk assessment, postoperative suggestions, and intraoperative suggestions.

[0343] Specifically, for data collection and preprocessing:

[0344] 1. Multi-source data integration

[0345] First, collect surgery-related data from multiple data sources. This includes obtaining surgical query instructions entered by doctors through the human-computer interaction interface, basic medical history information of patients, preoperative examination results, lesion imaging data, and real-time physiological monitoring data (such as heart rate, blood pressure, blood oxygen saturation, etc.) and real-time medical imaging data (such as intraoperative ultrasound, endoscopic imaging, etc.) from the operation process. Integrate and standardize these data so that they can be effectively used by decision-making support and control components. For example, convert image data in different formats into a unified image format, perform structured processing on text data in medical record information, and extract key information such as disease name, medical history duration, allergy history, etc.

[0346] Data cleaning algorithms are used to remove noise and erroneous information from the data. For image data, image filtering algorithms are used to remove speckle noise, salt and pepper noise, etc. in the image to improve image quality; for text data, grammar and semantic checking algorithms are used to correct typos, incorrect medical terminology, etc. For example, in the medical record text, if an incorrect term such as "high snow pressure" (should be "hypertension") appears, the data cleaning algorithm can identify and correct it to ensure the accuracy and reliability of the data, providing a good data foundation for subsequent analysis and decision-making.

[0347] Specifically, for analysis based on machine learning and knowledge rules:

[0348] 1. Machine learning model construction and training

[0349] Build a variety of machine learning models for different types of decision analysis. For example, for surgical risk assessment, a classification model is built using the random forest algorithm. Collect a large amount of historical surgical case data, including the patient's various physiological indicators, disease characteristics, surgery type, and corresponding surgical results (such as whether complications occur, whether the surgery is successful, etc.), and divide these data into training sets and test sets. Use the training set to train the random forest model, and optimize the model performance by adjusting parameters such as the number of decision trees and the depth of the tree. During the training process, the model learns the complex relationship between different features (such as patient age, blood pressure, tumor size, etc.) and surgical risks. For example, the model may find that older patients with multiple underlying diseases have a higher surgical risk when undergoing complex surgery.

[0350] For surgical step planning suggestions, a sequence-to-sequence (Seq2Seq) model is used, such as an architecture based on a recurrent neural network (RNN) or a long short-term memory network (LSTM). The surgical step records of a large number of successful surgical cases are used as training data, and the initial conditions of the surgery (such as the patient's condition, type of surgery, etc.) are used as input sequences, and the surgical step sequence is used as the output sequence for training. The model learns the reasonable sequence of surgical steps and key points of operation in different surgical scenarios. For example, in heart bypass surgery, the Seq2Seq model can generate planning suggestions for specific surgical steps including vascular acquisition and vascular anastomosis based on input information such as the patient's coronary artery blockage and heart function.

[0351] 2. Construction and application of knowledge rule base

[0352] Establish a medical knowledge rule base to convert the experience of medical experts and the knowledge in medical guidelines into computer-recognizable rules. For example, formulate rules on surgical contraindications, such as "If the patient's platelet count is less than 50×10□ / L, major surgery is prohibited unless special platelet supplementation measures are taken." In the decision-making support process, when a surgical query instruction is received, a preliminary judgment and screening is first performed based on the knowledge rule base. For example, if a doctor asks whether a patient with a low platelet count is suitable for surgery, the system directly gives a preliminary contraindication prompt by querying the knowledge rule base, and then combines the analysis results of the machine learning model to provide more comprehensive decision-making recommendations.

[0353] The output of the machine learning model is integrated with the knowledge rule base. For example, in surgical risk assessment, the machine learning model may give a risk probability value based on data, and the rules in the knowledge rule base can modify or supplement this probability value. If the model predicts that a certain operation has a medium risk, but the knowledge rule base points out that the risk of the operation will increase significantly under certain circumstances (such as if the patient has a special allergy history), then the final risk assessment result will take both into account, provide doctors with more accurate and detailed risk warnings, and improve the scientificity and reliability of decision-making.

[0354] Specifically, for response result generation and feedback:

[0355] 1. Personalized response generation

[0356] Generate personalized response results based on the doctor's surgical query instructions and the patient's specific situation. For example, when a doctor asks about a patient's surgical risk assessment, the decision support and control component comprehensively considers factors such as the patient's age, gender, disease type, physical condition, and surgery type. If an elderly diabetic patient undergoes orthopedic surgery, the system will focus on analyzing factors such as the impact of diabetes on wound healing and the cardiovascular risks of elderly patients, and combine the predictions of the machine learning model with the provisions of the knowledge rule base to generate a detailed surgical risk assessment report, including possible complications (such as infection, cardiovascular and cerebrovascular accidents, etc.), the probability of risk occurrence, and corresponding preventive measures.

[0357] Suggestions for surgical step planning are also customized according to individual differences of patients. If there are variations in the patient's anatomical structure (such as vascular malformations, abnormal organ position, etc.), the system will take these special circumstances into account when generating surgical steps and adjust conventional surgical paths and operation methods. For example, in liver surgery, if the patient's liver vascular distribution is different from normal, the decision support system will advise the doctor to pay special attention to the identification and treatment of blood vessels during surgery to avoid misoperation and improve the safety and success rate of the operation.

[0358] 2. Real-time feedback and updates

[0359] During the operation, as new data (such as real-time physiological monitoring data and imaging data) continue to flow in, the decision-making support and control components update the response results in real time. For example, when the patient's blood pressure suddenly drops during surgery, the system immediately re-evaluates the surgical risk, considers the impact of the drop in blood pressure on the progress of the operation, such as whether it is necessary to suspend the operation, take measures to increase blood pressure, etc., and provide timely intraoperative advice to the doctor. At the same time, according to the actual progress of the operation, such as the completed surgical steps and new situations discovered (such as unexpected tissue adhesions, bleeding points, etc.), the surgical step planning suggestions and postoperative suggestions are dynamically adjusted. For example, if the tumor is found to be more invasive than expected during surgery, the system will update the postoperative follow-up plan and possible auxiliary treatment suggestions to ensure that the doctor can obtain the decision support information that best meets the actual situation of the operation in a timely manner.

[0360] To this end, the innovation of the present invention in assisting decision-making lies in the integration of multi-source data for analysis. By collecting all-round information about the patient, including medical records, examination results, real-time intraoperative data, etc., a more comprehensive and in-depth understanding of the patient's surgical condition can be obtained. Based on these rich data, machine learning models are used to mine the complex relationships behind the data, combined with expert experience and medical guidelines in the knowledge rule base, to achieve accurate analysis of decisions such as surgical risk assessment and surgical step planning. Compared with traditional decision-making methods that rely solely on doctor experience or a single source of data, this multi-source data-driven approach can provide more scientific and accurate decision-making recommendations, reduce surgical risks, and improve surgical quality.

[0361] 2. Intelligent decision-making based on the integration of machine learning and knowledge rules

[0362] The integration of machine learning models and knowledge rule bases is an important innovation. Machine learning models can automatically learn rules and patterns from large amounts of data, but there may be data bias or problems that are difficult to explain; the knowledge rule base provides clear medical knowledge and expert experience, which is reliable and explainable. The integration of the two enables the decision-making support system to follow medical expertise and standards while leveraging the powerful data processing capabilities of machine learning, complement and correct each other, and generate more reasonable and reliable decision results. This integrated intelligent decision-making mechanism is a relatively new application in the field of surgical decision-making support, which helps to improve doctors' trust and adoption of decision-making recommendations and promote the intelligent development of surgical decision-making.

[0363] 3. Personalized and real-time dynamic decision support

[0364] The ability to generate personalized responses and update real-time feedback is the innovative highlight of this technology. According to the individual differences of patients and the real-time progress of the operation, the system can provide customized decision suggestions, whether it is pre-operative planning or adjustments during surgery, which can closely meet the actual needs of the operation. Real-time dynamic updates of decision results can respond to emergencies and changes during surgery in a timely manner, and provide doctors with timely and effective decision support, which is difficult to achieve in traditional surgical decision-making systems. It greatly improves the flexibility and adaptability of the operation, and ensures the smooth progress of the operation and the safety of the patient.

[0365] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A humanoid robot surgical system, characterized in that: include: A robotic arm, a trunk, a mobile mechanism, a host, and a human-machine interface. The robotic arm is mounted on the trunk and is communicatively connected to the host. The trunk is mounted on the mobile mechanism. A host accommodating space is provided in the trunk so that the host can be mounted in the host accommodating space. The human-machine interface is mounted on the trunk and is communicatively connected to the host. A generative surgical model is deployed on the host. The surgical model performs the following steps to implement the surgical procedure: The three-dimensional physiological structure model is constructed by receiving the patient's basic medical history information, preoperative examination results and lesion imaging data through the human-machine interface; Performing feature analysis on the three-dimensional physiological structure model to mark the target surgical site and formulate a robotic arm path accordingly; Establishing a mapping relationship between the path of the robotic arm and the position of the patient to generate a motion trajectory sequence of the robotic arm; According to the motion trajectory sequence, the moving mechanism is driven to move toward the patient's position to reach a specified point, and the robotic arm is controlled to drive the surgical instrument to move so as to perform surgery on the target surgical site.

2. The humanoid robot surgical system according to claim 1, characterized in that: The humanoid robot surgical system further includes a positioning module and a visual module. The positioning module is used to locate the target surgical site during the operation, and the visual module is used to visually track the robotic arm and the target surgical site during the operation. When the surgical large model performs feature analysis on the three-dimensional physiological structure model to mark the target surgical site and formulate the robotic arm path accordingly, the following steps are performed: Establishing a spatial conversion relationship between the lesion image data and the positioning module and between the positioning module and the visual module; Performing feature analysis on the three-dimensional physiological structure model to mark a target surgical site; Based on the spatial conversion relationship, the robotic arm path is formulated for the target surgical site.

3. The humanoid robot surgical system according to claim 2, characterized in that: When the large surgical model performs feature analysis on the three-dimensional physiological structure model to mark the target surgical site and formulates the robotic arm path accordingly, the following steps are also performed: the designated point for surgery on the target surgical site, the depth of surgery, the control torque of the robotic arm, and the control strategy of the robotic arm driving the surgical instrument to reach the designated point and reach the depth during the operation.

4. The humanoid robotic surgical system according to claim 1, characterized in that: The humanoid robotic surgical system also includes: a wireless module for receiving the patient's real-time physiological monitoring data and real-time medical imaging data through a constructed wireless channel, so that the large surgical model analyzes the real-time physiological monitoring data and real-time medical imaging data, and generates a surgical suggestion plan based on the analysis results.

5. The humanoid robotic surgical system according to claim 1, characterized in that: There are at least two robotic arms, each of which is composed of multiple joints of more than 5 axes connected in series, and the end of each robotic arm forms a wrist, wherein the wrist of one robotic arm is installed with a main surgical instrument, and the wrist of at least one of the remaining robotic arms is installed with an auxiliary surgical instrument.

6. The humanoid robotic surgical system according to claim 1, characterized in that: The torso is a humanoid torso, and the humanoid torso has at least two degrees of freedom, so as to match the surgical operation during the operation to perform at least one of forward and backward bending, left and right side bending, and height self-adjustment.

7. The humanoid robotic surgical system according to claim 1, characterized in that: The moving mechanism is at least one of a humanoid leg-and-foot type, a wheel type or a crawler type moving mechanism.

8. The humanoid robotic surgical system according to claim 1, characterized in that: A multi-language translation model is also deployed on the host, so as to enable the large surgical model to interact with the doctor in multiple languages ​​through the human-computer interaction interface.

9. The humanoid robotic surgical system according to claim 1, characterized in that: The humanoid robot surgery system further includes: a holographic projection module, which is used to project the three-dimensional physiological structure model into a holographic image through spatial mapping.

10. The humanoid robotic surgical system according to claim 1, characterized in that: The host is also deployed with auxiliary decision-making and control components to respond to surgical query instructions captured through the human-computer interaction interface and return response results, and the response results include: at least one of surgical operation suggestions, surgical step planning suggestions, hand speed risk assessment, postoperative suggestions, and intraoperative suggestions.

Images