A robotic control system, method, and medium for automatic set-up of surgical instruments

Abstract

This invention relates to a robot control system, method, and medium for automated surgical instrument placement. The system includes: a surgical scheduling parsing and preference list management module for generating an instrument requirement list; an instrument and ward mapping management module for querying ward information based on the instrument requirement list to obtain basic data for the placement and retrieval paths; a visual recognition and grasping target planning module for acquiring images of the target ward area, obtaining target detection and instance segmentation results, and combining them with camera calibration information to obtain the grasping point pose; a force control and grasping execution control module for generating motion trajectories, adjusting force control during the grasping process, generating task space correction values, and fusing them with the motion trajectory to form a complete force control and grasping process; and a tray layout and placement process control module for combining the instrument target positions with the complete force control and grasping process to form a placement process. Compared with existing technologies, this invention has advantages such as improved placement efficiency.

Description

Technical Field

[0001] This invention relates to the fields of medical device management and industrial robot control, and in particular to a robot control system, method and medium for automatic surgical instrument matching. Background Technology

[0002] Currently, the surgical instrument setup process in hospitals relies heavily on manual operation. Scrub nurses must frequently shuttle between the central instrument storage area and various operating rooms, relying on paper lists or personal experience, to locate, verify, retrieve, and place instruments on operating tables. This model, against the backdrop of continuously increasing surgical volume and the need for more refined medical management, highlights the following pressing technical bottlenecks that require resolution: 1. Fragmented information flow makes it impossible to automatically generate standardized tasks: There is a lack of effective data linkage and structured integration between the surgical scheduling system, the instrument inventory management system, and the surgical preference list (SPL) that includes the surgeon's personalized requirements. This "information silo" status prevents the system from automatically associating and parsing the required instrument kits and individual item lists based on the scheduling information of a specific surgery (such as surgical procedure and attending surgeon), thus failing to generate structured coordination instructions that can be understood and executed by automated equipment.

[0003] 2. Purely manual operation is inefficient and carries a high risk of error: Faced with thousands of diverse instruments and numerous kits designed for different surgical procedures, nurses rely entirely on visual identification and manual operation, resulting in a high workload and long processing time. Under this high-pressure environment, it is easy for instruments to be missed, over-equipped, or of incorrect specifications, directly affecting surgical efficiency and safety, and also hindering further improvements in operating room turnover.

[0004] 3. Existing automated equipment "can only move things, not coordinate with other equipment": The automated warehousing or handling equipment introduced by some hospitals is mostly limited to replacing manual transport of pre-packaged instrument trays or boxes. However, this does not solve the most critical and complex part of the instrument preparation process—accurately selecting each individual instrument from the vast array of scattered instruments based on the final surgical list, and precisely placing them on a sterile tray according to usage habits and optimal space layout. Existing equipment "can only move instruments, not prepare them," indicating insufficient intelligence.

[0005] 4. Lack of self-learning ability: Existing systems rarely optimize the matching process and pallet layout based on historical data, and cannot automatically improve efficiency and matching quality as usage accumulates.

[0006] Therefore, there is an urgent need for a solution that can fundamentally change this model. Its core lies in connecting the entire chain from surgical scheduling and SPL to the final physical placement of instruments, and realizing the intelligent generation and closed-loop execution of surgical information into precise, personalized, and automatically executable coordination tasks. Summary of the Invention

[0007] The purpose of this invention is to provide a robot control system, method, and medium for automated surgical instrument matching that enables intelligent instrument matching.

[0008] The objective of this invention can be achieved through the following technical solutions: A robot control system for automated surgical instrument delivery includes: Surgical scheduling analysis and preference list management module: used to acquire surgical scheduling data, analyze it, obtain surgical preference lists for each surgical procedure and surgeon, and then generate a list of instrument requirements corresponding to the target surgery through matching; Instrument and Ward Mapping Management Module: Connected to the Surgical Scheduling and Preference List Management Module, it is used to query instrument-ward mapping data based on the instrument demand list to obtain basic data of the pick-up and put-down path for the allocation task; Visual recognition and grasping target planning module: used to acquire images of the target warehouse area, perform target detection and instance segmentation, obtain the target detection and instance segmentation results, and combine them with camera calibration information to obtain the grasping point pose of the robotic arm; Force control and grasping execution control module: connected to the vision recognition and grasping target planning module, used to generate the motion trajectory of the robotic arm based on the basic data of the pick-and-place path and the posture of the grasping point, and to adjust the force control during the grasping process through the six-dimensional force / torque measurement signal at the end of the robotic arm, generate the task space correction amount, and fuse the task space correction amount with the motion trajectory, and combine it with the safety strategy to form a complete force control and grasping process; The tray layout and matching process control module is connected to the surgical scheduling analysis and preference list management module and the force control and grasping execution control module. It is used to determine the target position of each instrument in the instrument requirement list according to the tray layout rules corresponding to the surgical procedure and the surgeon, and combine it with the complete force control and grasping process to form a complete matching process and complete the automatic matching process.

[0009] Furthermore, the device-ward mapping data includes: The control parameters of the device shall include at least the unique identifier of the device, the gripping point, the clamping area, the prohibited gripping area, the vulnerability level, the upper limit of the clamping force, the upper limit of the movement speed and the upper limit of the acceleration corresponding to the vulnerability level; Position control parameters should include at least position identifier, position three-dimensional coordinates, approach direction, and pick-up / placement posture constraints. Task status parameters: mapping relationship between instruments and warehouse identifiers, availability status, triggering conditions for alternative instruments when instruments are unavailable, and priority of alternative instruments; The gripping point, clamping area, and no-grip area are used to determine the gripping pose offset of the end effector. The three-dimensional coordinates of the storage compartment, the approach direction, and the pick-and-place posture constraints are used to generate the basic data for the pick-and-place path of the robotic arm. The clamping force limit, motion speed limit, and acceleration limit are used to generate force control thresholds and trajectory constraint parameters; The alternative equipment triggering conditions and alternative equipment priority when the required equipment is unavailable are used to redetermine the required machinery and corresponding storage location when the required equipment does not meet the preset conditions.

[0010] Furthermore, in the visual recognition and grasping target planning module, the step of obtaining the grasping point pose of the robotic arm includes: The target detection and instance segmentation results are set as follows: ,in For the first Category labels for each target For the first The confidence score of each target. For the first The bounding box of each target. For the first Instance segmentation mask for each target. The number of targets identified; For each instance segmentation mask in the target detection and instance segmentation results Get its pixel set Calculate instance segmentation mask The representative pixel is used as the coordinate of the target pixel. , is represented as: , In the formula, These are the pixel coordinates of the imaging plane; At the target pixel coordinates Calculate depth value , is represented as: , In the formula, For depth images; Based on the target pixel coordinates and depth value By using camera calibration information to back-project the target pixel coordinates onto the camera coordinate system, the three-dimensional spatial position of the target in the camera coordinate system can be obtained. , is represented as: , In the formula, , For camera focal length, , Principal point coordinates; For each target in the target detection and instance segmentation results, the pose of each target in the camera coordinate system is obtained by the 6D pose estimation method. Based on the pose of each target in the camera coordinate system, coordinate system transformation is performed in conjunction with camera calibration information to obtain the pose of the robotic arm's grasping point.

[0011] Furthermore, the pose of each target in the camera coordinate system is obtained by at least one of the following 6D pose estimation methods: 1) Keypoint regression combined with PnP method: From the target pixel coordinates Obtain the target key points ; The target key points and its corresponding target 3D model points Perform matching to obtain matching pairs; For the matching pair, the rotation matrix satisfying the projection model is solved using the PnP algorithm. Translation vector The projection model is as follows: , , In the formula, As a scale factor, For the camera intrinsic parameter matrix, Let be the target attitude rotation matrix. The target position vector; Based on the rotation matrix Translation vector And by minimizing the reprojection error, the pose of each target in the camera coordinate system is obtained from the matching pairs. , is represented as: ; 2) Point cloud registration method: Obtain the point cloud of each target, perform ICP registration with the CAD / template point cloud, and obtain the rotation matrix. Translation vector ; Based on the rotation matrix Translation vector The pose of each target in the camera coordinate system is derived. 3) Planar constraint attitude method: If the target is a planar target, fit a plane to the point cloud within the instance segmentation mask based on its instance segmentation. : , In the formula, It is a plane normal vector. To segment 3D points in the point cloud within the mask for an instance, For planar offset; The local part of the planar target Axis and plane normal vector Align to obtain the alignment result; Principal component analysis is performed on the instance segmentation mask of the planar target to obtain the normal vector around the plane. The rotation angle is determined, and the rotation matrix is ​​determined in conjunction with the alignment result. ; Determine the plane The position of a reference point on the map is used as the translation vector. ; The rotation matrix Translation vector By combining the data, the pose of the planar target in the camera coordinate system can be obtained.

[0012] Furthermore, the step of obtaining the gripping point pose of the robotic arm includes: The pose of each target in the camera coordinate system is transformed, combined with camera calibration information, from the camera coordinate system to the robot arm base coordinate system / end-effector coordinate system, resulting in the pose in the robot arm base coordinate system / end-effector coordinate system. For scenarios with external / fixed cameras, the pose of the robotic arm in the base coordinate system is represented as follows: , For the scenario where the eye is on the hand, the pose in the end effector coordinate system is represented as follows: , In the formula, The pose of the robotic arm in the base coordinate system / end-effector coordinate system. This is a transformation from the camera coordinate system to the robot arm base coordinate system. The pose in the camera coordinate system. For real-time pose, This is a transformation from the camera coordinate system to the end effector coordinate system; Based on the pose in the robot arm base coordinate system / end-effector coordinate system, the grasping pose offset is superimposed to obtain the corresponding grasping point pose of the robot arm. , is represented as: , , In the formula, To capture pose offset, For orientation offset, To capture the displacement of the point.

[0013] Furthermore, the visual recognition and grasping target planning module also includes generating a pre-grasping pose along the approach direction based on the grasping point pose of the robotic arm. The pre-grasp pose Represented as: , In the formula, The gripping point pose of the robotic arm. To safely increase the distance, It is a pure translation operator along the approach direction.

[0014] Furthermore, in the force control and gripping execution control module, the force control adjustment process includes at least the following: Six-dimensional force / torque measurement signal acquisition and compensation: Acquire the six-dimensional force / torque measurement signal output by the six-dimensional force / torque sensor at the end of the robotic arm. Zero-point calibration and filtering are performed, and gravity and tool bias compensation are applied to obtain the estimated actual external force of the wrench. ,in It is the transpose of the three-dimensional force vector. The transpose of the three-dimensional torque vector, the actual external force wrench Represented as: , In the formula, For zero drift / tool ​​inherent bias, This is a gravity-compensated wrench calculated based on the tool's mass, center of gravity, and current orientation. Joint angle; The actual external force wrench is represented by the adjoint matrix of the homogeneous transformation. Transforming to the robot arm base coordinate system or contact coordinate system yields the forces and moments in the robot arm base coordinate system or contact coordinate system, where the transformation expression is: , , In the formula, , Forces and torques in the robot arm's base coordinate system or contact coordinate system constitute the actual external force wrench in the robot arm's base coordinate system or contact coordinate system. Let be a rotation matrix. , Forces and torques in the end effector coordinate system or sensor coordinate system. This is the position vector of the origin at the end point in the base coordinate system; Based on the actual external force wrench The calculation expression for force error is as follows: or , In the formula, , For force error, To make contact with the wrench, For normal expectation force, This is the actual normal force; Based on the force error, a force control strategy is used to generate a task space correction. This force control strategy includes at least one of admittance control, impedance control, and position-force hybrid control. For admittance control: The force error or Substituting into the admittance model, the admittance model is expressed as: , In the formula, , , For admittance parameters, This represents the position of the robotic arm's end effector in the contact coordinate system or its scalar displacement along the contact normal. For reference trajectory, The force error or ; Solving the admittance model yields... Relative to reference trajectory The deviation is used as a task space correction amount; For impedance control and position-force hybrid control: Based on the force error Both employ velocity-based force control formulas, performing impedance control and position-force hybrid control respectively, to obtain corresponding task space correction values. Joint velocity commands are then derived based on these task space correction values. The velocity-based force control formula is as follows: , In the formula, For controlling the speed in the contact direction, For reference speed, For force feedback gain; The joint speed command Represented as: , In the formula, This is a false rebellion of Jacobi.

[0015] Furthermore, in the force control and gripping execution control module, the steps for forming a complete force control and gripping process include: The task space correction amount is fused with the motion trajectory. In the non-contact phase, position control is the main method, and the task space correction amount obtained by the admittance control is fused with the motion trajectory to perform non-contact matching task execution control. In the contact phase, the task space correction amount obtained by the position-force hybrid control is fused with the motion trajectory to perform contact matching task execution control. Simultaneously, safety strategies are used to monitor abnormal forces and collision trends. When any of the following conditions are met, deceleration, stopping, or retraction Δz_back along the approach direction is executed, and abnormal events are recorded for tracing, ultimately obtaining the complete force control and grasping process. The conditions include at least the following: , , , In the formula, For strength, The threshold value for external force amplitude. For the rate of change of force, For the force change rate threshold, For torque, This is the torque threshold.

[0016] The present invention also provides a control method for a robot control system for automatic instrument loading according to the above description, comprising: Obtain surgical scheduling data, analyze it, and obtain a list of surgical preferences for each surgical procedure and surgeon. Then, generate a list of instrument requirements for the target surgery through matching. Based on the equipment requirement list, query the equipment-warehouse mapping data to obtain the basic data of the pick-up and put-down path for the allocation task; The target warehouse area image is acquired, target detection and instance segmentation are performed, the target detection and instance segmentation results are obtained, and the pose of the robotic arm's grasping point is obtained by combining the camera calibration information. The motion trajectory of the robotic arm is generated based on the pose of the grasping point, and the force control adjustment during the grasping process is performed through the six-dimensional force / torque measurement signal at the end of the robotic arm. The task space correction amount is generated, and the task space correction amount is fused with the motion trajectory. Combined with safety strategies, a complete force control and grasping process is formed. Based on the tray layout rules corresponding to the surgical procedure and the surgeon, the target position of each instrument in the instrument requirement list is determined, and combined with the complete force control and grasping process, a complete setup process is formed to complete the automatic setup process.

[0017] The present invention also provides a storage medium having a program stored thereon, wherein the program, when executed, implements the control method as described above.

[0018] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention effectively integrates surgical scheduling data and surgical preference list, matches the two to generate instrument demand list, takes into account the individual preferences of each surgeon, breaks the "information silo", and performs target detection and instance segmentation through target ward area image, obtains grasping point pose by combining camera calibration information conversion, and adjusts the grasping process by force control adjustment during the grasping process, realizing the intelligent matching task from surgical information to precise, personalized and automatically executed.

[0019] (2) By identifying the three-dimensional spatial position and posture of the target and combining the camera calibration information, the present invention transforms the identification results into the grasping point posture in the coordinate system of the robotic arm. This not only enables the robotic arm to move directly to the calculated optimal grasping point through precise calibration and coordinate transformation, and to grasp the device stably in the correct posture, greatly reducing grasping failure, slippage or collision caused by inaccurate positioning, but also eliminates the need for the device to be fixed in a certain preset position and orientation. By identifying its posture, the robotic arm can adaptively generate the corresponding grasping scheme, greatly improving the system's adaptability to real and complex environments. Thus, the system of the present invention is no longer a "blind mover" and solves the technical pain point of "only moving, not matching" in the prior art.

[0020] (3) The present invention also considers the optimal grasping measurement of planar targets (packages / pallets, etc.). By estimating their posture through the planar constraint posture method, the robotic arm can adaptively adjust the grasping posture, which greatly avoids the risks of slippage, uneven grasping force or collision.

[0021] (4) The force control adjustment introduced in this invention is not just pure position control. This force control adjustment allows the robotic arm to dynamically adjust its movements based on real-time feedback of force / torque information when it comes into contact with the equipment or environment, fundamentally avoiding damage caused by excessive force or collision. In addition, considering that in a real assembly environment, the equipment may be partially obstructed, slightly stuck, or there may be visual recognition errors in the placement posture, the force control gives the robotic arm "tactile sense", enabling it to detect and trigger adjustments in a timely manner, realizing the transformation of the robotic arm from a "mechanical handling hand" to a "dexterous operating hand", solving the most intractable physical interaction safety problem in automated assembly, and greatly expanding the range of fine operations that the system can perform.

[0022] (5) The present invention also introduces a self-learning mechanism, which enables the system to have self-learning and continuous optimization capabilities, forming a closed-loop design, thereby automatically improving efficiency and matching quality as it is used.

[0023] (6) The system of the present invention adopts a modular architecture, which is easy to promote in multiple campuses. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the control method of the present invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0026] This embodiment provides a robot control system for automatic surgical instrument allocation. The system includes a surgical scheduling analysis and preference list management module, an instrument and warehouse mapping management module, a visual recognition and grasping target planning module, a force control and grasping execution control module, and a tray layout and allocation process control module. The surgical scheduling analysis and preference list management module and the instrument and warehouse mapping management module are connected, and the visual recognition and grasping target planning module, the force control and grasping execution control module, and the tray layout and allocation process control module are connected sequentially.

[0027] The surgical scheduling analysis and preference list management module relies on a control server and data interface to implement its functions, which are as follows: First, it uses a data interface to retrieve surgical scheduling data from the hospital's surgical management system. Then, it uses a control server to perform field parsing, rule matching, and other functions to generate an equipment requirement list. Specifically, the control server has the following functions: Used to analyze information such as surgical procedure, surgeon, time, and operating room; It is used to manage the Surgical Preference List (SPL) corresponding to each surgical procedure and surgeon, and to match scheduling information with the SPL to generate the instrument requirement list corresponding to the target surgery.

[0028] The instrument and ward mapping management module also relies on the aforementioned control server. Based on the instrument requirement list, it queries the ward information corresponding to the target surgery from the instrument-ward mapping data, generating basic data for the robotic arm's pick-and-place path control. The functions of the instrument and ward mapping management module are as follows: This data is used to maintain surgical instrument master data, bin / location data in automated storage and retrieval systems (AS / RS) or shelving, and instrument-bin / location mapping data. Surgical instrument master data includes at least a unique instrument identifier (instrument ID), gripping point, clamping area, prohibited gripping area, vulnerability level, and corresponding upper limits for clamping force, movement speed, and acceleration. It may also include instrument category, physical dimensions and weight, material or surface properties, sterility / sterilization properties, and maintenance cycle. Bin / location information includes at least a bin / location identifier (bin / location ID), bin / location 3D coordinates, approach direction, and pick-and-place posture constraints. It may also include bin / location coordinate system definition, maximum load capacity, and maximum capacity volume. Instrument-bin / location mapping data supports one-to-many or many-to-one relationships and includes instrument control parameters (i.e., surgical instrument master data), bin / location control parameters (i.e., bin / location information), and task status parameters. Task status parameters include at least the mapping relationship between instruments and bin / location identifiers, available quantity, reserved quantity, availability status, and the trigger conditions and priority of alternative instruments when an instrument is unavailable. The mapping relationship between instruments and warehouse identifiers is achieved through the mapping between instrument ID and warehouse ID.

[0029] This is used to query the warehouse information corresponding to the equipment-warehouse mapping data based on the equipment demand list, and form the "basic data of pick-up and drop-off path" for the allocation task.

[0030] The process involves generating basic data for the robotic arm's pick-and-place path based on the three-dimensional coordinates of the storage location, the approach direction, and the pick-and-place posture constraints. Furthermore, it involves determining the end effector's gripping posture offset based on the gripping point, clamping area, and no-grip area, generating force control thresholds and trajectory constraint parameters based on the upper limit of clamping force, the upper limit of motion speed, and the upper limit of acceleration, and re-determining the required machinery and corresponding storage location when the required machinery does not meet the preset conditions, based on the triggering conditions and priority of the alternative machinery when the machinery is unavailable.

[0031] Furthermore, the aforementioned instrument dimensions, weight, material, or surface properties are used to select the end effector type, gripping method, and initial gripping force value; the vulnerability level is used to generate the corresponding upper limit of gripping force, upper limit of movement speed, and upper limit of acceleration; during the grasping process, when the six-dimensional force / torque sensor detects that the actual gripping force or contact force is close to the upper limit of gripping force (i.e., when an abnormal force is detected), the robot controller reduces the approach speed, decreases the gripping increment, or performs a retraction action; the reserved quantity and availability status are used for task filtering: when the instrument is in an unavailable state or has been reserved by other surgical tasks, the control server does not generate a grasping instruction for that instrument; when alternative instrument triggering conditions and alternative instrument priorities are preset, the control server reselects the instrument and corresponding compartment according to the alternative instrument priority and regenerates the basic data for the pick-and-place path.

[0032] This module also allows for version control and traceability. The content of version control and traceability includes: master data version number, effective time, expiration time, modification record (operator, time, summary of changes), data source identifier (HIS / CSSD / asset system / manual maintenance), and association ID with surgical scheduling tasks (used for "traceable matching tasks").

[0033] The visual recognition and target acquisition planning module relies on the visual acquisition device and the aforementioned control server. Its functions are as follows: This device is used to drive a camera to acquire images of a target storage area using a vision acquisition device; wherein the camera includes at least one of an RGB camera, a depth camera, or an RGB-D camera, and the acquired images of the target storage area include: color images. With depth images (or dot cloud) Subsequently, the control server uses target detection / segmentation algorithms to identify the spatial position and orientation of the device or package to be grasped; and combines this with camera calibration information to convert the identification results into the grasping point pose in the robotic arm coordinate system.

[0034] For the force control and gripping execution control module, this module generates the motion trajectory for the robotic arm based on the robot controller under trajectory constraint parameters, and uses an end effector force / torque sensor for force control adjustment, wherein the force control adjustment process should not exceed the force control threshold. Specifically, the functions of the force control and gripping execution control module are as follows: This is used to generate the motion trajectory of the robotic arm based on the basic data of the pick-and-place path and the pose of the gripping point. The motion trajectory of the robotic arm is based on the optimal distance selection mode between two points of the visual training model (based on human simulated action), and the motion trajectory coordinates are generated segment by segment to complete the optimal trajectory selection planning. Force control adjustment during the grasping process is achieved using an end effector force / torque sensor; When abnormal forces or collision trends are detected, safety strategies such as deceleration, stopping, or retraction are implemented to achieve complete force control and gripping process.

[0035] For the pallet layout and loading process control module, this module implements the following functions based on the aforementioned control server and robot controller: This is used to control the server to determine the target position of each instrument in the instrument requirement list on the tray according to the tray layout rules corresponding to the surgical procedure and the surgeon. The robot controller programs the machine grasping action and the tray placement action into a complete setup process, that is, it uses the above complete force control and grasping process to form a complete setup process; Check the completion rate of equipment matching against the equipment requirement list, generate matching results, and complete the matching task.

[0036] The system may also include a user interface for instrument nurses to view the status of tray placement tasks, perform manual verification, mark the causes of abnormalities, and adjust SPL or tray layout rules.

[0037] In conjunction with the above control system, such as Figure 1 As shown, the control method corresponding to this control system includes the following steps: S1. Obtain surgical scheduling data, analyze it, and obtain a list of surgical preferences for each surgical procedure and surgeon. Then, generate a list of instrument requirements corresponding to the target surgery through matching.

[0038] This step utilizes the surgical scheduling parsing and preference list management module. First, it obtains surgical scheduling data from the hospital's surgical management system through a data interface. Then, it uses the control server to parse out various information from the data and obtain the surgical preference lists for each surgical procedure and surgeon. Finally, it matches the two to obtain the instrument requirement list corresponding to each target surgery.

[0039] S2. Query the corresponding warehouse information based on the equipment demand list to obtain the basic data of the pick-up and drop-off path for the equipment allocation task.

[0040] This step utilizes the instrument and ward mapping management module. Based on the instrument requirement list corresponding to the target surgery and the data maintained by this module, the control server queries the wards to form the "basic data of pick-up and drop-off path" (continuous three-dimensional coordinate data) for the allocation task.

[0041] S3. Acquire the image of the target warehouse area, perform target detection and instance segmentation, obtain the target detection and instance segmentation results, and combine them with camera calibration information to obtain the gripping point pose of the robotic arm.

[0042] This step is executed using a visual recognition and target planning module, and specifically includes the following: 1. Image acquisition and preprocessing In this module, the vision acquisition device drives the camera to acquire images of the target warehouse area. The acquired raw data includes: color images. With depth images (or dot cloud) The control server preprocesses the collected data, including at least: distortion correction, noise filtering, depth hole filling, illumination normalization, and ROI pruning (based on the prior boundaries of the warehouse area).

[0043] 2. Target detection / segmentation and instrument / packet instance recognition The system's control server uses object detection / instance segmentation algorithms to identify instruments or packages in the target ward area image, outputting object detection and instance segmentation results, including the category, confidence score, and pixel-level contour of each object. The output can be represented as follows: ,in For the first Category label for each target (device type / package type). For the first The confidence score of each target. For the first The bounding box of each target. For the first An instance segmentation mask for each target (as a binary image containing pixel-level contours). The number of targets identified.

[0044] To reduce false captures, this module can introduce multi-source consistency verification: when the target belongs to the "package / pallet" category, further check the mask area, aspect ratio, and expected shape template matching degree; when the target belongs to the "disassembled equipment" category, check whether its key parts are visible (e.g., the clamping area is not obstructed).

[0045] 3. Three-dimensional position calculation (from 2D pixels to camera coordinate system) Segmentation mask for each instance Obtain its pixel set from the depth map. Calculate the representative pixel point of this instance (e.g., mask centroid, capture candidate key points): , In the formula, For the pixel coordinates of the imaging plane, Target pixel coordinates (unit: pixel); And obtain the depth value at that pixel: , In the formula, The depth value is in m or mm (or the median depth within the instance segmentation mask for noise reduction).

[0046] By back-projecting the pixel coordinates onto the camera coordinate system using camera intrinsic parameters, the target's 3D point in the camera coordinate system is obtained, which serves as its 3D spatial position. : , Three-dimensional spatial position Its unit is consistent with the depth value. , The focal length of the camera (in pixels). , The coordinates of the main point (in pixels).

[0047] Alternatively, for planar targets such as packages / pallets, a plane can be fitted based on the point cloud within the mask. : , In the formula, It is a plane normal vector. To segment 3D points in the point cloud within the mask for an instance, For planar offset; The center point of the plane is used as the reference position for grasping / placing, and the normal vector is used as the constraint for attitude estimation.

[0048] 4. Orientation / Pose estimation This module performs 6D pose estimation on the target and outputs the target's pose in the camera coordinate system. : , In the formula, Represents the target attitude rotation matrix. Let the target position vector be... It is a rotation matrix.

[0049] In this step, the 6D pose estimation method can be implemented using one or more of the following methods: 1) Keypoint Regression + PnP (Perspective-n-Point) Algorithm: From target pixel coordinates Obtain the target key points ; Key points of the target and its corresponding target 3D model points Perform matching to obtain matching pairs; For a matching pair, the rotation matrix satisfying the projection model is solved using the PnP algorithm. Translation vector The core projection model is: , , In the formula, s is the scale factor and K is the camera intrinsic parameter matrix; Based on the rotation matrix Translation vector And by minimizing the reprojection error, the pose of each of the above targets in the camera coordinate system is obtained from the matching pairs. .

[0050] 2) Point cloud registration: Obtain the point cloud of each target, perform ICP registration with the CAD / template point cloud, and obtain the rotation matrix. Translation vector Then based on the rotation matrix Translation vector The pose of each target in the camera coordinate system is derived.

[0051] 3) Planar constrained posture (package / pallet): This method considers the case where the target is a planar target (package / pallet), based on the fitted plane described above. To localize the planar target Axis and plane normal vector Alignment yields the alignment result, which determines the "up / down" orientation of the target in space; Then, principal component analysis (PCA) is performed on all pixel coordinates within the instance segmentation mask of the planar target to obtain the normal vector around the plane. The rotation angle is determined, and the rotation matrix is ​​determined by combining the alignment results. ; Through the plane The position of a reference point on the map is used as the translation vector. Combine it with the rotation matrix By combining the data, the pose of the planar target in the camera coordinate system can be obtained.

[0052] 5. Camera calibration and coordinate transformation: from recognition results to the pose of the robotic arm's grasping point. This section describes the camera calibration and coordinate system transformation process, used to convert the target pose obtained from visual recognition into a grasping point pose that the robotic arm can execute. The system pre-compiles camera calibration and hand-eye calibration, obtaining at least the camera intrinsic parameters. The distortion parameters, and one or more of the following homogeneous transformation matrices: Transformation from camera coordinate system to robot arm base coordinate system (external / fixed camera scene).

[0053] Transformation from camera coordinate system to end effector coordinate system (hand-eye calibration results for scenes where the eye is on the hand).

[0054] Real-time pose of the end effector to the robot arm base coordinate system (obtained by reading back from the robot controller or by forward kinematics calculation).

[0055] (1) Coordinate system transformation (transforming the target pose from the camera coordinate system to the robot arm base coordinate system) In scenarios with external / fixed cameras: , In the scenario where the eye is on the hand: , In the formula, The pose of the robotic arm in the base coordinate system / end-effector coordinate system. The pose in the camera coordinate system; (2) Generate the pose of the grab point (by superimposing the target pose with the grab template offset) , in For the grab pose offset (grab template offset) defined in the target coordinate system, it must be at least determined by the displacement of the grab point. Orientation offset The composition can be represented as: .

[0056] In the formula, To capture pose offset, it is provided by the instrument master data / capture strategy library.

[0057] (3) Pre-grab pose (optional, used for collision avoidance and trajectory smoothing) To reduce collision risk, a pre-grabbing pose can be generated along the approach direction based on the grasping pose. : , in, To safely increase the distance, It is a pure translation operator along the approach direction.

[0058] S4. Generate the motion trajectory of the robotic arm based on the pose of the gripping point, and adjust the force control during the gripping process through the six-dimensional force / torque measurement signal at the end of the robotic arm to generate the task space correction amount, and fuse the task space correction amount with the motion trajectory, and combine it with safety strategies to form a complete force control and gripping process.

[0059] This step is executed using the force control and grasping execution control module. Based on the grasping point pose, the robot controller generates the robot arm's motion trajectory and uses a six-dimensional force / torque sensor at the end effector to achieve force control adjustment during the grasping process. When abnormal forces or collision tendencies are detected, safety strategies such as deceleration, stopping, or retraction are implemented. In this embodiment, the force control adjustment process includes at least: force / torque measurement signal acquisition and compensation, contact coordinate system construction, force error calculation, generation of task space correction parameters using the force control law, fusion with the position trajectory, and transmission to the robot controller.

[0060] 1. Force / torque measurement signal acquisition and compensation The force / torque sensor at the end effector of the robotic arm outputs a six-dimensional force / torque measurement signal in the sensor coordinate system, which serves as the raw measurement value. , It is the transpose of the three-dimensional force vector. This is the transpose of the three-dimensional torque vector. The system performs zero-point calibration and filtering (e.g., low-pass filtering or moving average) on the original measurements, and compensates for gravity and tool bias to obtain the estimated actual external force of the wrench. : , In the formula, For zero drift / tool ​​inherent bias, This is a gravity-compensated wrench calculated based on the tool's mass, center of gravity, and current orientation. This refers to the joint angle.

[0061] 2. Wrench coordinate transformation and contact coordinate system To facilitate force control in the gripping / contact direction, the system transforms the external force wrench from the sensor / end-effector coordinate system to the robot arm base coordinate system or contact coordinate system. In a simplified scenario considering only rotations, it can be done according to the rotation matrix. Perform force / torque transformation; in general scenarios, the wrench transformation is achieved using the adjoint matrix of homogeneous transformation.

[0062] , , In the formula, , Forces and torques in the robot arm's base coordinate system or contact coordinate system constitute the actual external force wrench in the robot arm's base coordinate system or contact coordinate system. , Forces and torques in the end effector coordinate system or sensor coordinate system. This is the position vector of the end effector origin in the robot arm base coordinate system. (Contact coordinate system) The z-axis can be set as the contact normal (estimated by the visual plane normal or the force direction at the moment of contact), thereby projecting the wrench onto the contact normal and tangential directions for normal force control and tangential fine-tuning, respectively.

[0063] 3. Force error calculation and force control law (core of force control adjustment) Let the desired contact wrench (or normal desired force) be... (or The actual external force wrench is (or Based on coordinate transformation, the system calculates the force error. (or The system generates task space corrections based on the force control strategy. The force control strategy includes at least one of admittance control, impedance control, or position-force hybrid control.

[0064] (1) Admittance control (displacement / velocity correction driven by external force error): Force error or Substituting into the admittance model, the admittance model is expressed as: , In the formula, , , For admittance parameters, This represents the position of the robotic arm's end effector in the contact coordinate system or its scalar displacement along the contact normal. For reference trajectory, The force error or ; Solving the admittance model yields... Relative to reference trajectory The deviation is used as a task space correction.

[0065] (2) Impedance control and position-force hybrid control: Both of these control methods employ velocity-based force control (commonly used for constant normal force). This is based on force error. Both employ velocity-based force control formulas, performing impedance control and position-force hybrid control respectively, to obtain corresponding task space correction values. Joint velocity commands are then derived based on these task space correction values. The velocity-based force control formula is as follows: , In the formula, For controlling the speed in the contact direction, For reference speed, For force feedback gain; Mapping task-space correction velocity / displacement to joint-space commands can be achieved using a Jacobian matrix. The pseudo-inverse or transpose method, for example: in q̇ is the Jacobian pseudo-inverse, and q̇ is the joint velocity command.

[0066] 4. Location trajectory fusion and security strategies The system integrates the task space correction value generated by force control with the original position trajectory: in the non-contact phase, position control is the main method, and the task space correction value obtained by admittance control is integrated with the motion trajectory to perform the matching task execution control during non-contact; in the contact phase, force control is used for the contact normal, and position / impedance control is used for the tangential and attitude maintenance, thus forming a position-force hybrid control, that is, the task space correction value obtained by the position-force hybrid control is integrated with the motion trajectory to perform the matching task execution control during contact.

[0067] Simultaneously monitor abnormal forces and collision trends, including at least: , ,as well as When the threshold is triggered, deceleration, shutdown, or retreat along the approach direction (Δz_back) is executed, and the abnormal event is recorded for traceability. For strength, The threshold value for external force amplitude. For the rate of change of force, For the force change rate threshold, For torque, This is the torque threshold.

[0068] S5. Based on the procedure and the tray layout rules corresponding to the surgeon, determine the target position of each instrument in the instrument requirement list, and combine it with the complete force control and grasping process to form a complete setup process and complete the automatic setup process.

[0069] This step is executed using the tray layout and setup process control module. First, based on the tray layout rules corresponding to each surgical procedure and surgeon, the control server determines the target location of each instrument in the instrument requirement list. Then, the robot controller programs the instrument grasping and tray placement actions into a complete setup process, which the robot then follows to perform the setup. Furthermore, this step also checks the setup completion against the instrument requirement list and generates the setup result.

[0070] The system also includes an anomaly labeling module, which allows for human-computer interaction through a user interface. Instrument nurses can view the status of tray preparation tasks, perform manual verification, label anomalies, and adjust SPL or tray layout rules.

[0071] Preferably, the control server in this embodiment can be an industrial computer, an edge computing server, or a computing device with image processing capabilities; the robot controller is communicatively connected to the robotic arm; the end effector can be a gripper, a suction cup, a combination of gripper and suction cup, or a customized gripper suitable for grasping surgical instruments; the vision acquisition device is set above the target storage area, on the side of the storage unit, or installed at the end of the robotic arm; the six-dimensional force / torque sensor is set between the end of the robotic arm and the end effector, or integrated inside the end effector.

[0072] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0073] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0074] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0077] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0078] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A robot control system for automated surgical instrument delivery, characterized in that, include: Surgical scheduling analysis and preference list management module: used to acquire surgical scheduling data, analyze it, obtain surgical preference lists for each surgical procedure and surgeon, and then generate a list of instrument requirements corresponding to the target surgery through matching; Instrument and Ward Mapping Management Module: Connected to the Surgical Scheduling and Preference List Management Module, it is used to query instrument-ward mapping data based on the instrument demand list to obtain basic data of the pick-up and put-down path for the allocation task; Visual recognition and grasping target planning module: used to acquire images of the target warehouse area, perform target detection and instance segmentation, obtain the target detection and instance segmentation results, and combine them with camera calibration information to obtain the grasping point pose of the robotic arm; Force control and grasping execution control module: connected to the vision recognition and grasping target planning module, used to generate the motion trajectory of the robotic arm based on the basic data of the pick-and-place path and the posture of the grasping point, and to adjust the force control during the grasping process through the six-dimensional force / torque measurement signal at the end of the robotic arm, generate the task space correction amount, and fuse the task space correction amount with the motion trajectory, and combine it with the safety strategy to form a complete force control and grasping process; The tray layout and matching process control module is connected to the surgical scheduling analysis and preference list management module and the force control and grasping execution control module. It is used to determine the target position of each instrument in the instrument requirement list according to the tray layout rules corresponding to the surgical procedure and the surgeon, and combine it with the complete force control and grasping process to form a complete matching process and complete the automatic matching process.

2. The robot control system for automatic surgical instrument matching according to claim 1, characterized in that, The instrument-position mapping data includes: The control parameters of the device shall include at least the unique identifier of the device, the gripping point, the clamping area, the prohibited gripping area, the vulnerability level, the upper limit of the clamping force, the upper limit of the movement speed and the upper limit of the acceleration corresponding to the vulnerability level; Position control parameters should include at least position identifier, position three-dimensional coordinates, approach direction, and pick-up / placement posture constraints. Task status parameters: mapping relationship between instruments and warehouse identifiers, availability status, triggering conditions for alternative instruments when instruments are unavailable, and priority of alternative instruments; The gripping point, clamping area, and no-grip area are used to determine the gripping posture offset of the end effector; The three-dimensional coordinates of the storage compartment, the approach direction, and the pick-and-place posture constraints are used to generate the basic data for the pick-and-place path of the robotic arm. The clamping force limit, motion speed limit, and acceleration limit are used to generate force control thresholds and trajectory constraint parameters; The alternative equipment triggering conditions and alternative equipment priority when the required equipment is unavailable are used to redetermine the required machinery and corresponding storage location when the required equipment does not meet the preset conditions.

3. A robot control system for automatic surgical instrument matching according to claim 1, characterized in that, In the visual recognition and grasping target planning module, the step of obtaining the grasping point pose of the robotic arm includes: The target detection and instance segmentation results are set as follows: ,in For the first Category labels for each target For the first The confidence score of each target. For the first The bounding box of each target. For the first Instance segmentation mask for each target. The number of targets identified; For each instance segmentation mask in the target detection and instance segmentation results Get its pixel set Calculate instance segmentation mask The representative pixel is used as the coordinate of the target pixel. , is represented as: ， In the formula, These are the pixel coordinates of the imaging plane; At the target pixel coordinates Calculate depth value , is represented as: ， In the formula, For depth images; Based on the target pixel coordinates and depth value By using camera calibration information to back-project the target pixel coordinates onto the camera coordinate system, the three-dimensional spatial position of the target in the camera coordinate system can be obtained. , is represented as: ， In the formula, , For camera focal length, , Principal point coordinates; For each target in the target detection and instance segmentation results, the pose of each target in the camera coordinate system is obtained by the 6D pose estimation method. Based on the pose of each target in the camera coordinate system, coordinate system transformation is performed in conjunction with camera calibration information to obtain the pose of the robotic arm's grasping point.

4. A robot control system for automatic surgical instrument matching according to claim 3, characterized in that, The pose of each target in the camera coordinate system is obtained by at least one of the following 6D pose estimation methods: 1) Keypoint regression combined with PnP method: From the target pixel coordinates Obtain the target key points ; The target key points and its corresponding target 3D model points Perform matching to obtain matching pairs; For the matching pair, the rotation matrix satisfying the projection model is solved using the PnP algorithm. Translation vector The projection model is as follows: ， ， In the formula, As a scale factor, For the camera intrinsic parameter matrix, Let be the target attitude rotation matrix. The target position vector; Based on the rotation matrix Translation vector And by minimizing the reprojection error, the pose of each target in the camera coordinate system is obtained from the matching pairs. , is represented as: ； 2) Point cloud registration method: Obtain the point cloud of each target, perform ICP registration with the CAD / template point cloud, and obtain the rotation matrix. Translation vector ; Based on the rotation matrix Translation vector The pose of each target in the camera coordinate system is derived. 3) Planar constraint attitude method: If the target is a planar target, fit a plane to the point cloud within the instance segmentation mask based on its instance segmentation. : ， In the formula, It is a plane normal vector. To segment 3D points in the point cloud within the mask for an instance, For planar offset; The local part of the planar target Axis and plane normal vector Align to obtain the alignment result; Principal component analysis is performed on the instance segmentation mask of the planar target to obtain the normal vector around the plane. The rotation angle is determined, and the rotation matrix is ​​determined in conjunction with the alignment result. ; Determine the plane The position of a reference point on the map is used as the translation vector. ; The rotation matrix Translation vector By combining the data, the pose of the planar target in the camera coordinate system can be obtained.

5. A robot control system for automatic surgical instrument matching according to claim 3, characterized in that, The steps for obtaining the gripping point pose of the robotic arm include: The pose of each target in the camera coordinate system is transformed, combined with camera calibration information, from the camera coordinate system to the robot arm base coordinate system / end-effector coordinate system, resulting in the pose in the robot arm base coordinate system / end-effector coordinate system. For scenarios with external / fixed cameras, the pose of the robotic arm in the base coordinate system is represented as follows: ， For the scenario where the eye is on the hand, the pose in the end effector coordinate system is represented as follows: ， In the formula, The pose of the robotic arm in the base coordinate system / end-effector coordinate system. This is a transformation from the camera coordinate system to the robot arm base coordinate system. The pose in the camera coordinate system. For real-time pose, This is a transformation from the camera coordinate system to the end effector coordinate system; Based on the pose in the robot arm base coordinate system / end-effector coordinate system, the grasping pose offset is superimposed to obtain the corresponding grasping point pose of the robot arm. , is represented as: ， ， In the formula, To capture pose offset, For orientation offset, To capture the displacement of the point.

6. A robot control system for automatic surgical instrument matching according to claim 1, characterized in that, The visual recognition and grasping target planning module also includes generating a pre-grasping pose along the approach direction based on the obtained grasping point pose of the robotic arm. The pre-grasp pose Represented as: ， In the formula, The gripping point pose of the robotic arm. To safely increase the distance, It is a pure translation operator along the approach direction.

7. A robot control system for automatic surgical instrument matching according to claim 1, characterized in that, In the force control and gripping execution control module, the force control adjustment process includes at least the following: Six-dimensional force / torque measurement signal acquisition and compensation: Acquire the six-dimensional force / torque measurement signal output by the six-dimensional force / torque sensor at the end of the robotic arm. Zero-point calibration and filtering are performed, and gravity and tool bias compensation are applied to obtain the estimated actual external force of the wrench. ,in It is the transpose of the three-dimensional force vector. The transpose of the three-dimensional torque vector, the actual external force wrench Represented as: ， In the formula, For zero drift / tool ​​inherent bias, This is a gravity-compensated wrench calculated based on the tool's mass, center of gravity, and current orientation. Joint angle; The actual external force wrench is represented by the adjoint matrix of the homogeneous transformation. Transforming to the robot arm base coordinate system or contact coordinate system yields the forces and moments in the robot arm base coordinate system or contact coordinate system, where the transformation expression is: ， ， In the formula, , Forces and torques in the robot arm's base coordinate system or contact coordinate system constitute the actual external force wrench in the robot arm's base coordinate system or contact coordinate system. Let be a rotation matrix. , Forces and torques in the end effector coordinate system or sensor coordinate system. This is the position vector of the origin at the end point in the base coordinate system; Based on the actual external force wrench The calculation expression for force error is as follows: or , In the formula, , For force error, To make contact with the wrench, For normal expectation force, This is the actual normal force; Based on the force error, a force control strategy is used to generate a task space correction. This force control strategy includes at least one of admittance control, impedance control, and position-force hybrid control. For admittance control: The force error or Substituting into the admittance model, the admittance model is expressed as: ， In the formula, , , For admittance parameters, This represents the position of the robotic arm's end effector in the contact coordinate system or its scalar displacement along the contact normal. For reference trajectory, The force error or ; Solving the admittance model yields... Relative to reference trajectory The deviation is used as a task space correction amount; For impedance control and position-force hybrid control: Based on the force error Both employ velocity-based force control formulas, performing impedance control and position-force hybrid control respectively, to obtain corresponding task space correction values. Joint velocity commands are then derived based on these task space correction values. The velocity-based force control formula is as follows: ， In the formula, For controlling the speed in the contact direction, For reference speed, For force feedback gain; The joint speed command Represented as: ， In the formula, This is a false rebellion of Jacobi.

8. A robot control system for automatic surgical instrument matching according to claim 7, characterized in that, In the force control and gripping execution control module, the steps for forming a complete force control and gripping process include: The task space correction amount is fused with the motion trajectory. In the non-contact phase, position control is the main method, and the task space correction amount obtained by the admittance control is fused with the motion trajectory to perform non-contact matching task execution control. In the contact phase, the task space correction amount obtained by the position-force hybrid control is fused with the motion trajectory to perform contact matching task execution control. Simultaneously, safety strategies are used to monitor abnormal forces and collision trends. When any of the following conditions are met, deceleration, stopping, or retraction Δz_back along the approach direction is executed, and abnormal events are recorded for tracing, ultimately obtaining the complete force control and grasping process. The conditions include at least the following: ， ， ， In the formula, For strength, The threshold value for external force amplitude. For the rate of change of force, For the force change rate threshold, For torque, This is the torque threshold.

9. A control method for a robot control system for automatic surgical instrument matching according to any one of claims 1-8, characterized in that, include: Obtain surgical scheduling data, analyze it, and obtain a list of surgical preferences for each surgical procedure and surgeon. Then, generate a list of instrument requirements for the target surgery through matching. Based on the equipment requirement list, query the equipment-warehouse mapping data to obtain the basic data of the pick-up and put-down path for the allocation task; The target warehouse area image is acquired, target detection and instance segmentation are performed, the target detection and instance segmentation results are obtained, and the pose of the robotic arm's grasping point is obtained by combining the camera calibration information. The motion trajectory of the robotic arm is generated based on the pose of the grasping point, and the force control adjustment during the grasping process is performed through the six-dimensional force / torque measurement signal at the end of the robotic arm. The task space correction amount is generated, and the task space correction amount is fused with the motion trajectory. Combined with safety strategies, a complete force control and grasping process is formed. Based on the tray layout rules corresponding to the surgical procedure and the surgeon, the target position of each instrument in the instrument requirement list is determined, and combined with the complete force control and grasping process, a complete setup process is formed to complete the automatic setup process.

10. A storage medium having a program stored thereon, characterized in that, When the program is executed, it implements the control method as described in claim 9.

Images