Medical surgical robot pre-collision detection method, system and device and storage medium

Abstract

The invention discloses a medical surgical robot pre-collision detection method, system and device and a storage medium. According to the method, the position and the speed of the robot end effector and the relative position of the robot end effector and the object can be calculated in real time, and whether the motion path of the robot can intersect with objects in the environment, especially a patient or not can be predicted. If a potential intersection is detected, the system may adjust the motion of the robot in advance, avoiding any direct contact with the human body or other sensitive object. The detection method and system are particularly suitable for a medical environment, the operation safety is improved, and the surgical fineness and efficiency can be kept.

Description

Technical Field

[0001] The present invention relates to the field of medical equipment graphics technology, and in particular to a medical surgical robot pre-collision detection method, system, equipment and storage medium. Background Art

[0002] Robotic collision detection can be roughly divided into two categories:

[0003] After a collision occurs, the occurrence of the collision is determined by detecting the current, torque or encoder value;

[0004] Before a collision occurs, the robot is judged to be in collision through modeling. This method is called robot pre-collision.

[0005] By detecting current, torque and encoder, it is determined whether a collision has occurred. This requires contact with the surrounding environment to stop. This is not allowed in medical environments where hard contact with the human body may occur, so a decision needs to be made before a collision. Summary of the invention

[0006] In order to improve the above technical problems, the present invention provides a pre-collision detection method, comprising:

[0007] S1. Acquire medical image information of the object, process it into spatial geometric information aligned with the robot navigation and positioning device, and use the spatial geometric information as a scene;

[0008] S2. Simplified scene geometry model;

[0009] S3. Perform pre-collision detection according to the scene and the state of the robot arm; wherein self-collision is ignored and collision detection of the external environment is performed;

[0010] In the collision detection link of the external environment, based on the half-space method, the intersection between the geometric body and the half-space boundary is determined by calculating the directed distance from the vertex of the geometric body of the end effector of the robot arm to the boundary plane of the half-space;

[0011] The half-space boundary plane is obtained by selecting a point on the object model plane and the normal vector of the plane.

[0012] According to an embodiment of the present invention, the medical image includes but is not limited to a head image of a subject, such as an oral image of a subject.

[0013] According to an implementation scheme of the present invention, step S2 includes: processing the robot arm connecting rod into a capsule bounding box, processing the robot arm joints into a cylindrical bounding box, processing the trolley used to fix the robot arm into a rectangular bounding box, fixing the robot arm end effector to the robot arm wrist, and using triangular facets as graphic element information.

[0014] According to an embodiment of the present invention, the self-collision includes a collision between any two or three of the robot arm link, the robot arm joint and the trolley.

[0015] According to an embodiment of the present invention, the collision detection of the external environment includes collision detection between the robot end effector and the object position. In one embodiment, it refers to the collision between the robot end effector and the object's oral cavity model.

[0016] According to an embodiment of the present invention, the state of the robotic arm is determined by the real-time position and posture data of the robotic arm.

[0017] According to an embodiment of the present invention, in step S3, the directed distance represents the shortest distance from the vertex to the plane.

[0018]

[0019] Where d is the mechanical arm and The Euclidean distance of is the normal vector of the patient plane, point P is a point in the set on the end effector of the robotic arm, and Q is a point on the plane where the patient is located.

[0020] According to an implementation scheme of the present invention, in step S3, if the directed distances of all vertices are positive or negative, the robot arm end effector geometry is completely located on one side of the half-space; if the directed distances of the vertices include both positive and negative numbers, the robot arm end effector geometry intersects with the boundary of the half-space.

[0021] According to an embodiment of the present invention, step S3 includes coarse detection and fine collision detection. When the coarse detection reaches a preset threshold, the collision distance is returned and the fine collision detection process is turned on. The fine collision detection process needs to process a bounding volume hierarchy (BVH) data structure, which describes the shape and structure information of the geometric body in detail.

[0022] According to an embodiment of the present invention, the detection method also includes collecting spatial information of the navigation and positioning end robot arm and the target position of the navigation and positioning end, uniformly converting the spatial information into the robot arm base coordinate system, and outputting the robot arm posture transformation matrix to complete the spatial transformation.

[0023] According to an embodiment of the present invention, the detection method further comprises step S4 of adjusting the action of the robot arm according to the collision detection result.

[0024] The present invention also provides a pre-collision detection system, comprising:

[0025] An image processing module is used to collect and receive medical scan images, process the information of the medical scan images, align and process the information with the navigation and positioning device of the manipulator, and transfer the obtained geometric body information with the spatial position to the geometric processing unit as a scene;

[0026] A navigation and positioning processing unit includes a transmitting end, a mechanical arm receiving end and a target position receiving end. The transmitting end optically locates the position of the receiving end and obtains the position matrix of the current receiving end in the navigation space.

[0027] A geometry processing unit, used for modeling system components, using triangular facets as primitive information; obtaining the geometry information with spatial position transmitted by the image processing module; and obtaining the robot arm posture data transmitted by the mechanical control module;

[0028] A robotic arm control module transmits robotic arm position and posture data to the geometric processing unit;

[0029] A central control unit is connected to the image processing module, the navigation and positioning processing unit, the geometry processing unit and the robotic arm control module respectively; used to process and analyze the data in the scene and the posture information of the robotic arm, and process the navigation space positioning information; manage the collision pair strategy of the geometry processing unit and use triangular facets to detect collisions between collision pairs; and process and interact with pre-collision detection results.

[0030] Preferably, the pre-collision detection method is performed in the pre-collision detection system.

[0031] The present invention also provides a surgical robot, comprising the above-mentioned pre-collision detection system.

[0032] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned pre-collision detection method when executing the computer program.

[0033] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned pre-collision detection method are implemented.

[0034] Beneficial Effects

[0035] The method of the present invention can calculate the position and speed of the robot end effector and its relative position to the object in real time, and can predict whether its motion path will intersect with objects in the environment, especially patients. If a potential intersection is detected, the system can adjust the robot's movement in advance to avoid any direct contact with the human body or other sensitive objects. The detection method and system are particularly suitable for use in medical environments, not only improving the safety of operations, but also maintaining the precision and efficiency of surgery.

[0036] Definition of terms:

[0037] The term "subject" refers to a subject to be treated, including but not limited to mammals, such as humans.

[0038] The “medical images” include but are not limited to three-dimensional medical images such as computed tomography (CT) images, magnetic resonance imaging (MRI), and positron emission tomography (PET).

[0039] The “computer-readable storage medium” includes, but is not limited to, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of a system usage scenario in a collision detection method of a surgical robot in an embodiment of the present invention, which includes 10-central control PC; 20-mechanical arm trolley; 201-mechanical arm controller; 30-305 represent joint links of mechanical arms 1, 2, 3, 4, 5, and 6 respectively; 306-end effector; 40-navigation positioning end transmitting end; 401-navigation positioning end mechanical arm receiving end; 402-navigation positioning end target position receiving end; 50-end power assembly; 60-patient;

[0041] Figure 2 It is a flowchart of the execution of various modules of the surgical robot to implement pre-collision detection in one embodiment of the present invention, which includes 701-central control unit, 702-image processing module, 703-navigation and positioning processing unit, 704-geometry processing unit, 705-mechanical arm control module;

[0042] Figure 3It is a dynamic schematic diagram of 60-patient's oral cavity model after the image is registered in an embodiment of the present invention, which includes the maxillary model, the dynamic process of the oral mandible rotating due to the rotation offset of the condyle to open and close the oral cavity; 601-oral maxillary model, 602-oral opening state, 603-oral mandibular model, 604-oral closing state, 605-condyle part.

[0043] Figure 4 It is the rotation transformation of the geometric model coordinate system of the oral robot surgical robot in one embodiment of the present invention: the receiving end coordinate system and the oral condyle fixed coordinate system model under the navigation positioning unit;

[0044] 403-receiving end coordinate system, 6012-oral condyle fixed fixed coordinate system model, 6013-oral condyle fixed relative motion coordinate system model, 4-navigation positioning unit. DETAILED DESCRIPTION

[0045] The technical scheme of the present invention will be further described in detail below in conjunction with specific embodiments. It should be understood that the following embodiments are only exemplary descriptions and explanations of the present invention and should not be construed as limiting the scope of protection of the present invention. All technologies implemented based on the above content of the present invention are included in the scope that the present invention is intended to protect.

[0046] The present invention provides a pre-collision detection system, such as Figure 2 As shown, the system includes a central control unit 701, a manipulator control module 705, a geometry processing unit 704, a navigation and positioning processing unit 703, and an image processing module 702. Pre-collision detection of the manipulator is achieved through navigation and positioning technology and based on known geometric information in the environment.

[0047] Take the pre-collision detection system module of the oral surgery navigation control system in the medical robot as an example. Figure 1 As shown, it includes a central control PC 10, a robotic arm trolley 20, a robotic arm controller 201, robotic arms 1, 2, 3, 4, 5, 6 joint links 30-305 end effectors 306, a navigation and positioning end transmitting end 40, a navigation and positioning end robotic arm receiving end 401, a navigation and positioning end target position receiving end 402, and an end power assembly 50.

[0048] The navigation and positioning processing unit 703 uses optical positioning as a positioning method, including a navigation and positioning end transmitter 40, a navigation and positioning end mechanical arm receiver 401, and a navigation and positioning end target position receiver 402. The navigation and positioning end transmitter locates the position of the navigation and positioning end mechanical arm receiver, and uniformly transforms the collected spatial information into the mechanical arm base coordinate system, and outputs the mechanical arm posture conversion matrix to complete the spatial conversion.

[0049] The image processing module 702 is used to collect and receive the medical image information of the object, and process it into an image geometry model, reconstruct the identified information, and process it into a geometric surface model. Specifically, the medical image information is processed into spatial geometry information that is aligned with the navigation and positioning end robotic arm receiving end, and the spatial geometry information is used as a scene: the posture information of the navigation and positioning end target position receiving end 402 under the navigation and positioning end robotic arm receiving end 401 is known, and the posture relationship of the navigation and positioning end target position receiving end 402 in the image coordinate system is also known, and the corresponding point of the target position is mapped to the coordinate system of the navigation and positioning end robotic arm receiving end 401. Since the oral maxillary bone model 601 and the oral mandibular bone model 603 are rigidly connected to the navigation and positioning end robotic arm receiving end 402, the geometric model is processed in the navigation coordinate system (such as Figure 4 shown).

[0050] The geometry processing unit 704 is the manager of the robot and the scene, receiving the sensor information of the robot control module 705 and the results of the navigation and positioning processing unit 703. It is responsible for managing the geometry model and the pose information obtained by each sensor during real-time motion. The geometry model is divided into a simplified model of the robot 1, 2, 3, 4, 5, 6 joint links 30-305, a simplified model of the robot trolley, and an image processing model under navigation vision.

[0051] The central control unit 701 is responsible for managing the data interaction of the above units and modules, including the posture information of the robot arm, processing the spatial positioning information of the navigation system; managing the collision pair strategy of the geometry processing unit and using triangular patches to detect collisions between collision pairs; and processing and interacting with pre-collision detection results.

[0052] Furthermore, in the self-collision detection link, for some negligible collision pairs, such as the collision between the robot arms 1, 2, 3, 4, 5, 6 joint links 30-305 and the robot arm trolley 20, the detection efficiency can be improved by simplifying the model. The central control unit can select a suitable collision detection strategy according to the actual situation.

[0053] For collision detection in the external environment, the image processing unit obtains image information, converts it into geometric model information, provides scene and positioning conversion processing in the navigation and positioning module, combines the posture information in the robot control module, and analyzes and processes the position and speed information in the central control unit.

[0054] Through the collaborative work of the above components, the pre-collision detection system can effectively identify and avoid potential collision risks, improving the safety of medical robot surgeries.

[0055] In a specific embodiment, the system is used for pre-collision detection in oral surgery, wherein the image processing module is used to receive and process medical scan images, align the patient's head image information with the external positioning device, obtain image information that matches the positioning information of the target position receiving end of the navigation positioning end, and after processing, obtain geometric body information with the attached spatial position and transmit it to the geometry processing unit as scene information.

[0056] For the geometry processing unit, as a non-complex component system, a simple geometry model was processed: the robot arm link model was processed as a capsule bounding box, and the joint was processed as a cylindrical bounding box; the robot arm trolley was processed as a rectangular bounding box, which was fixed to the robot wrist as the end effector, and triangular facets were used as primitive information. The processing of the scene geometry model should be simplified as much as possible.

[0057] The external collision mainly involves calculating the collision with the patient's oral cavity model. The main collision pairs involved are: the end effector 306 and the patient 60 position.

[0058] For the pre-collision detection method of the collision pair, the primitive information stored in the geometric model includes: basic data structures of geometric information - vertices, normal vectors, and topological information - adjacent edge information; its pre-processing (coarse detection) stage mainly occurs before approaching the target. The core of this process is to determine the position of the vertices, edges or faces of the geometric body relative to the boundary of the half-space. Specifically, this involves calculating the directed distance of each vertex in the geometric body to the boundary plane of the half-space, which can be obtained by the distance formula from the point to the plane.

[0059] Further, the method steps based on half-space collision detection are described as follows:

[0060] Define the half-space: First, define a half-space boundary plane, which is obtained by selecting the combined model of the patient's mouth ( Figure 3 ) plane and the normal vector of the plane.

[0061] Calculate signed distances: Then, for each vertex in the robot end effector geometry, calculate their signed distance to the half-space boundary plane. This distance represents the shortest distance from the vertex to the plane, and may be positive or negative depending on the position of the vertex relative to the plane.

[0062]

[0063] Where d is the mechanical arm and The Euclidean distance of is the normal vector of the patient plane, point P is a point in the set on the end effector, and Q is a point on the plane where the patient is located.

[0064] Detect intersection:

[0065] For the model:

[0066] If the directed distances (pointing distances) of all vertices are either all positive or all negative, the end effector geometry of the robotic arm is completely on one side of the half - space.

[0067] If there are both positive and negative numbers among the directed distances of the vertices, the end effector geometry of the robotic arm intersects the boundary of the half - space.

[0068] Specifically, during pre - processing, the patient 60 model is discretized into vertices, and a point cloud set is constructed to generate a rough model and a relatively accurate triangular mesh model.

[0069] Furthermore, for the coarse model, traverse the point cloud to obtain the maximum point Q max and the minimum point Q min The coordinate values form a bounding box, and the result is saved. These two points participate in the operation.

[0070] For the module that controls the robotic arm, its working frequency is set to 500Hz, and the maximum movement speed is limited to 100mm / s. On a computer with a 3.5GHz frequency and 6 - core CPU, the collision detection method needs to refresh the geometric model at a sufficient frequency and quickly respond to the calculation results to meet the performance requirements. The system continuously updates the pose, searches the distance range, and evaluates the Euclidean distance between the robotic arm 1, 2, 3, 4, 5, 6 joint linkages 30 - 305 robotic arm geometric model and the coarse detection model. Once the detected distance Δd < tor, where tor represents the coarse detection threshold, the algorithm returns the results of the intersection test and the query results of the nearest distance, and the system enters the fine detection stage. It should be emphasized that the robotic arm control module and the pre - collision detection are parallel computations.

[0071] In the fine detection stage, the system queries each vertex P i =(x i ,y i ,z i ) in the end effector 306 model, where i = 1, 2,..., n for the Boolean judgment with the defined half - space. For the accurate model data, compare the opening and closing movements of the mandible with the initialized patient model, and then register through the navigation positioning end robotic arm receiving ends 401 and 402 - the positions in the images of the navigation positioning end target position receiving ends, so as to predict the oral opening amplitude and obtain the estimated poses of the upper jaw and its linkage model.

[0072] The simplified mathematical expression for the open - mouth model is as follows:

[0073] Set Q0 as the initial state where the mandible is rigidly bound to the receiving end coordinate system 403, Q tis the position and posture of the receiving end coordinate system 403 when the mandible is in the open mouth state. Let the rotation matrix be R(θ) to represent the relative rotational motion of the mandible, and the translational motion d to represent the relative translational motion of the mandible. Q t =R(θ)Q0+d describes the transition from Q0 to Q t The transformation of θ represents the opening width. The present invention involves the case where the maxillary bone model does not involve movement.

[0074] Using the registration points captured by the navigation system, the skull model discrete points are bound to the initial stage to accurately obtain the pose data of the final model. For a triangular mesh, assume that there are n vertices, each vertex is represented by a three-dimensional coordinate as Q i =(x i ,y i ,z i ), where i = 1, 2, ..., n. For each point Q i , find the set of points A in its neighborhood i = {Q i1 ,Q i2 ,...,Q ik}, and use the Delaunay triangulation algorithm to construct the geometric figure to obtain the set of normal vectors of the triangle face The constructed collision pair primitive information is stored in the BVH data to speed up the collision detection efficiency.

[0075] The overall implementation process of the system pre-collision detection method is as follows in pseudo code:

[0076]

[0077] Handling collisions: When the coarse detection reaches a preset threshold, the system should return the collision distance and enter the fine collision detection process. At this time, it is necessary to process the bounding volume hierarchy (BVH) data structure, which describes the shape and structure information of the geometry in detail.

[0078] In the preprocessing stage of collision detection, the goal is to simplify the complex scene geometry model as much as possible, thereby improving calculation efficiency and reducing errors; when the coarse detection finds potential collision risks, the system will enter a more detailed detection. In this way, fast collision detection can be achieved while ensuring accuracy.

[0079] The above is an explanation of the embodiments of the present invention. However, the present invention is not limited to the above embodiments. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A pre-collision detection method, characterized in that: The detection method comprises: S1. Acquire medical image information of the object, process it into spatial geometric information aligned with the robot navigation and positioning device, and use the spatial geometric information as a scene; S2. Simplified scene geometry model; S3. Perform pre-collision detection according to the scene and the state of the robot arm; wherein self-collision is ignored and collision detection of the external environment is performed; In the collision detection link of the external environment, based on the half-space method, the intersection between the geometric body and the half-space boundary is determined by calculating the directed distance from the vertex of the geometric body of the end effector of the robot arm to the boundary plane of the half-space; The half-space boundary plane is obtained by selecting a point on the object model plane and the normal vector of the plane.

2. The pre-collision detection method according to claim 1, characterized in that: The medical image includes but is not limited to a head image of a subject, such as an oral image of a subject.

3. The pre-collision detection method according to claim 1, characterized in that: Step S2 includes: processing the robot arm connecting rod into a capsule bounding box, processing the robot arm joint into a cylindrical bounding box, processing the trolley used to fix the robot arm into a rectangular bounding box, fixing the robot arm end effector to the robot arm wrist, and using triangular facets as primitive information.

4. The pre-collision detection method according to claim 1, characterized in that: The self-collision includes a collision between any two or three of the robot arm connecting rod, the robot arm joint and the trolley; The collision detection of the external environment includes collision detection between the end effector of the robot arm and the position of the object; The state of the robotic arm is determined by the real-time position and posture data of the robotic arm.

5. The pre-collision detection method according to claim 1, characterized in that: In step S3, the directed distance represents the shortest distance from the vertex to the plane. Where d is the mechanical arm and The Euclidean distance of is the normal vector of the patient plane, point P is a point in the set on the end effector of the robot arm, and Q is a point on the plane where the patient is located; If all the signed distances of the vertices are positive or negative, the robot end effector geometry is completely located on one side of the half-space; if the signed distances of the vertices are both positive and negative, the robot end effector geometry intersects the half-space boundary.

6. The pre-collision detection method according to claim 1, characterized in that: Step S3 includes coarse detection and fine collision detection. When the coarse detection reaches a preset threshold, the collision distance is returned and the fine collision detection process is entered. The fine collision detection process needs to process the bounding volume hierarchy (BVH) data structure, which describes the shape and structure information of the geometric body in detail; And / or, the detection method also includes collecting spatial information of the navigation and positioning end manipulator and the navigation and positioning end target position, uniformly converting the spatial information into the manipulator base coordinate system, and outputting the manipulator posture conversion matrix to complete the spatial conversion. And / or, the detection method further includes step S4, adjusting the action of the robot arm according to the collision detection result.

7. A pre-collision detection system, characterized in that: The detection system comprises: An image processing module is used to collect and receive medical scan images, process the information of the medical scan images, align and process the information with the navigation and positioning device of the manipulator, and transfer the obtained geometric body information with the spatial position to the geometric processing unit as a scene; A navigation and positioning processing unit includes a transmitting end, a mechanical arm receiving end and a target position receiving end. The transmitting end optically locates the position of the receiving end and obtains the position matrix of the current receiving end in the navigation space. A geometry processing unit, used for modeling system components, using triangular facets as primitive information; obtaining the geometry information with spatial position transmitted by the image processing module; and obtaining the robot arm posture data transmitted by the mechanical control module; A robotic arm control module transmits robotic arm position and posture data to the geometric processing unit; A central control unit is connected to the image processing module, the navigation and positioning processing unit, the geometry processing unit and the robotic arm control module respectively; used to process and analyze the data in the scene and the posture information of the robotic arm, and process the navigation space positioning information; manage the collision pair strategy of the geometry processing unit and use triangular facets to detect collisions between collision pairs; and process and interact with pre-collision detection results. Preferably, the pre-collision detection method according to any one of claims 1 to 6 is performed in the pre-collision detection system.

8. A surgical robot, characterized in that: The surgical robot comprises the pre-collision detection system of claim 7.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the pre-collision detection method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the pre-collision detection method according to any one of claims 1 to 6 are implemented.

Images