A neurosurgical robot visual servo control method, device and storage medium

Abstract

A method, device and storage medium for visual servo control of a neurosurgical robot The present invention relates to the field of medical equipment technology, and discloses a method, device and storage medium for visual servo control of a neurosurgical robot. The method comprises: modeling obstacles in the clinical surgical environment; constructing a virtual artificial potential field in the robot workspace in combination with the target puncture posture; establishing a six-dimensional force spinor acting on the end of the robot based on a two-point force constraint; modeling the visual servo tracking process guided by the potential field force as a mass-spring-damper system, and improving the model; using the numerical integration method, through iterative solution, generating a single-step planning trajectory, and delivering it to the robot's underlying controller for execution. The present invention improves the accuracy, stability and real-time performance of the visual servo tracking control of the neurosurgical robot in a complex clinical environment.

Description

Technical Field

[0001] The present invention relates to the technical field of medical equipment, and in particular to a visual servo control method, device and storage medium for a neurosurgery robot. Background Art

[0002] Neurosurgical diseases such as stroke carry extremely high mortality and disability rates, posing a significant threat to the lives and health of people worldwide. In recent years, robotic-assisted surgery has entered the field of neurosurgery. With its precise stereotactic positioning and stable instrument grip, it has improved treatment outcomes and surgical efficiency, opening up a new paradigm for innovation in neurosurgery. However, in clinical neurosurgery, immobilizing the patient's head significantly limits surgical space, increases the complexity of the procedure, and causes iatrogenic damage to the patient. With the advancement of robotics and artificial intelligence technologies, the autonomy of neurosurgical robots will be further enhanced, and accurate real-time registration and tracking capabilities will be an important future development direction.

[0003] After the target puncture position is established preoperatively, tracking and control of the surgical robot is currently mostly performed remotely by the physician. However, this is very laborious and inefficient for mobile patients. Physicians should focus more on the specific surgical procedure rather than repeatedly adjusting the surgical robot. Common path planning methods for autonomously positioning the surgical robot to the target position include graph search, numerical methods, and sampling-based methods. However, these methods rarely consider posture tracking and struggle to meet the real-time tracking requirements of mobile patients.

[0004] It can be seen that for non-fixed patients, how to design a safe and reliable visual servo tracking control method for neurosurgical robots is still a key problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0005] In view of this, the present invention provides a visual servo control method for a neurosurgical robot. First, the intraoperative obstacles are modeled, the robot's terminal force rotation is calculated based on an artificial potential field, an improved mass-spring-damper system is introduced, the robot's visual servo tracking process is modeled, and the desired trajectory is solved through numerical integration to achieve a safe, accurate, and reliable visual servo tracking process, laying the foundation for the intelligent and autonomous development of neurosurgical robots.

[0006] In a first aspect, the present invention provides a method for visual servo control of a neurosurgery robot, comprising the following steps:

[0007] S1: In a clinical surgical environment, obstacles are modeled using a convex hull algorithm. Considering the patient's head perturbations, a randomized consistency algorithm is used to construct a spherical surface that completely envelops the patient's head. This sphere is considered a typical dynamic obstacle during surgery.

[0008] S2: Based on the preoperative planning posture of the surgical robot, the intraoperative eye posture, and the static and dynamic obstacle models in the environment, the virtual artificial potential field is updated in the effective workspace of the robot;

[0009] S3: Calculate the six-dimensional force screw acting on the robot end tool through the two-point force constraint method;

[0010] S4: Generate the acceleration twist of the end tool at the current moment based on the improved mass-spring-damper system model;

[0011] S5: Iteratively solve the problem through numerical integration to generate the planned trajectory, which is then executed by the robot's underlying controller.

[0012] S6: Return to S2 and repeat the above steps until the visual servo tracking process of the surgical robot is completed.

[0013] Specifically, in step S1, the geometric features of the patient's head are extracted by an optical camera, and then a random consistency algorithm is used to set reasonable tolerance parameters to fit the spherical envelope surface of the head, and finally a dynamic obstacle model that needs to be considered in the tracking process is generated.

[0014] Specifically, in step S2, the quadratic potential field and the conical potential field are combined to construct a gravitational potential field; in the repulsive potential field, when the robot's end tool approaches the obstacle boundary, the repulsive force will approach infinity, and when it is away from the obstacle boundary to the set threshold, the repulsive force will become 0.

[0015] Specifically, in step S3, the six-dimensional force twist at the tip and distal ends of the robot end tool are calculated in the spatial coordinate system, and the sum of the six-dimensional force twist at the end tool is used to obtain the force twist on the robot end tool. This twist is then converted to the object coordinate system using the adjoint matrix.

[0016] Specifically, in step S4, the kinematic spin between the target pose and the current pose is calculated in the object coordinate system. This kinematic spin can be viewed as the stretched length of a virtual spring. Combined with the six-dimensional force spinor, the potential field-guided visual servo tracking process is modeled as a modified mass-spring-damper system in the object coordinate system. The velocity spinor and acceleration spinor represent the tracking velocity and acceleration of the robot end-tool in the object coordinate system.

[0017] Specifically, the improved mass-spring-damper system can feature multiple variable stiffness levels and adaptive damping based on target velocity. The stiffness of the virtual spring can be set to multiple values ​​depending on the spring length, and the damping can be adaptively adjusted based on the robot's tracking velocity.

[0018] Specifically, in step S5, at the start of tracking, the velocity screw is zero. The force screw and the motion screw are combined to generate the tracking acceleration based on the improved mass-spring-damper model. The velocity screw is calculated within the control cycle and mapped to the robot's joint angular velocities using the velocity Jacobian matrix. The joint angular velocities are then executed by the robot's underlying controller.

[0019] In a second aspect, the present invention provides a visual servo control device for a neurosurgery robot, which mainly includes a processor and a memory, and the visual servo control method described in the first aspect is stored in the memory and can be performed on the processor.

[0020] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the steps of the visual servo tracking method described in the first aspect.

[0021] It can be seen from the above technical solution that, compared with the prior art, the beneficial effects of the present invention include:

[0022] The visual servo tracking control method provided by the present invention, especially when used in conjunction with a surgical robot, (1) realizes robot spatial trajectory planning based on an improved artificial potential field method, which can adapt to dynamic and complex surgical environments; (2) does not require the doctor to adjust the robot end position during surgery, thereby improving surgical efficiency; (3) based on the mass-spring-damper system, the potential field force is mapped into acceleration, thereby improving the safety and stability of the tracking process; (4) the visual servo tracking control method provided by the present invention has important research significance and application value for further improving the autonomous operation level of surgical robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only one or more embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 A schematic diagram of the visual servo control flow of a neurosurgery robot provided by an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of a neurosurgery procedure for a non-fixed patient according to an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of a potential field force admittance visual servo control method provided by an embodiment of the present invention;

[0027] Figure 4 A schematic diagram of a robot tracking trajectory when a target moves at different speeds provided by an embodiment of the present invention;

[0028] Figure 5 This is a diagram showing the robot's motion performance analysis under different target movement speeds provided by an embodiment of the present invention;

[0029] Figure 6 Schematic diagram of the visual servo control device for a neurosurgery robot provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to help those skilled in the art better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Therefore, based on the embodiments of the present invention, other embodiments obtained by those skilled in the art should fall within the scope of protection of the embodiments of the present invention.

[0031] See Figure 2 , the neurosurgery robot participates in the operation, which is mainly divided into two stages: (1) Preoperative stage: the doctor analyzes the medical image, determines the lesion area, plans the puncture path, and then establishes the puncture posture of the robot relative to the patient's head; (2) Intraoperative stage: mainly includes alignment, robot positioning, automatic operation and doctor's surgery. Among them, for patients with non-fixed heads, the patient's head is disturbed during the operation, and the robot needs to be able to align in real time, and use the visual servo control method accordingly to ensure that the robot's end tool is consistent with the preoperative plan. An embodiment of the present invention provides a visual servo control method for a neurosurgery robot, see Figure 1 , mainly including obstacle modeling in the working environment, construction of virtual artificial potential field, modeling and numerical integration solution of visual servo control process based on improved mass-spring-damper system.

[0032] Obstacle Modeling: For general obstacles in the surgical environment, a fast convex hull method is used to quickly and accurately model the obstacles. Specifically for the patient's head, an optical camera is used to extract typical feature points of the patient's head (which can be fiducial points or facial features). A random sampling consistency method is then used to fit these feature points onto a sphere, thereby enveloping the patient's head. It should be noted that to ensure that the enveloping sphere encloses the patient's head as closely as possible without sacrificing too much safe working space, the tolerance parameters of the random sampling consistency method need to be appropriately set.

[0033] Construction of virtual artificial potential field: construct an artificial potential field for the effective working space of the robot in the surgical environment. In the environment, the desired posture exerts a gravitational force on the robot end; obstacles exert a repulsive force on the robot end. Among them, the gravitational potential field combines the quadratic potential field with the conical potential field. At this time, the gravitational force F att,i (q) Yes

[0034]

[0035] ξ i is the gravitational potential field coefficient at the i-th iteration, d is the distance threshold for the conical potential field to be converted into a parabolic potential field, o i (q),o i (q f ) represents the current position and the desired position of the robot in the i-th iteration. Similarly, in the repulsive potential field, when the robot position is close to the boundary, the repulsive force is infinite; when it is far away from the set position to a certain threshold, the repulsive force is 0. Repulsive force F rep,i (q) Yes

[0036]

[0037] Among them, n i is the repulsive potential field coefficient at the i-th iteration, ρ(o i (q)) indicates that the robot is in o i (q) is the shortest distance between any obstacles in the workspace, and q0 is the boundary threshold of the repulsive potential field. Here, the patient's head is an obstacle in the robot's tracking process and an object that requires special safety considerations.

[0038] Modeling of visual servo control process based on improved mass-spring-damper system: Figure 3 First, we introduce the force spinor in the space coordinate system {S} and Represents P top With P rem The force acting on the artificial potential field is

[0039]

[0040] here, Indicates P top With P rem The three-dimensional coordinates under {S}, Indicates P top With P rem The force under the artificial potential field. Further, in the robot base coordinate system {B}, we have

[0041]

[0042] is the adjoint transformation matrix from {S} to {B}, through Describe the six-dimensional force spinor in the spatial coordinate system {S} and It can be converted into the force torque describing the robot base coordinate system {B} and Finally, we obtain the six-dimensional force spinor That is the six-dimensional potential field force of the artificial potential field acting on the puncture needle, The calculation of is as follows:

[0043]

[0044] The visual servo tracking process guided by the potential field force can be modeled as a mass-spring-damper system, where the virtual mass of the puncture needle is M and the target position of the puncture needle is Target. i and the current pose Pose i The difference between A virtual spring is formed with a coefficient of K. Finally, the velocity damping coefficient of the system is N:

[0045]

[0046] in, and P on the puncture needle mid Velocity spinor and acceleration spinor. Under the action of , the M mass term will generate acceleration for the motion of the puncture needle, avoiding the acceleration discontinuity problem caused by the direct mapping of the potential field force to the tracking speed in the traditional artificial potential field method; the B impedance term can prevent excessive tracking speed; the K spring term has an inspiring effect, which can guide the puncture needle to escape from the local minimum. It should be emphasized that since the motion spinor cannot be directly differentiated, The transformation matrix is ​​the motion spinor The exponential mapping of

[0047]

[0048] Furthermore, the mass-spring-damper system model is improved, mainly from the perspective of multi-stage variable stiffness and impedance adaptation rate based on tracking speed. Specifically, assuming that the motion rotation between the robot target posture and the tracking posture is have

[0049]

[0050] Among them, K1 and K2 can be manually adjusted through experience, and the different stiffness of the virtual spring at different lengths can be established; M is The Euclidean norm threshold of can characterize the error between the target posture and the current posture. Similarly, the impedance adaptation rate B(t) is designed to satisfy:

[0051]

[0052] Where B0 is the initial impedance and γ is the adjustment coefficient related to the target speed. As the target speed increases, the damping coefficient increases to smooth the tracking of fast motion. When the target speed decreases, the damping coefficient decreases to avoid excessive oscillation. That is, the velocity spinor describing the tracking target in the object coordinate system.

[0053] Numerical integration solution: Through the method of numerical integration, iterative solution is performed. Assuming that the control period is Δt, then

[0054]

[0055] Among them, The solution process is recorded as APF(*) is the artificial potential field force function, and the velocity spinor output during the iteration process is Through the velocity Jacobian matrix J B Mapped to robot joint angular velocity

[0056]

[0057] final, It is then handed over to the robot's underlying controller for execution. Thus, the design and introduction of the visual servo control method for the neurosurgery robot are completed.

[0058] See Figure 4 , the embodiment of the present invention provides a neurosurgery robot, the robot end tool motion trajectory when the tracking target has three motion speeds of high, medium and low; refer to Figure 5 The present invention provides a robot that tracks the end-tool pose and joint angular velocity curves at high, medium, and low tracking speeds. The visual servo control method for a neurosurgical robot provided by the present invention exhibits excellent task accessibility, motion smoothness, environmental adaptability, and real-time tracking, effectively meeting the requirements of surgical robots in clinical procedures.

[0059] See Figure 6 , which is a system structure diagram of the robot visual servo control device provided by an embodiment of the present invention, which mainly includes:

[0060] One or more processors 101;

[0061] One or more memories 102 may store one or more computer programs.

[0062] When the one or more programs are executed by the one or more processors, the neurosurgery robot visual servo control method described in the embodiment of the present invention can be implemented.

[0063] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, can implement the robot visual servo control method.

[0064] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0065] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A visual servo control method for a neurosurgery robot, characterized in that: include: S1: Obstacles in the clinical surgical environment are modeled using a fast convex hull algorithm. A randomized consistency algorithm is used to build a spherical surface that envelops the patient's head. This sphere is considered a typical dynamic obstacle during surgery. S2: combining the current / target puncture posture of the surgical robot with the static / dynamic obstacles in the environment, and updating the virtual artificial potential field in the effective working space of the robot; S3: Calculate the six-dimensional force screw acting on the robot end tool through the two-point force constraint method; S4: Generate the acceleration of the end tool at the current moment based on the improved mass-spring-damper system model; S5: Use numerical integration methods to iteratively solve and generate single-step planning trajectories, which are then delivered to the robot's underlying controller for execution. S6: Return to S2 and repeat the above steps until the visual servo tracking process of the surgical robot is completed.

2. The visual servo control method for a neurosurgery robot according to claim 1, characterized in that: The gravitational potential field combines the quadratic potential field with the conical potential field. In the repulsive potential field, when the robot's end tool approaches the obstacle boundary, the repulsive force will approach infinity. When it moves away from the obstacle boundary to the set threshold, the repulsive force will become 0.

3. The visual servo control method for a neurosurgery robot according to claim 2, characterized in that: The artificial potential field forces acting on the tip and distal ends of the robot tool are calculated respectively, and then the six-dimensional force spinor acting on the end tool is calculated and described in the object coordinate system.

4. The visual servo control method for a neurosurgery robot according to claim 3, wherein: The difference between the target pose and the current pose of the robot's end tool can be established as a six-dimensional motion spinor describing the object coordinate system. The motion spinor can be regarded as the stretching length of the virtual spring. Combined with the six-dimensional force spinor, the potential field force-guided visual servo tracking process can be modeled as an improved mass-spring-damper system in the object coordinate system, where the velocity spinor and acceleration spinor are the tracking speed and acceleration of the robot's end tool.

5. The visual servo control method for a neurosurgery robot according to claim 4, characterized in that: The improved mass-spring-damper system can have multiple variable stiffness levels and adaptive damping based on target speed. The stiffness can be set to multiple stiffness values ​​according to the length of the virtual spring, and the damping can be adaptively adjusted according to the tracking target movement speed.

6. The visual servo control method for a neurosurgery robot according to claim 5, characterized in that: At the beginning of tracking, the velocity spinor is 0. Combining the six-dimensional force spinor and the motion spinor, based on the improved mass-spring-damper model, the tracking acceleration spinor is generated. The velocity spinor is calculated using the control cycle and mapped to the robot's joint angular velocity through the robot's velocity Jacobian matrix. The joint angular velocity is then executed by the robot's underlying controller.

7. A visual servo control device for a neurosurgery robot, characterized in that: include: A memory for storing program instructions to be executed; A processor is configured to call the program instructions stored in the memory to implement the visual servo control method for a neurosurgical robot according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores program code, and the program code can be used to implement the visual servo control method for a neurosurgical robot according to any one of claims 1 to 6.

Images