Visual servo control method and device for neurosurgical robot and storage medium

Abstract

The invention relates to the technical field of medical equipment, and discloses a visual servo control method and device for a neurosurgical robot and a storage medium. The method comprises the following steps: modeling a clinical operation environment obstacle; constructing a virtual artificial potential field under the working space of the robot in combination with the target puncture pose; based on the two-point force constraint, the six-dimensional force spinor acting on the tail end of the robot is determined; the method comprises the following steps: modeling a visual servo tracking process guided by a potential field force into a quality-spring-damping system, and improving the model; and a numerical integration method is utilized, a single-step planning track is generated through iterative solution, and the single-step planning track is delivered to a robot bottom layer controller for execution. According to the invention, the accuracy, the stability and the real-time performance of visual servo tracking control of the neurosurgical robot in a complex clinical environment are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical equipment, and in particular to a visual servo control method, device and storage medium for a neurosurgery robot. Background Art

[0002] Neurosurgical diseases such as stroke have extremely high mortality and disability rates, posing a great threat to the lives and health of people around the world. In recent years, robot-assisted surgery has entered the field of neurosurgery. With its precise stereoscopic positioning and stable instrument holding capabilities, it has improved treatment effects and surgical efficiency, opening up a new paradigm for innovation in neurosurgery. However, in clinical neurosurgery, fixing the patient's head greatly limits the surgical space, increases the complexity of the surgical process, and causes iatrogenic damage to the patient. With the development of robotics and artificial intelligence technology, the autonomy of neurosurgical robots will be further improved, and having accurate real-time registration and tracking capabilities will be an important development direction in the future.

[0003] After the target puncture posture is established before the operation, the tracking and control of the surgical robot is currently mostly completed by the doctor remotely, but for non-fixed patients, this is very laborious and inefficient. Doctors should focus more on the specific surgical operation rather than repeatedly adjusting the surgical robot. The surgical robot autonomously positions itself to the target posture. Common path planning methods include graph search method, numerical method, sampling-based method, etc., but they rarely consider posture tracking and it is difficult to meet the real-time tracking requirements of non-fixed patients.

[0004] It can be seen that for non-fixed patients, how to design a safe and reliable visual servo tracking control method for a neurosurgical robot is still a key problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a visual servo control method for a neurosurgical robot. First, the intraoperative obstacles are modeled, the robot's terminal force spinor is calculated based on an artificial potential field, an improved mass-spring-damper system is introduced, the robot's visual servo tracking process is modeled, and the expected trajectory is solved by numerical integration to achieve a safe, accurate, and reliable visual servo tracking process, laying the foundation for the intelligent and autonomous development of neurosurgical robots.

[0006] In a first aspect, the present invention provides a method for visual servo control of a neurosurgery robot, comprising the following steps:

[0007] S1: In the clinical surgical environment, existing obstacles are modeled using the convex hull algorithm. Considering the disturbance of the patient's head, a random consistency algorithm is used to establish a spherical surface to achieve a complete envelope of the patient's head. The sphere is considered a typical dynamic obstacle during surgery;

[0008] S2: Combine the surgical robot's preoperative planning posture, intraoperative eye posture, and static and dynamic obstacle models in the environment to update the virtual artificial potential field in the robot's effective workspace;

[0009] S3: Calculate the six-dimensional force screw acting on the robot end tool by using the two-point force constraint method;

[0010] S4: Generate the acceleration spin of the end tool at the current moment based on the improved mass-spring-damper system model;

[0011] S5: Iterate and solve the problem through numerical integration to generate the planning trajectory, which is then executed by the robot's underlying controller.

[0012] S6: Return to S2 and repeat the above steps until the visual servo tracking process of the surgical robot is completed.

[0013] Specifically, in step S1, the geometric features of the patient's head are extracted by an optical camera, and then a random consistency algorithm is used to set reasonable tolerance parameters to fit the spherical envelope surface of the head, and finally a dynamic obstacle model that needs to be considered in the tracking process is generated.

[0014] Specifically, in step S2, the quadratic potential field and the conical potential field are combined to construct a gravitational potential field; in the repulsive potential field, when the robot's end tool approaches the obstacle boundary, the repulsive force will approach infinity, and when it is away from the obstacle boundary to a set threshold, the repulsive force will become 0.

[0015] Specifically, in step S3, the six-dimensional force spin of the tip and distal end of the robot end tool is calculated in the space coordinate system, and the force spin of the robot end tool is obtained by adding them together. The adjoint matrix is ​​used to convert it to the object coordinate system.

[0016] Specifically, in step S4, the motion spin between the target posture and the current posture is calculated in the object coordinate system. The motion spin can be regarded as the stretching length of the virtual spring. Combined with the six-dimensional force spinor, the visual servo tracking process guided by the potential field force is modeled as an improved mass-spring-damper system in the object coordinate system. Among them, the velocity spinor and the acceleration spinor are the tracking speed and acceleration of the robot end tool described in the object coordinate system.

[0017] Specifically, in the improved mass-spring-damper system, multiple variable stiffness and adaptive damping based on target speed can be provided, wherein the stiffness of the virtual spring can be set to multiple stiffness values ​​according to the spring length, and the damping can be adaptively adjusted according to the robot tracking speed.

[0018] Specifically, in step S5, when tracking starts, the velocity screw is 0, and the force screw and the motion screw are combined to generate the tracking acceleration based on the improved mass-spring-damper model. The velocity screw is calculated using the control cycle, and is mapped to the joint angular velocity of the robot through the velocity Jacobian matrix, and the joint angular velocity is finally handed over to the robot's underlying controller for execution.

[0019] In a second aspect, the present invention provides a visual servo control device for a neurosurgical robot, which mainly includes a processor and a memory, and the visual servo control method described in the first aspect is stored in the memory and can be performed on the processor.

[0020] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the steps of the visual servo tracking method described in the first aspect.

[0021] It can be seen from the above technical solution that, compared with the prior art, the beneficial effects of the present invention include:

[0022] The visual servo tracking control method provided by the present invention, especially when used in conjunction with a surgical robot, (1) realizes robot spatial trajectory planning based on an improved artificial potential field method, and can adapt to dynamic and complex surgical environments; (2) does not require the doctor to adjust the robot's end position during surgery, thereby improving surgical efficiency; (3) based on a mass-spring-damper system, the potential field force is mapped into acceleration, thereby improving the safety and stability of the tracking process; (4) the visual servo tracking control method provided by the present invention has important research significance and application value for further improving the autonomous operation level of surgical robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate one or more embodiments of the present specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only one or more embodiments of the present specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A schematic diagram of the visual servo control flow of a neurosurgery robot provided by an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of a neurosurgery procedure for a non-fixed patient provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of a potential field force admittance visual servo control method provided by an embodiment of the present invention;

[0027] Figure 4 A schematic diagram of a robot tracking trajectory when a target moves at different speeds provided by an embodiment of the present invention;

[0028] Figure 5 The robot motion performance analysis diagram under different target moving speeds provided by the embodiment of the present invention;

[0029] Figure 6 Schematic diagram of the visual servo control device for a neurosurgery robot provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make those skilled in the art better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Therefore, based on the embodiments in the embodiments of the present invention, other embodiments obtained by ordinary technicians in the field should all fall within the scope of protection of the embodiments of the present invention.

[0031] See also Figure 2 The participation of a neurosurgical robot in surgery is mainly divided into two stages: (1) Preoperative stage: The doctor analyzes medical images, determines the lesion area, plans the puncture path, and then establishes the puncture posture of the robot relative to the patient's head; (2) Intraoperative stage: mainly includes alignment, robot positioning, automatic operation and doctor's surgery. Among them, for patients with non-fixed heads, disturbances may occur during the operation. The robot needs to be able to align in real time and use a visual servo control method to ensure that the robot's end tool is consistent with the preoperative plan. An embodiment of the present invention provides a visual servo control method for a neurosurgical robot, see Figure 1 , mainly including obstacle modeling in the working environment, construction of virtual artificial potential field, modeling and numerical integration solution of visual servo control process based on improved mass-spring-damper system.

[0032] Obstacle modeling: For general obstacles in the surgical environment, the fast convex hull method is used to quickly and accurately model the obstacles; especially for the patient's head, the optical camera is used to extract the typical feature points of the patient's head (which can be reference points or facial features of the patient, etc.), and the random sampling consistency method is used to fit the feature points through the sphere to achieve the envelopment of the patient's head. It should be pointed out that in order to ensure that the enveloping sphere wraps the patient's head as much as possible without sacrificing too much safe working space, the tolerance parameters of the random sampling consistency method need to be reasonably set.

[0033] Construction of virtual artificial potential field: construct artificial potential field for the effective working space of the robot in the intraoperative surgical environment. In the environment, the desired posture exerts gravitational force on the end of the robot; obstacles exert repulsive force on the end of the robot. Among them, the gravitational potential field combines the quadratic potential field with the conical potential field. At this time, the gravitational force F att,i (q) Yes

[0034]

[0035] ξ i is the gravitational potential field coefficient at the i-th iteration, d is the distance threshold for the conversion of the conical potential field to the parabolic potential field, o i (q),o i (q f ) represents the current position and the expected position of the robot in the i-th iteration. Similarly, in the repulsive potential field, when the robot position is close to the boundary, the repulsive force is infinite; when it is far away from the set position to a certain threshold, the repulsive force is 0. Repulsive force F rep,i (q) Yes

[0036]

[0037] Among them, n i is the repulsive potential field coefficient at the i-th iteration, ρ(o i (q)) indicates that the robot is at o i (q) is the shortest distance between any obstacles in the workspace, and q0 is the boundary threshold of the repulsive potential field. Here, the patient's head is an obstacle in the robot tracking process and is also an object that needs to be considered for safety.

[0038] Modeling of visual servo control process based on improved mass-spring-damper system: Figure 3 First, we introduce the force spinor in the space coordinate system {S} and Respectively represent P top With P rem The force under the artificial potential field is

[0039]

[0040] here, Indicates P top With P rem The three-dimensional coordinates under {S}, Indicates P top With P rem The force under the artificial potential field. Furthermore, in the robot base coordinate system {B}, we have

[0041]

[0042] is the adjoint transformation matrix from {S} to {B}, through Describe the six-dimensional force spinor in the spatial coordinate system {S} and It can be converted into the force torque describing the robot base coordinate system {B} and Finally, we get the six-dimensional force spinor That is, the six-dimensional potential field force of the artificial potential field acting on the puncture needle. The calculation of is as follows:

[0043]

[0044] The visual servo tracking process guided by the potential field force can be modeled as a mass-spring-damper system, where the virtual mass of the puncture needle is M and the target position of the puncture needle is Target. i With the current pose Pose i The difference between A virtual spring is constructed with a coefficient of K. Finally, the velocity damping coefficient of the system is N:

[0045]

[0046] in, and P mid The velocity screw and acceleration screw. Under the action, the M mass term will generate acceleration for the motion of the puncture needle, avoiding the acceleration discontinuity problem caused by the direct mapping of the potential field force to the tracking speed in the traditional artificial potential field method; the B impedance term can prevent excessive tracking speed; the K spring term has an inspiring effect and can guide the puncture needle to escape from the trap of the local minimum. It should be emphasized that since the motion spinor cannot be directly differentiated, The transformation matrix is ​​the motion spinor The exponential mapping of

[0047]

[0048] Furthermore, the mass-spring-damper system model is improved, mainly from the perspective of multi-stage variable stiffness and impedance adaptation rate based on tracking speed. Specifically, it is assumed that the motion rotation between the robot target posture and the tracking posture have

[0049]

[0050] Among them, K1 and K2 can be manually adjusted through experience, and the different stiffness of the virtual spring at different lengths can be established; M is The Euclidean norm threshold of can characterize the error between the target posture and the current posture. Similarly, the impedance adaptation rate B(t) is designed to satisfy:

[0051]

[0052] Among them, B0 is the initial impedance, γ is the adjustment coefficient related to the target speed. As the target speed increases, the damping coefficient increases accordingly to smooth the tracking of fast motion; when the target speed decreases, the damping coefficient decreases to avoid excessive oscillation. That is, the velocity spinor describing the tracking target in the object coordinate system.

[0053] Numerical integration solution: Through the method of numerical integration, iterative solution is performed. Assuming that the control period is Δt, we have

[0054]

[0055] Among them, The solution process is recorded as APF(*) is the artificial potential field force function, and the velocity spinor output during the iteration process is Through the velocity Jacobian matrix J B Mapped to robot joint angular velocity

[0056]

[0057] final, It is then executed by the robot's underlying controller. Thus, the design and introduction of the visual servo control method for the neurosurgery robot are completed.

[0058] See also Figure 4 , an embodiment of the present invention provides a neurosurgery robot, wherein the motion trajectory of the robot end tool is determined when the tracking target has three motion speeds: high, medium and low; refer to Figure 5 The embodiment of the present invention provides a robot, and the robot end tool posture change and joint angular velocity curves under three tracking speeds: high, medium and low. It can be found that the neurosurgery robot visual servo control method provided by the present invention has good task accessibility, motion smoothness, environmental adaptability and tracking real-time performance, and can better meet the task requirements of surgical robots in clinical operations.

[0059] See also Figure 6 , which is a system structure diagram of a robot visual servo control device provided by an embodiment of the present invention, which mainly includes:

[0060] One or more processors 101;

[0061] One or more memories 102 may store one or more computer programs.

[0062] When the one or more programs are executed by the one or more processors, the visual servo control method for a neurosurgery robot described in an embodiment of the present invention can be implemented.

[0063] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, can implement the robot visual servo control method.

[0064] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0065] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A visual servo control method for a neurosurgery robot, characterized in that: include: S1: The obstacles in the clinical surgical environment are modeled using a fast convex hull algorithm. In particular, for the patient's head, a random consistency algorithm is used to establish a spherical surface to envelop the patient's head. The sphere is considered a typical dynamic obstacle during surgery; S2: combining the current / target puncture posture of the surgical robot with the static / dynamic obstacles in the environment, and updating the virtual artificial potential field in the effective working space of the robot; S3: Calculate the six-dimensional force screw acting on the robot end tool through the two-point force constraint method; S4: Generate the acceleration of the end tool at the current moment based on the improved mass-spring-damper system model; S5: Use the numerical integration method to iteratively solve and generate a single-step planning trajectory, which is then delivered to the robot's underlying controller for execution. S6: Return to S2 and repeat the above steps until the visual servo tracking process of the surgical robot is completed.

2. A visual servo control method for a neurosurgery robot, characterized in that: The gravitational potential field combines the quadratic potential field with the conical potential field. In the repulsive potential field, when the robot's end tool approaches the obstacle boundary, the repulsive force will approach infinity, and when it moves away from the obstacle boundary to a set threshold, the repulsive force will become 0.

3. A visual servo control method for a neurosurgery robot, characterized in that: The artificial potential field forces acting on the tip and distal ends of the robot tool are calculated respectively, and then the six-dimensional force spinor acting on the end tool is calculated and described in the object coordinate system.

4. A visual servo control method for a neurosurgery robot, characterized in that: The difference between the target pose and the current pose of the robot end tool can be established as a six-dimensional motion spinor describing the object coordinate system. The motion spinor can be regarded as the stretching length of the virtual spring. Combined with the six-dimensional force spinor, the visual servo tracking process guided by the potential field force can be modeled as an improved mass-spring-damper system in the object coordinate system. Among them, the velocity spinor and the acceleration spinor are the tracking speed and acceleration of the robot end tool.

5. A visual servo control method for a neurosurgery robot, characterized in that: Improved mass-spring-damper system, with multiple variable stiffness and adaptive damping based on target speed. The stiffness can be set to multiple stiffness values ​​according to the length of the virtual spring, and the damping can be adaptively adjusted according to the speed of the target movement.

6. A visual servo control method for a neurosurgery robot, characterized in that: At the beginning of tracking, the velocity screw is 0. Combining the six-dimensional force screw and the motion screw, the tracking acceleration screw is generated based on the improved mass-spring-damper model. The velocity screw is calculated using the control cycle and mapped to the robot joint angular velocity through the robot's velocity Jacobian matrix. The joint angular velocity is executed by the robot's underlying controller.

7. A visual servo control device for a neurosurgery robot, characterized in that: include: A memory, used to store program instructions to be executed; A processor is used to call the program instructions stored in the memory to implement the visual servo control method of a neurosurgical robot as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, and the program code can be used to implement the visual servo control method for a neurosurgical robot as described in any one of claims 1 to 6.

Images