A control device for inserting needles in eye surgery

By using the eyeball and needle input tool images acquired by the camera equipment, adjusting the posture of the trolley robot and the position of the needle input tool, the problems of accurate and low efficiency of needle input operation in the prior art are solved, and more flexible and efficient Eye surgery operation.

CN118542773BActive Publication Date: 2024-12-06GUANGZHOU WEIMOU MEDICAL INSTR CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411026364.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-12-06
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

The existing eyeball surgical needle insertion control device fails to effectively utilize the preoperative preparation stage and operating room positioning of robot-assisted surgery, resulting in low accuracy and efficiency of needle insertion operation and reducing the flexibility of the operation.

Method used

The eyeball image of the target patient is obtained by the first camera device to adjust the posture position of the trolley robot, and the needle input tool image collected by the second camera device is used to determine the posture position of the needle input tool, realize accurate needle input operation, and optimize the robot utilization rate.

Benefits of technology

It improves the accuracy and efficiency of eyeball surgery needle insertion, enhances the flexibility of surgical operation, and ensures minimized preoperative approach exercise and utilization of maximum intraoperative motor ability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118542773B_ABST
    Figure CN118542773B_ABST
Patent Text Reader

Abstract

The present application provides a control device for eyeball surgery needle insertion, the control device includes a first image acquisition module, used to control the first camera device to aim at the eyeball of the target patient, and obtain the eyeball image corresponding to the eyeball of the target patient; a second image acquisition module, used to adjust the posture position parameters of the trolley robot to correspond to the eyeball posture position parameters corresponding to the eyeball image, and obtain the needle insertion area image corresponding to the needle insertion tool collected by the second camera device; a needle insertion posture determination module, used to determine the needle insertion posture position data corresponding to the needle insertion tool according to the needle insertion area image; a surgical needle insertion control module, used to control the needle insertion tool to insert the needle into the eyeball of the target patient according to the needle insertion posture position data. Through the above device, the accuracy, efficiency and operational flexibility of eyeball surgery needle insertion are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of robot control technology, and in particular to a control device for inserting a needle during eye surgery. Background Art

[0002] Needle insertion is very important during eye surgery, which is directly related to the success of the operation and the safety and normal recovery of the patient. For example, for eye surgery that requires precise injection of drugs, such as intravitreal injection, accurate needle insertion is the key to the success of the operation. If the injection position deviates, the drug may fail to reach the intended treatment area, thereby affecting the efficacy. In addition, inaccurate needle insertion may cause complications such as ruptured blood vessels and bleeding in the eye, and may even lead to serious consequences such as decreased vision.

[0003] At present, the control device for needle insertion in eye surgery generally controls the docking of the needle by assuming that the trolley robot is in the optimal position, but does not pay attention to the preoperative preparation stage of robot-assisted ophthalmic surgery and the positioning of the robot platform itself in the operating room, which reduces the accuracy and efficiency of needle insertion in eye surgery and further reduces the flexibility of eye surgery operations. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a control device for eye surgery needle insertion, which adjusts the posture position of the trolley robot for surgery according to the eye image corresponding to the eye of the target patient acquired by the first camera device, further determines the needle insertion posture position of the needle insertion tool according to the needle insertion area image corresponding to the needle insertion tool acquired by the second camera device, and controls the needle insertion tool to perform surgical needle insertion operation on the eye of the target patient, thereby minimizing the target approach movement in the preoperative stage and maintaining the maximum movement ability of the robot during the operation to accurately optimize the utilization rate of the robot, improve the accuracy and efficiency of eye surgery needle insertion, and thereby improve the flexibility of eye surgery operations.

[0005] The present application embodiment provides a control device for inserting a needle in an eyeball surgery, which is applied to a trolley robot; wherein the trolley robot is equipped with a first camera device, a second camera device and a needle insertion tool, and the control device includes:

[0006] A first image acquisition module, used for controlling the first camera device to aim at the eyeball of a target patient, and acquiring an eyeball image corresponding to the eyeball of the target patient;

[0007] A second image acquisition module is used to adjust the posture position parameters of the trolley robot to correspond to the eyeball posture position parameters corresponding to the eyeball image, and obtain the needle insertion area image corresponding to the needle insertion tool collected by the second camera device;

[0008] A needle insertion posture determination module, used to determine the needle insertion posture position data corresponding to the needle insertion tool according to the needle insertion area image;

[0009] The surgical needle insertion control module is used to control the needle insertion tool to insert the needle into the eyeball of the target patient according to the needle insertion posture position data.

[0010] Furthermore, when the first image acquisition module is used to control the first camera device to aim at the eyeball of the target patient, the first image acquisition module is used to:

[0011] Controlling the first camera device to face the eyeball of the target patient, and acquiring a camera field of view image captured by the first camera device;

[0012] Performing edge detection on the camera field of view image, and extracting eyeball contour data determined by edge detection in the camera field of view image;

[0013] Determining whether the eyeball contour data meets the pre-calibrated eyeball model standard corresponding to the eyeball of the target patient;

[0014] If the eyeball contour data meets the eyeball model standard, it is determined that the first camera device is aimed at the eyeball of the target patient.

[0015] Furthermore, the first image acquisition module is also used for:

[0016] If the eyeball contour data does not meet the eyeball model standard, based on the eyeball contour data, using a preset visual servo controller to obtain deviation data between the camera field of view image and the eyeball of the target patient;

[0017] Based on the deviation data, a preset feedback controller is used to adjust the orientation of the first camera device until the eyeball contour data meets the eyeball model standard.

[0018] Furthermore, when the second image acquisition module is used to determine the eyeball posture position parameters corresponding to the eyeball image, the second image acquisition module is used to:

[0019] Segmenting the eyeball image to obtain eyeball sub-images segmented from the eyeball image;

[0020] Performing edge detection on the eyeball sub-image to determine contour position data corresponding to the eyeball of the target patient;

[0021] Based on the contour position data, performing principal axis rotation calculation and ellipse fitting calculation on the eyeball sub-image respectively to obtain eyeball posture data corresponding to the eyeball of the target patient;

[0022] Based on the contour position data and the eyeball posture data, as well as the posture prediction data output by a preset detection model according to the eyeball image, the eyeball posture position parameters corresponding to the eyeball image are determined.

[0023] Further, when the needle insertion posture determination module is used to determine the needle insertion posture position data corresponding to the needle insertion tool according to the needle insertion area image, the needle insertion posture determination module is used to:

[0024] Based on the angle data corresponding to the needle insertion area image, determining the needle insertion posture angle value corresponding to the needle insertion tool;

[0025] Based on the needle insertion area image, determining tool position data corresponding to the needle insertion tool;

[0026] The needle insertion posture angle value and the tool position data are determined as the needle insertion posture position data corresponding to the needle insertion tool.

[0027] Further, when the needle insertion posture determination module is used to determine the needle insertion posture angle value corresponding to the needle insertion tool based on the angle data corresponding to the needle insertion area image, the needle insertion posture determination module is used to:

[0028] According to the surgical requirements corresponding to the eye surgery, a target needle insertion area is divided in the needle insertion area image;

[0029] Determine the optical axis center angle corresponding to the target needle insertion area as the needle insertion area center angle value corresponding to the needle insertion area image;

[0030] For the target needle insertion area, a preset inscribed angle relationship is used to determine a needle insertion area limit angle value corresponding to the needle insertion area image;

[0031] Based on the central angle value of the needle insertion area and the limit angle value of the needle insertion area, the needle insertion posture angle value corresponding to the needle insertion tool is determined.

[0032] Further, when the needle insertion posture determination module is used to determine the tool position data corresponding to the needle insertion tool based on the needle insertion area image, the needle insertion posture determination module is used to:

[0033] Taking the eyeball of the target patient as the center point and according to preset horizontal and vertical axis directions, a plane coordinate system is established in the needle insertion area image;

[0034] Determining the plane coordinate data corresponding to the needle insertion tool in the plane coordinate system;

[0035] Based on the parameter matrix corresponding to the second camera device and the plane coordinate data, the tool position data corresponding to the needle insertion tool is determined.

[0036] The control device for eye surgery needle insertion provided in an embodiment of the present application is applied to a trolley robot; wherein the trolley robot is equipped with a first camera device, a second camera device and a needle insertion tool, and the control device includes: a first image acquisition module, used to control the first camera device to aim at the eyeball of a target patient, and obtain an eyeball image corresponding to the eyeball of the target patient; a second image acquisition module, used to adjust the posture position parameters of the trolley robot to correspond to the eyeball posture position parameters corresponding to the eyeball image, and obtain a needle insertion area image corresponding to the needle insertion tool collected by the second camera device; a needle insertion posture determination module, used to determine the needle insertion posture position data corresponding to the needle insertion tool according to the needle insertion area image; a surgical needle insertion control module, used to control the needle insertion tool to insert the needle into the eyeball of the target patient according to the needle insertion posture position data.

[0037] Compared with the device in the prior art that controls the docking of the needle by assuming that the trolley robot is in the optimal position, the posture position of the trolley robot for surgery is adjusted according to the eye image corresponding to the eye of the target patient acquired by the first camera device, and the needle posture position of the needle tool is further determined according to the needle insertion area image corresponding to the needle insertion tool acquired by the second camera device, and the needle insertion tool is controlled to perform surgical needle insertion operation on the eye of the target patient, thereby minimizing the target approach movement in the preoperative stage and maintaining the maximum movement ability of the robot during the operation to accurately optimize the utilization rate of the robot, thereby improving the accuracy and efficiency of eye surgery needle insertion, and thereby improving the flexibility of eye surgery operations.

[0038] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A schematic diagram of the structure of a control device for inserting a needle into an eyeball surgery provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of an eye surgery device based on a trolley robot provided in an embodiment of the present application;

[0042] Figure 3A schematic diagram of the operation of a needle insertion tool provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0044] Research has found that, at present, the control device for needle insertion in eye surgery generally controls the docking of the needle by assuming that the trolley robot is in the optimal position, but does not pay attention to the preoperative preparation stage of robot-assisted ophthalmic surgery and the positioning of the robot platform itself in the operating room, which reduces the accuracy and efficiency of needle insertion in eye surgery, and thus reduces the flexibility of eye surgery operations.

[0045] Based on this, an embodiment of the present application provides a control device for eye surgery needle insertion, which adjusts the posture position of the trolley robot for surgery according to the eye image corresponding to the eye of the target patient acquired by the first camera device, and further determines the needle insertion posture position of the needle insertion tool according to the needle insertion area image corresponding to the needle insertion tool acquired by the second camera device, and controls the needle insertion tool to perform surgical needle insertion operation on the eye of the target patient, thereby minimizing the target approach movement in the preoperative stage and maintaining the maximum movement ability of the robot during the operation to accurately optimize the utilization rate of the robot, thereby improving the accuracy and efficiency of eye surgery needle insertion, and thereby improving the flexibility of eye surgery operations.

[0046] See also Figure 1 , Figure 1 This is a schematic diagram of the structure of a control device for inserting a needle into an eyeball surgery provided by an embodiment of the present application. Figure 1 As shown in , the control device 100 is applied to a trolley robot; wherein the trolley robot is equipped with a first camera device, a second camera device and a needle insertion tool, and the control device 100 includes:

[0047] The first image acquisition module 110 is used to control the first camera device to aim at the eyeball of the target patient, and acquire the eyeball image corresponding to the eyeball of the target patient.

[0048] It should be noted that when performing eye surgery on the target patient, an eye surgery device based on a trolley robot is required to perform the surgical operation; wherein the trolley robot is equipped with a first camera device, a second camera device and a needle insertion tool.

[0049] In one possible implementation of the present application, the first camera device may include a depth color camera (RGBD camera); the second camera device may include a color camera (RGB camera); and the needle insertion tool may include a puncture card and an injection needle.

[0050] For details, please refer to Figure 2 , Figure 2 A schematic diagram of an eye surgery device based on a trolley robot provided in an embodiment of the present application. Figure 2 As shown in the figure, a trolley robot is set on the operating bed where the target patient is located, including a four-degree-of-freedom robot and a five-degree-of-freedom robot; wherein the degree of freedom of the robot is the number of joints that the robot can move independently; the four-degree-of-freedom robot is equipped with a second camera device (RGB camera) on the bracket set in conjunction with the operating bed; the five-degree-of-freedom robot is equipped with a first camera device (RGBD camera) on the bracket set in conjunction with the operating bed; in addition, the operating bed is connected to a surgical analysis instrument and a graphical terminal.

[0051] During the specific implementation of the first image acquisition module 110, when the trolley robot is perpendicular to the head direction of the target patient, the trolley robot is positioned close to the eyes of the target patient, and the first camera device is controlled to face the eyeball of the target patient to completely capture the eyeball of the target patient; the trolley robot is controlled to perform translational movement to adjust the position and height of the trolley robot to a predetermined range; and an eye image corresponding to the eyeball of the target patient is acquired.

[0052] In one embodiment of the present application, in a specific implementation, when the first image acquisition module is used to control the first camera device to aim at the eyeball of the target patient, the first image acquisition module is used to:

[0053] Step S1101: control the first camera device to face the eyeball of a target patient, and obtain a camera field of view image captured by the first camera device.

[0054] In this step, the movement of the trolley robot is adjusted according to the preset position and direction parameters, and the first camera device is controlled to face the eyeball of the target patient until the field of view of the first camera device can completely include the entire eyeball of the target patient; and the camera field of view image captured by the first camera device at this time is obtained.

[0055] Step S1102: perform edge detection on the camera field of view image, and extract eyeball contour data determined by edge detection in the camera field of view image.

[0056] In this step, edge detection is performed on the camera field of view image using a preset computer vision algorithm, and eyeball contour data determined by edge detection is extracted from the camera field of view image.

[0057] Step S1103: determine whether the eyeball contour data meets the pre-calibrated eyeball model standard corresponding to the eyeball of the target patient.

[0058] In this step, during specific implementation, first, it is evaluated whether the eyeball contour data conforms to the standard eyeball shape and size to determine whether there is a problem with the image captured by the first camera device; then, if it conforms to the standard eyeball shape and size, a preset matching algorithm is used to map the 2D image corresponding to the collected eyeball contour data to an eyeball model corresponding to the eyeball of a pre-calibrated target patient; finally, it is determined whether the eyeball contour data conforms to the eyeball model standard corresponding to the eyeball of the target patient.

[0059] Step S1104: If the eyeball contour data meets the eyeball model standard, determine that the first camera device is aimed at the eyeball of the target patient.

[0060] In this step, if the eyeball contour data meets the eyeball model standard, indicating that the camera field of view image captured by the first camera device at this moment includes the entire eyeball of the target patient, it is determined that the first camera device is aimed at the eyeball of the target patient.

[0061] Furthermore, if the eyeball contour data does not meet the eyeball model standard, the first image acquisition module 110 is further configured to:

[0062] Step S1105 : Based on the eyeball contour data, a preset visual servo controller is used to obtain deviation data between the camera field of view image and the eyeball of the target patient.

[0063] In this step, if the eyeball contour data does not meet the eyeball model standard, the visual servo control algorithm set in the preset visual servo controller is used to calculate the deviation data between the camera field of view image and the eyeball of the target patient based on the eyeball contour data and the camera field of view image collected by the first camera device.

[0064] Step S1106: Based on the deviation data, use a preset feedback controller to adjust the orientation of the first camera device until the eyeball contour data meets the eyeball model standard.

[0065] In this step, based on the deviation data, the speed and direction of the trolley robot are adjusted using the feedback control algorithm set in the preset feedback controller, so that the trolley robot gradually approaches the position of the target patient to adjust the orientation of the first camera device; the deviation data is iteratively calculated until the eyeball contour data meets the eyeball model standard; and the adjustment is stopped after the standard is met.

[0066] In one possible implementation manner of the present application, the feedback control algorithm includes but is not limited to a PID filtering algorithm and a Kalman filtering algorithm.

[0067] The second image acquisition module 120 is used to adjust the posture position parameters of the trolley robot to correspond to the eye posture position parameters corresponding to the eye image, and obtain the needle insertion area image corresponding to the needle insertion tool captured by the second camera device.

[0068] During the specific implementation of the second image acquisition module 120, first, based on the predefined position and posture restrictions, the posture position parameters of the trolley robot are fine-tuned, and the sensor data or visual feedback of the trolley robot is used to ensure that the trolley robot reaches the desired position and posture; then, the eye posture position parameters corresponding to the eye image are determined; thereafter, the posture position parameters of the trolley robot are adjusted to correspond to the eye posture position parameters corresponding to the eye image; finally, the needle insertion area image corresponding to the needle insertion tool captured by the second camera device is acquired.

[0069] Here, after the first camera device captures the complete eyeball, due to the angle problem of the camera device at this time, or the center point of the target patient's eyeball is too far from the center point of the picture captured by the first camera device, the focus of the camera device is out of focus. Therefore, the posture position parameters of the trolley robot are slightly adjusted to overcome the above problems.

[0070] Furthermore, when the second image acquisition module 120 is used to determine the eyeball posture position parameters corresponding to the eyeball image, the second image acquisition module 120 is used to:

[0071] Step S1201: segment the eyeball image to obtain eyeball sub-images segmented from the eyeball image.

[0072] In this step, during the specific implementation, first, the eyeball image is segmented into image windows to obtain a plurality of sub-window images corresponding to the eyeball image; then, the eyeball image is segmented into iris contours using a preset edge detector to obtain an iris contour image corresponding to the eyeball image; finally, the plurality of sub-window images and the iris contour image are determined as the eyeball sub-images.

[0073] Step S1202: perform edge detection on the eyeball sub-image to determine contour position data corresponding to the eyeball of the target patient.

[0074] In this step, the preset Canny edge detector is used to perform edge detection on the eyeball sub-image, and the contour position data corresponding to the eyeball of the target patient is determined according to the edge detection result.

[0075] Step S1203: Based on the contour position data, main axis rotation calculation and ellipse fitting calculation are performed on the eyeball sub-image to obtain eyeball posture data corresponding to the eyeball of the target patient.

[0076] In this step, during the specific implementation, first, based on the contour position data, the main axis rotation calculation is performed on the multiple sub-window images in the eyeball sub-image to obtain the first eyeball posture data corresponding to the eyeball of the target patient; then, based on the contour position data, the iris contour image in the eyeball sub-image is performed with an ellipse fitting calculation to obtain the second eyeball posture data corresponding to the eyeball of the target patient.

[0077] Here, the first eyeball posture data is the eyeball posture data obtained by performing main axis rotation calculation on multiple sub-window images; the second eyeball posture data is the eyeball posture data obtained by performing ellipse fitting calculation on the iris contour image; wherein the eyeball posture data includes the center point position coordinates and rotation angle value corresponding to the target patient's eyeball.

[0078] Further, the first eyeball posture data corresponding to the eyeball of the target patient is obtained through the following steps:

[0079] Step 1.1, decentralize the contour position data to obtain a contour point set matrix.

[0080] Step 1.2: Based on the contour point set matrix, a preset covariance matrix calculation formula is used to determine a plurality of eigenvectors corresponding to the contour point set matrix.

[0081] Step 1.3: Based on the number of eigenvalues ​​corresponding to the multiple eigenvectors, determine a eigenvector having a maximum number of eigenvalues ​​as a first eigenvector indicating the main axis direction corresponding to the eyeball of the target patient.

[0082] Step 1.4: Based on the first feature vector, a preset rotation angle calculation formula is used to determine a first rotation angle corresponding to the eyeball of the target patient.

[0083] Step 1.5: Determine a plurality of pixel point coordinates and a window center point coordinate corresponding to each of the plurality of sub-window images.

[0084] Step 1.6: Based on the first rotation angle, the coordinates of the multiple pixel points and the coordinates of the center point of the window, a preset center angle calculation formula is used to determine the first center position coordinates corresponding to the eyeball of the target patient.

[0085] Step 1.7: Determine the first rotation angle and the first center position coordinates as first eyeball posture data corresponding to the eyeball of the target patient.

[0086] Further, the second eyeball posture data corresponding to the eyeball of the target patient is obtained by the following steps:

[0087] Step 2.1, performing partial derivative convolution processing on the contour position data to obtain an elliptic curve relationship equation for fitting the eyeball of the target patient in the iris contour image.

[0088] Step 2.2: Based on the focal length parameter corresponding to the first camera device and the elliptic curve relationship, an elliptical focal length matrix is ​​established.

[0089] Step 2.3: Based on the elliptical focal length matrix, a preset diagonalization calculation formula is used to determine the second eigenvector corresponding to the eyeball of the target patient.

[0090] Step 2.4: Based on the eigenvalue corresponding to the second eigenvector and the set normal vector parameters, a preset normal vector calculation formula is used to determine the normal vector corresponding to the eyeball of the target patient.

[0091] Step 2.5: Based on the direction parameter corresponding to the normal vector, determine the rotation angle of the direction parameter relative to the iris contour image as a second rotation angle corresponding to the eyeball of the target patient.

[0092] Step 2.6: Calculate the coordinate mean of the iris contour image to obtain the second center position coordinates corresponding to the eyeball of the target patient.

[0093] Step 2.7: Determine the second rotation angle and the second center position coordinates as second eyeball posture data corresponding to the eyeball of the target patient.

[0094] Step S1204: determining eyeball posture position parameters corresponding to the eyeball image based on the contour position data and the eyeball posture data, as well as posture prediction data output by a preset detection model according to the eyeball image.

[0095] In this step, during the specific implementation, first, the eyeball image is input into a preset detection model to obtain the posture prediction data output by the detection model; then, the contour position data and eyeball posture data corresponding to the posture prediction data are determined; finally, by averaging the obtained contour position data and then averaging the obtained eyeball posture data, the calculated average result is determined as the eyeball posture position parameter corresponding to the eyeball image.

[0096] The needle insertion posture determination module 130 is used to determine the needle insertion posture position data corresponding to the needle insertion tool according to the needle insertion area image.

[0097] When the needle insertion posture determination module 130 is implemented specifically, first, the needle insertion posture angle value corresponding to the needle insertion tool is determined based on the angle data corresponding to the needle insertion area image corresponding to the needle insertion tool acquired by the second camera device; then, the tool position data corresponding to the needle insertion tool is determined with the eyeball of the target patient as the center point; finally, the needle insertion posture angle value and the tool position data are determined as the needle insertion posture position data corresponding to the needle insertion tool.

[0098] For details, please refer to Figure 3 , Figure 3 This is a schematic diagram of the operation of a needle insertion tool provided in an embodiment of the present application. Figure 3 As shown in Figure 3 The needle insertion area image corresponding to the needle insertion tool is captured by the second camera device; wherein the needle insertion area image includes the eyeball, puncture card and injection needle of the target patient.

[0099] Specifically, Figure 3 As shown in , the poking card and the injection needle are components of the needle insertion tool; wherein, the poking card is used to fix the injection needle, and the injection needle is used to perform surgery on the eyeball.

[0100] When the needle insertion posture determination module is used to determine the needle insertion posture position data corresponding to the needle insertion tool according to the needle insertion area image, the needle insertion posture determination module is used to:

[0101] Further, when the needle insertion posture determination module 130 is used to determine the needle insertion posture position data corresponding to the needle insertion tool according to the needle insertion area image, the needle insertion posture determination module 130 is used to:

[0102] Step S1301: Determine the needle insertion posture angle value corresponding to the needle insertion tool based on the angle data corresponding to the needle insertion area image.

[0103] In this step, during the specific implementation, first, when the eyeball of the target patient is in a non-rotating state, the target needle insertion area is divided in the needle insertion area image; then, the optical axis center angle corresponding to the target needle insertion area is determined as the needle insertion area center angle value corresponding to the needle insertion area image; thereafter, for the target needle insertion area, the preset inscribed angle relationship is used to determine the needle insertion area limit angle value corresponding to the needle insertion area image; finally, based on the needle insertion area center angle value and the needle insertion area limit angle value, the needle insertion posture angle value corresponding to the needle insertion tool is determined.

[0104] The needle insertion posture angle value corresponding to the needle insertion tool represents the relative insertion angle between the injection needle and the eyeball when the injection needle is inserted into the eyeball of the target patient.

[0105] Here, when the eyeball of the target patient does not rotate (i.e., the eyeball is not tilted), it can be assumed that the front of the eyeball is a perfect sphere, and the needle insertion posture angle value can be calculated by the inscribed angle theorem; wherein the inscribed angle theorem is applied to any arc within the circumference, indicating the relationship between the central angle and the inscribed angle corresponding to the arc.

[0106] Further, when the needle insertion posture determination module 130 is used to determine the needle insertion posture angle value corresponding to the needle insertion tool based on the angle data corresponding to the needle insertion area image, the needle insertion posture determination module 130 is used to:

[0107] Step S13011: according to the surgical requirements corresponding to the eye surgery, a target needle insertion area is divided in the needle insertion area image.

[0108] In this step, when the eyeball of the target patient is in a non-rotating state, the fundus area of ​​the eyeball of the target patient is determined as the target needle insertion area in the needle insertion area image in view of the surgical scope and surgical target site of the eyeball surgery.

[0109] Step S13012: determine the optical axis center angle corresponding to the target needle insertion area as the needle insertion area center angle value corresponding to the needle insertion area image.

[0110] In this step, during the specific implementation, first, determine the working area angle corresponding to the target needle insertion area; then, determine the optical axis center angle corresponding to the target needle insertion area based on the correspondence between the working area angle and the optical axis center angle; finally, determine the optical axis center angle as the needle insertion area center angle value corresponding to the needle insertion area image.

[0111] In one possible implementation of the present application, the working area angle corresponding to the target needle insertion area (fundus) is set to 60 degrees according to the corneal specification angle corresponding to the eyeball, so as to avoid the problem of possible insertion deviation of the injection needle;

[0112] Furthermore, the corresponding relationship between the working area angle and the optical axis center angle is that the working area angle is twice the optical axis center angle, that is, the center angle value of the needle insertion area (optical axis center angle) is 30 degrees.

[0113] Step S13013: for the target needle insertion area, determine the needle insertion area limit angle value corresponding to the needle insertion area image using the preset inscribed angle relationship.

[0114] It should be noted that the needle insertion area limit angle value includes the needle insertion area maximum angle value and the needle insertion area minimum angle value; wherein, the needle insertion area limit angle value defines the angle value of the nearest and farthest working areas that can access the target needle insertion area.

[0115] In this step, a circular plane is established with the center of the eyeball included in the target needle insertion area as the center of the circle and the cornea of ​​the eyeball as the diameter; within the circular plane, the maximum angle value and the minimum angle value of the needle insertion area corresponding to the needle insertion area image are calculated through the inscribed angle relationship corresponding to the inscribed angle theorem.

[0116] Step S13014: Determine the needle insertion posture angle value corresponding to the needle insertion tool based on the center angle value of the needle insertion area and the limit angle value of the needle insertion area.

[0117] In this step, the central angle value of the needle insertion area, the maximum angle value of the needle insertion area, and the minimum angle value of the needle insertion area are input into a preset posture angle calculation tool to determine the needle insertion posture angle value corresponding to the needle insertion tool.

[0118] Step S1302: Determine tool position data corresponding to the needle insertion tool based on the needle insertion area image.

[0119] In this step, during the specific implementation, a plane coordinate system is established in the needle insertion area image with the target patient's eyeball as the center point; then, the plane coordinate data corresponding to the needle insertion tool is determined; finally, based on the parameter matrix corresponding to the second camera device and the plane coordinate data, the tool position data corresponding to the needle insertion tool is determined.

[0120] Further, when the needle insertion posture determination module 130 is used to determine the tool position data corresponding to the needle insertion tool based on the needle insertion area image, the needle insertion posture determination module 130 is used to:

[0121] Step S13021: Taking the eyeball of the target patient as the center point and following preset horizontal and vertical axis directions, a plane coordinate system is established in the needle insertion area image.

[0122] In this step, according to the eyeball posture position data corresponding to the eyeball of the target patient, a plane coordinate system is established in the needle insertion area image according to preset horizontal and vertical axis directions with the eyeball of the target patient as the center point.

[0123] Furthermore, the plane coordinate system is calibrated; specifically, through the preset relationship between the second camera device and the end effector installed on the trolley robot, based on the eye posture position parameters corresponding to the target patient's eyeball, and according to the tool position data of the injection needle, it is determined whether the end effector (needle end) is on the plane coordinate system to align the surgical area.

[0124] Step S13022: Determine the plane coordinate data corresponding to the needle insertion tool in the plane coordinate system.

[0125] In this step, the puncture point corresponding to the injection needle included in the needle insertion tool in the target needle insertion area is determined; then, the position coordinates corresponding to the puncture point are determined in the plane coordinate system; and the position coordinates are determined as the plane coordinate data corresponding to the needle insertion tool.

[0126] Step S13023: Determine the tool position data corresponding to the needle insertion tool based on the parameter matrix corresponding to the second camera device and the plane coordinate data.

[0127] In this step, during the specific implementation, first, obtain the parameter matrix corresponding to the second camera device; then, use the parameter matrix to convert the plane coordinate data corresponding to the needle insertion tool, and convert the plane coordinate data into three-dimensional coordinate data; finally, determine the three-dimensional coordinate data as the tool position data corresponding to the needle insertion tool.

[0128] In one possible implementation of the present application, the parameter matrix corresponding to the second camera device includes a camera intrinsic parameter matrix and a camera extrinsic parameter matrix; wherein the camera extrinsic parameter matrix is ​​composed of a set rotation matrix and a translation vector position parameter, and the translation vector position parameter refers to the position parameter of the translation vector corresponding to the center point position coordinates in the eye posture position parameter in the plane coordinate system.

[0129] In this way, by assuming that the four-degree-of-freedom robot is perpendicular to the head direction of the target patient, the five-degree-of-freedom problem of the injection needle posture is solved; by controlling the movement of the trolley robot and the tool position data corresponding to the needle insertion tool, the three-dimensional coordinate data of the injection needle is calculated using the parameter matrix transformation method, so that the trolley robot can translate in the space of the operating room according to the tool position data of the injection needle.

[0130] Step S1303: determine the needle insertion posture angle value and the tool position data as the needle insertion posture position data corresponding to the needle insertion tool.

[0131] In this step, the determined needle insertion posture angle value and the tool position data representing the three-dimensional coordinate data corresponding to the needle insertion tool are determined as the needle insertion posture position data corresponding to the needle insertion tool.

[0132] The surgical needle insertion control module 140 is used to control the needle insertion tool to insert the needle into the eyeball of the target patient according to the needle insertion posture position data.

[0133] During the specific implementation of the surgical needle insertion control module 140, first, the height of the trolley robot is lowered to align the poking card and the injection needle; then, the protective cover of the surgical instrument is removed, and the docking procedure of the needle insertion tool is started; finally, the needle insertion tool is controlled to perform the surgical needle insertion operation on the eyeball of the target patient according to the needle insertion posture position data.

[0134] Here, the height adjustment mechanism of the trolley robot controls the trolley robot to adjust its height according to the position and angle of the poking card, ensuring that the needle insertion tool consisting of the poking card and the injection needle contacts the eyeball and is aligned with the target needle insertion area.

[0135] The control device for eye surgery needle insertion provided in the embodiment of the present application adjusts the posture position of the trolley robot for surgery according to the eye image corresponding to the eye of the target patient acquired by the first camera device, further determines the needle insertion posture position of the needle insertion tool according to the needle insertion area image corresponding to the needle insertion tool acquired by the second camera device, and controls the needle insertion tool to perform surgical needle insertion operation on the eye of the target patient, thereby minimizing the target approach movement in the preoperative stage and maintaining the maximum movement ability of the robot during the operation to accurately optimize the utilization rate of the robot, thereby improving the accuracy and efficiency of eye surgery needle insertion, and thereby improving the flexibility of eye surgery operations.

[0136] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned device embodiments and will not be repeated here.

[0137] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0139] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0140] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0141] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A control device for inserting a needle during eye surgery, characterized in that: Applied to a trolley robot; wherein the trolley robot is equipped with a first camera device, a second camera device and a needle insertion tool, and the control device includes: A first image acquisition module, used for controlling the first camera device to aim at the eyeball of a target patient, and acquiring an eyeball image corresponding to the eyeball of the target patient; When the first image acquisition module is used to control the first camera device to aim at the eyeball of the target patient, the first image acquisition module is used to: Controlling the first camera device to face the eyeball of the target patient, and acquiring a camera field of view image captured by the first camera device; Performing edge detection on the camera field of view image, and extracting eyeball contour data determined by edge detection in the camera field of view image; Determining whether the eyeball contour data meets the pre-calibrated eyeball model standard corresponding to the eyeball of the target patient; If the eyeball contour data meets the eyeball model standard, determining that the first camera device is aimed at the eyeball of the target patient; The first image acquisition module is also used for: If the eyeball contour data does not meet the eyeball model standard, based on the eyeball contour data, using a preset visual servo controller to obtain deviation data between the camera field of view image and the eyeball of the target patient; Based on the deviation data, adjusting the orientation of the first camera device using a preset feedback controller until the eyeball contour data meets the eyeball model standard; A second image acquisition module is used to adjust the posture position parameters of the trolley robot to correspond to the eyeball posture position parameters corresponding to the eyeball image, and obtain the needle insertion area image corresponding to the needle insertion tool collected by the second camera device; A needle insertion posture determination module, used to determine the needle insertion posture position data corresponding to the needle insertion tool according to the needle insertion area image; When the needle insertion posture determination module is used to determine the needle insertion posture position data corresponding to the needle insertion tool according to the needle insertion area image, the needle insertion posture determination module is used to: Based on the angle data corresponding to the needle insertion area image, determining the needle insertion posture angle value corresponding to the needle insertion tool; Based on the needle insertion area image, determining tool position data corresponding to the needle insertion tool; Determine the needle insertion posture angle value and the tool position data as the needle insertion posture position data corresponding to the needle insertion tool; When the needle insertion posture determination module is used to determine the needle insertion posture angle value corresponding to the needle insertion tool based on the angle data corresponding to the needle insertion area image, the needle insertion posture determination module is used to: According to the surgical requirements corresponding to the eye surgery, a target needle insertion area is divided in the needle insertion area image; Determine the optical axis center angle corresponding to the target needle insertion area as the needle insertion area center angle value corresponding to the needle insertion area image; For the target needle insertion area, a preset inscribed angle relationship is used to determine a needle insertion area limit angle value corresponding to the needle insertion area image; Determine the needle insertion posture angle value corresponding to the needle insertion tool based on the needle insertion area center angle value and the needle insertion area limit angle value; The surgical needle insertion control module is used to control the needle insertion tool to insert the needle into the eyeball of the target patient according to the needle insertion posture position data.

2. The control device according to claim 1, characterized in that: When the second image acquisition module is used to determine the eyeball posture position parameters corresponding to the eyeball image, the second image acquisition module is used to: Segmenting the eyeball image to obtain eyeball sub-images segmented from the eyeball image; Performing edge detection on the eyeball sub-image to determine contour position data corresponding to the eyeball of the target patient; Based on the contour position data, performing principal axis rotation calculation and ellipse fitting calculation on the eyeball sub-image respectively to obtain eyeball posture data corresponding to the eyeball of the target patient; The eyeball image is input into a preset detection model to obtain posture prediction data output by the detection model, and the eyeball posture position parameters corresponding to the eyeball image are determined based on the contour position data, the eyeball posture data and the posture prediction data.

3. The control device according to claim 1, characterized in that: When the needle insertion posture determination module is used to determine the tool position data corresponding to the needle insertion tool based on the needle insertion area image, the needle insertion posture determination module is used to: Taking the eyeball of the target patient as the center point and according to preset horizontal and vertical axis directions, a plane coordinate system is established in the needle insertion area image; Determining the plane coordinate data corresponding to the needle insertion tool in the plane coordinate system; Based on the parameter matrix corresponding to the second camera device and the plane coordinate data, the tool position data corresponding to the needle insertion tool is determined.

Citation Information

Patent Citations

  • Human-computer interaction camera shooting method, device and system and medium

    CN114637399A

  • OCT (Optical Coherence Tomography)-based vitreous body injection data processing method, robot, equipment and medium

    CN116549216A

  • Three-dimensional human eye image generation method and device, electronic equipment and storage medium

    CN116664394A