Control apparatus for needle insertion in eyeball surgery

Abstract

Provided is a control apparatus for needle insertion in an eyeball surgery. The control apparatus comprises: a first image acquisition module (110), configured to control a first camera device to align with an eyeball of a target patient, and acquire an eyeball image corresponding to the eyeball of the target patient; a second image acquisition module (120), configured to adjust a posture position parameter of a trolley robot to correspond to an eyeball posture position parameter corresponding to the eyeball image, and acquire a needle insertion area image corresponding to a needle insertion tool collected by a second camera device; a needle insertion posture determination module (130), configured to determine, based on the needle insertion area image, needle insertion posture position data corresponding to the needle insertion tool; and a surgical needle insertion control module (140), configured to control the needle insertion tool to perform needle insertion on the eyeball of the target patient according to the needle insertion posture position data. By means of the control apparatus, the accuracy, the efficiency, and the operation flexibility of needle insertion in an eyeball surgery are improved.

Description

Control device for eye surgery needle insertion

[0001] Cross-reference to related applications

[0002] The present disclosure claims priority to the Chinese patent application No. 2024110263647, filed on July 30, 2024, and entitled "Control device for eye surgery needle insertion", the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to the technical field of robot control, in particular to a control device for eye surgery needle insertion. BACKGROUND

[0004] The needle insertion operation in eye surgery is very important, which is directly related to the success of the surgery and the safety and normal recovery of the patient. For example, for eye surgery that requires precise drug injection, such as intravitreal injection, accurate needle insertion is the key to the success of the surgery. If the injection position deviates, the drug may not reach the intended treatment area, thereby affecting the therapeutic effect. In addition, inaccurate needle insertion may cause complications such as blood vessel rupture and bleeding in the eye, and even serious consequences such as vision loss.

[0005] Currently, the control device for eye surgery needle insertion generally controls the docking of the needle insertion by assuming that the trolley robot is in the best position, without paying attention to the preoperative preparation stage of the robot-assisted eye surgery and the positioning of the robot platform itself in the operating room, which reduces the accuracy and efficiency of the eye surgery needle insertion, and further reduces the flexibility of the eye surgery operation.

[0006] SUMMARY

[0007] Therefore, the present application provides a control device for eye surgery needle insertion, which adjusts the posture position of the trolley robot for surgery according to the eye image of the target patient's eye obtained by the first camera device, further determines the needle insertion posture position of the needle insertion tool according to the needle insertion area image of the needle insertion tool collected by the second camera device, and controls the needle insertion tool to perform the needle insertion operation on the target patient's eye, which minimizes the target proximity motion in the preoperative stage and maintains the maximum motion capability of the robot during the intraoperative process to accurately optimize the utilization rate of the robot, thereby improving the accuracy and efficiency of the eye surgery needle insertion, and further improving the flexibility of the eye surgery operation.

[0008] The present application provides a control device for eye surgery needle insertion, which is applied to a trolley robot; wherein the trolley robot is installed with a first camera device, a second camera device and a needle insertion tool, and the control device comprises:

[0009] The first image acquisition module is configured to control the first camera device to aim at the eyeball of the target patient and acquire an eyeball image corresponding to the eyeball of the target patient.

[0010] The second image acquisition module is configured to adjust the posture position parameters of the trolley robot to correspond to the eye posture position parameters corresponding to the eye image, and acquire the needle insertion area image corresponding to the needle insertion tool captured by the second camera device;

[0011] The needle insertion posture determination module is configured to determine the needle insertion posture position data corresponding to the needle insertion tool based on the needle insertion area image;

[0012] The surgical needle insertion control module is configured to control the needle insertion tool to insert a needle into the eyeball of the target patient according to the needle insertion posture and position data.

[0013] Optionally, when the first image acquisition module is configured to control the first camera device to aim at the target patient's eyeball, the first image acquisition module is configured to:

[0014] The first camera device is controlled to face the eyeball of the target patient, and the camera field of view image captured by the first camera device is obtained;

[0015] Edge detection is performed on the camera field-of-view image to extract the eyeball contour data determined by edge detection in the camera field-of-view image;

[0016] Determine whether the eyeball contour data conforms to the pre-calibrated eyeball model standard corresponding to the target patient's eyeball;

[0017] If the eyeball contour data conforms to the eyeball model standard, then the first camera device is determined to be aimed at the target patient's eyeball.

[0018] Optionally, the first image acquisition module is further configured to:

[0019] If the eyeball contour data does not conform to the eyeball model standard, then based on the eyeball contour data, the deviation data between the camera field of view image and the target patient's eyeball is obtained using a preset visual servo controller.

[0020] Based on the deviation data, the orientation of the first camera device is adjusted using a preset feedback controller until the eyeball contour data conforms to the eyeball model standard.

[0021] Optionally, when the second image acquisition module is configured to determine the eye pose position parameters corresponding to the eye image, the second image acquisition module is configured to:

[0022] The eyeball image is segmented to obtain eyeball sub-images segmented from the eyeball image;

[0023] Edge detection is performed on the sub-image of the eyeball to determine the contour position data corresponding to the eyeball of the target patient;

[0024] Based on the contour position data, principal axis rotation calculation and ellipse fitting calculation are performed on the eyeball sub-images to obtain the eyeball pose data corresponding to the target patient's eyeball.

[0025] Based on the contour position data and the eye pose data, as well as the pose prediction data output by the preset detection model based on the eye image, the eye pose position parameters corresponding to the eye image are determined.

[0026] Optionally, when the needle insertion posture determination module is configured to determine the needle insertion posture position data corresponding to the needle insertion tool based on the needle insertion area image, the needle insertion posture determination module is configured to:

[0027] Based on the angle data corresponding to the needle insertion area image, the needle insertion posture angle value corresponding to the needle insertion tool is determined;

[0028] Based on the image of the needle insertion area, determine the tool position data corresponding to the needle insertion tool;

[0029] The needle insertion posture angle value and the tool position data are determined as the needle insertion posture position data corresponding to the needle insertion tool.

[0030] Optionally, when the needle insertion posture determination module is configured to determine the needle insertion posture angle value corresponding to the needle insertion tool based on the angle data corresponding to the needle insertion area image, the needle insertion posture determination module is configured to:

[0031] Based on the surgical requirements corresponding to the eye surgery, the target needle insertion area is divided in the needle insertion area image;

[0032] The optical axis center angle corresponding to the target needle insertion area is determined as the needle insertion area center angle value corresponding to the needle insertion area image;

[0033] For the target needle insertion area, the limit angle value of the needle insertion area corresponding to the needle insertion area image is determined using a preset inscribed angle relationship;

[0034] Based on the center angle value of the needle insertion area and the limit angle value of the needle insertion area, the needle insertion posture angle value corresponding to the needle insertion tool is determined.

[0035] Optionally, when the needle insertion posture determination module is configured to determine the tool position data corresponding to the needle insertion tool based on the needle insertion area image, the needle insertion posture determination module is configured to:

[0036] With the target patient's eyeball as the center point, and according to the preset horizontal and vertical axis directions, a planar coordinate system is established in the needle insertion area image;

[0037] Determine the planar coordinate data corresponding to the needle insertion tool in the planar coordinate system;

[0038] Based on the parameter matrix corresponding to the second camera device and the planar coordinate data, the tool position data corresponding to the needle insertion tool is determined.

[0039] The ocular surgery needle insertion control device provided in this application embodiment is applied to a trolley robot. The trolley robot is equipped with a first camera device, a second camera device, and a needle insertion tool. The control device includes: a first image acquisition module configured to control the first camera device to aim at the eyeball of a target patient and acquire an eyeball image corresponding to the target patient's eyeball; a second image acquisition module configured to adjust the posture position parameters of the trolley robot to correspond to the eyeball posture position parameters corresponding to the eyeball image, and acquire an insertion area image corresponding to the needle insertion tool collected by the second camera device; a needle insertion posture determination module configured to determine the needle insertion posture position data corresponding to the needle insertion tool based on the insertion area image; and a surgical needle insertion control module configured to control the needle insertion tool to insert a needle into the target patient's eyeball according to the needle insertion posture position data.

[0040] Compared to existing technologies that control needle insertion by assuming the trolley robot is in an optimal position, this method adjusts the surgical posture of the trolley robot based on the eye image of the target patient acquired by the first camera device. Furthermore, it determines the needle insertion posture of the needle insertion tool based on the needle insertion area image acquired by the second camera device, and controls the needle insertion tool to perform the surgical needle insertion operation on the target patient's eye. This method minimizes the target approach motion in the preoperative stage and maintains the robot's maximum motion capability during the operation, thereby precisely optimizing the robot's utilization rate, improving the accuracy and efficiency of needle insertion in eye surgery, and thus enhancing the flexibility of eye surgery operations.

[0041] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 is a schematic diagram of the control device for needle insertion in ocular surgery provided in an embodiment of this application;

[0044] Figure 2 is a schematic diagram of the ocular surgery device based on a trolley robot provided in an embodiment of this application;

[0045] Figure 3 is a schematic diagram of the operation of the needle insertion tool provided in the embodiment of this application.

[0046] Icons: 100-Control device; 110-First image acquisition module; 120-Second image acquisition module; 130-Needle insertion posture determination module; 140-Surgical needle insertion control module; 1-RGBD camera; 2-Support; 3-RGB camera; 4-Five-DOF robot; 5-Operating table; 6-Four-DOF robot; 7-Analytical instrument; 8-Graphical terminal; 9-Target patient's eyeball; 10-Stamp card; 11-Injection needle. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0048] Research has found that current control devices for needle insertion in ophthalmic surgery generally control needle insertion by assuming the trolley robot is in the optimal position, without paying attention to the preoperative preparation stage of robot-assisted ophthalmic surgery and the positioning of the robot platform itself in the operating room. This reduces the accuracy and efficiency of needle insertion in ophthalmic surgery, and consequently reduces the flexibility of ophthalmic surgical operations.

[0049] Based on this, this application provides a control device for needle insertion in ocular surgery. By adjusting the posture and position of the trolley robot for surgery based on the eye image corresponding to the target patient's eye acquired by a first camera device, and further determining the needle insertion posture and position of the needle insertion tool based on the needle insertion area image corresponding to the needle insertion tool acquired by a second camera device, the device controls the needle insertion tool to perform surgical needle insertion operations on the target patient's eye. This achieves precise optimization of robot utilization by minimizing target approach motion in the preoperative stage and maintaining the robot's maximum motion capability during the operation, thereby improving the accuracy and efficiency of needle insertion in ocular surgery and thus enhancing the flexibility of ocular surgical operations.

[0050] Please refer to Figure 1, which is a schematic diagram of the structure of a control device for needle insertion in ocular surgery provided in an embodiment of this application. As shown in Figure 1, the control device 100 is applied to a trolley robot; wherein, the trolley robot is equipped with a first camera device, a second camera device, and a needle insertion tool, and the control device 100 includes:

[0051] The first image acquisition module 110 is configured to control the first camera device to aim at the eyeball of the target patient and acquire an eyeball image corresponding to the eyeball of the target patient.

[0052] It should be noted that when performing eye surgery on the target patient, an eye surgery device based on a trolley robot must be used for the surgical operation; the trolley robot is equipped with a first camera device, a second camera device, and a needle insertion tool.

[0053] Optionally, the first camera device may include a depth color camera (RGBD camera); the second camera device may include a color camera (RGB camera); and the needle insertion tool may include a puncture card and an injection needle.

[0054] Specifically, please refer to Figure 2, which is a schematic diagram of an ocular surgery device based on a trolley robot provided in this application embodiment. As shown in Figure 2, a trolley robot, including a four-degree-of-freedom robot and a five-degree-of-freedom robot, is set on the operating table where the target patient is located. The robot's degrees of freedom are the number of joints that the robot can move independently. A second camera device (RGB camera) is installed on the support set up with the four-degree-of-freedom robot and the operating table. A first camera device (RGBD camera) is installed on the support set up with the five-degree-of-freedom robot and the operating table. In addition, the operating table is connected to surgical analysis instruments and a graphical terminal.

[0055] In a specific implementation, the first image acquisition module 110, with the trolley robot perpendicular to the head direction of the target patient, positions the trolley robot close to the eyes of the target patient, controls the first camera device to face the eyeballs of the target patient, so as to completely capture the eyeballs of the target patient; controls the trolley robot to perform translational movement, so as to adjust the position and height of the trolley robot to a predetermined range; and acquires the eyeball image corresponding to the eyeballs of the target patient.

[0056] In one embodiment of this application, when the first image acquisition module is configured to control the first camera device to aim at the eyeball of the target patient, the first image acquisition module is configured to:

[0057] Step S1101: Control the first camera device to face the eyeball of the target patient and acquire the camera field of view image collected by the first camera device.

[0058] In this step, the movement of the trolley robot is adjusted according to the preset position and direction parameters, and the first camera device is controlled to face the target patient's eyeball until the field of view of the first camera device can completely cover the entire eyeball of the target patient; the camera field of view image captured by the first camera device at this time is obtained.

[0059] Step S1102: Perform edge detection on the camera field of view image and extract the eyeball contour data determined by edge detection in the camera field of view image.

[0060] In this step, a preset computer vision algorithm is used to perform edge detection on the camera's field of view image, and the eyeball contour data determined by the edge detection is extracted from the camera's field of view image.

[0061] Step S1103: Determine whether the eyeball contour data conforms to the pre-calibrated eyeball model standard corresponding to the target patient's eyeball.

[0062] In this step, in specific implementation, firstly, the eyeball contour data is evaluated to determine whether it conforms to the standard eyeball shape and size, in order to determine whether there are any problems with the image captured by the first camera device; then, if it conforms to the standard eyeball shape and size, a preset matching algorithm is used to map the 2D image corresponding to the captured eyeball contour data onto the pre-calibrated eyeball model corresponding to the target patient's eyeball; finally, it is determined whether the eyeball contour data conforms to the standard eyeball model corresponding to the target patient's eyeball.

[0063] Step S1104: If the eyeball contour data conforms to the eyeball model standard, then determine that the first camera device is aligned with the target patient's eyeball.

[0064] In this step, if the eyeball contour data conforms to the eyeball model standard, it indicates that the camera field of view image acquired by the first camera device at this moment contains the entire eyeball of the target patient, and it is determined that the first camera device is aimed at the eyeball of the target patient.

[0065] Optionally, if the eyeball contour data does not conform to the eyeball model standard, the first image acquisition module 110 is further configured to:

[0066] Step S1105: Based on the eyeball contour data, obtain the deviation data between the camera field of view image and the target patient's eyeball using a preset visual servo controller.

[0067] In this step, if the eyeball contour data does not conform to the eyeball model standard, the visual servo control algorithm set in the preset visual servo controller is used to calculate the deviation data between the camera field of view image and the eyeball of the target patient based on the eyeball contour data and the camera field of view image acquired by the first camera device.

[0068] Step S1106: Based on the deviation data, adjust the orientation of the first camera device using a preset feedback controller until the eyeball contour data conforms to the eyeball model standard.

[0069] In this step, based on the deviation data, the speed and direction of the trolley robot are adjusted using the feedback control algorithm set in the preset feedback controller, so that the trolley robot gradually approaches the position of the target patient to adjust the orientation of the first camera device; through iterative calculation of the deviation data, until the eyeball contour data meets the eyeball model standard; after meeting the standard, the adjustment stops.

[0070] Optionally, the feedback control algorithm includes, but is not limited to, PID filtering algorithm and Kalman filtering algorithm.

[0071] The second image acquisition module 120 is configured to adjust the posture position parameters of the trolley robot to correspond to the eye posture position parameters corresponding to the eye image, and acquire the needle insertion area image corresponding to the needle insertion tool collected by the second camera device.

[0072] In specific implementation, the second image acquisition module 120 firstly fine-tunes the posture position parameters of the trolley robot based on predefined position and posture constraints, using sensor data or visual feedback from the trolley robot to ensure that the trolley robot reaches the desired position and posture; then, it determines the eye posture position parameters corresponding to the eye image; subsequently, it adjusts the posture position parameters of the trolley robot to correspond to the eye posture position parameters corresponding to the eye image; finally, it acquires the needle insertion area image corresponding to the needle insertion tool collected by the second camera device.

[0073] Here, after the first camera device captures the complete eyeball, due to the angle of the camera device at this time, or because the center point of the target patient's eyeball is too far away from the center point of the image captured by the first camera device, the focus of the camera device is out of focus. Therefore, the posture and position parameters of the trolley robot are slightly adjusted to overcome the above problems.

[0074] Optionally, when the second image acquisition module 120 is configured to determine the eyeball pose position parameters corresponding to the eyeball image, the second image acquisition module 120 is configured to:

[0075] Step S1201: Segment the eyeball image to obtain the segmented eyeball sub-image.

[0076] In this step, in specific implementation, firstly, the eyeball image is segmented into image windows to obtain multiple sub-window images corresponding to the eyeball image; then, the eyeball image is segmented into iris contours using a preset edge detector to obtain iris contour images corresponding to the eyeball image; finally, the multiple sub-window images and the iris contour images are determined as the eyeball sub-image.

[0077] Step S1202: Perform edge detection on the eyeball sub-image to determine the contour position data corresponding to the target patient's eyeball.

[0078] In this step, the eyeball sub-image is edge-detected using a preset Canny edge detector, and the contour position data corresponding to the target patient's eyeball is determined based on the edge detection results.

[0079] Step S1203: Based on the contour position data, perform principal axis rotation calculation and ellipse fitting calculation on the eyeball sub-image to obtain the eyeball pose data corresponding to the target patient's eyeball.

[0080] In this step, in specific implementation, firstly, based on the contour position data, the principal axis rotation calculation is performed on the multiple sub-window images in the eyeball sub-image to obtain the first eyeball pose data corresponding to the target patient's eyeball; then, based on the contour position data, the ellipse fitting calculation is performed on the iris contour image in the eyeball sub-image to obtain the second eyeball pose data corresponding to the target patient's eyeball.

[0081] Here, the first eye pose data is eye pose data obtained by performing principal axis rotation calculation on multiple sub-window images; the second eye pose data is eye pose data obtained by performing ellipse fitting calculation on the iris contour image; wherein, the eye pose data includes the coordinates of the center point position and the rotation angle value corresponding to the target patient's eyeball.

[0082] Optionally, the first eyeball pose data corresponding to the target patient's eyeball is obtained through the following steps:

[0083] Step 1.1: Decentralize the contour position data to obtain a contour point set matrix.

[0084] Step 1.2: Based on the contour point set matrix, determine multiple feature vectors corresponding to the contour point set matrix using a preset covariance matrix calculation formula.

[0085] Step 1.3: Based on the number of feature values ​​corresponding to the multiple feature vectors, the feature vector whose number of feature values ​​is the maximum value is determined as the first feature vector indicating the principal axis direction of the target patient's eyeball.

[0086] Step 1.4: Based on the first feature vector, determine the first rotation angle corresponding to the eyeball of the target patient using a preset rotation angle calculation formula.

[0087] Step 1.5: Determine the coordinates of multiple pixels and the center point of the window for each of the multiple sub-window images.

[0088] Step 1.6: Based on the first rotation angle, the coordinates of the multiple pixels and the coordinates of the window center point, determine the first center position coordinates of the target patient's eyeball using a preset center angle calculation formula.

[0089] Step 1.7: Determine the first rotation angle and the first center position coordinates as the first eyeball posture data corresponding to the target patient's eyeball.

[0090] Optionally, the second eyeball pose data corresponding to the target patient's eyeball is obtained through the following steps:

[0091] Step 2.1: Perform partial derivative convolution processing on the contour position data to obtain the elliptic curve relationship fitted to the iris contour image of the target patient's eyeball.

[0092] Step 2.2: Based on the focal length parameters corresponding to the first camera device and the elliptic curve relationship, establish an elliptic focal length matrix.

[0093] Step 2.3: Based on the elliptical focal length matrix, determine the second feature vector corresponding to the eyeball of the target patient using a preset diagonalization calculation formula.

[0094] Step 2.4: Based on the eigenvalues ​​corresponding to the second feature vector and the set normal vector parameters, determine the normal vector corresponding to the eyeball of the target patient using a preset normal vector calculation formula.

[0095] Step 2.5: Based on the direction parameter corresponding to the normal vector, determine the rotation angle of the direction parameter relative to the iris contour image as the second rotation angle corresponding to the eyeball of the target patient.

[0096] Step 2.6: Calculate the coordinate mean of the iris contour image to obtain the coordinates of the second center position of the target patient's eyeball.

[0097] Step 2.7: Determine the second rotation angle and the second center position coordinates as the second eyeball pose data corresponding to the target patient's eyeball.

[0098] Step S1204: Based on the contour position data and the eye pose data, and the pose prediction data output by the preset detection model according to the eye image, determine the eye pose position parameters corresponding to the eye image.

[0099] In this step, in specific implementation, firstly, the eye image is input into a preset detection model to obtain the pose prediction data output by the detection model; then, the corresponding contour position data and eye pose data in the pose prediction data are determined; finally, by averaging the obtained contour position data and then averaging the obtained eye pose data, the calculated mean result is determined as the eye pose position parameter corresponding to the eye image.

[0100] The needle insertion posture determination module 130 is configured to determine the needle insertion posture position data corresponding to the needle insertion tool based on the needle insertion area image.

[0101] In specific implementation, the needle insertion posture determination module 130 first determines the needle insertion posture angle value corresponding to the needle insertion tool based on the angle data corresponding to the needle insertion area image acquired by the second camera device; then, with the eyeball of the target patient as the center point, it determines the tool position data corresponding to the needle insertion tool; finally, it determines the needle insertion posture angle value and the tool position data as the needle insertion posture position data corresponding to the needle insertion tool.

[0102] Specifically, please refer to Figure 3, which is a schematic diagram of the operation of a needle insertion tool provided in an embodiment of this application. As shown in Figure 3, Figure 3 is an image of the needle insertion area corresponding to the needle insertion tool captured by the second camera device; wherein, the needle insertion area image includes the target patient's eyeball, puncture card, and injection needle.

[0103] Specifically, as shown in Figure 3, the puncture card and the injection needle are components of the needle insertion tool; the puncture card is configured to fix the injection needle, and the injection needle is configured to perform surgery on the eyeball.

[0104] When the needle insertion posture determination module is configured to determine the needle insertion posture position data corresponding to the needle insertion tool based on the needle insertion area image, the needle insertion posture determination module is configured as follows:

[0105] Optionally, when the needle insertion posture determination module 130 is configured to determine the needle insertion posture position data corresponding to the needle insertion tool based on the needle insertion area image, the needle insertion posture determination module 130 is configured to:

[0106] Step S1301: Based on the angle data corresponding to the needle insertion area image, determine the needle insertion posture angle value corresponding to the needle insertion tool.

[0107] In this step, in specific implementation, firstly, with the target patient's eyeball in a non-rotating state, the target needle insertion area is delineated in the needle insertion area image; then, the optical axis center angle corresponding to the target needle insertion area is determined as the needle insertion area center angle value corresponding to the needle insertion area image; next, for the target needle insertion area, the needle insertion area limit angle value corresponding to the needle insertion area image is determined using a preset inscribed angle relationship; finally, based on the needle insertion area center angle value and the needle insertion area limit angle value, the needle insertion posture angle value corresponding to the needle insertion tool is determined.

[0108] The needle insertion angle value corresponding to the needle insertion tool represents the relative insertion angle between the injection needle and the eyeball when the injection needle is inserted into the eyeball of the target patient.

[0109] Here, when the patient's eyeball is not rotating (i.e., the eyeball is not tilted), it can be assumed that the front part of the eyeball is a perfect sphere, and the needle insertion posture angle value can be calculated by the inscribed angle theorem; where the inscribed angle theorem should be configured as any arc in the circumference, representing the relationship between the central angle and the inscribed angle corresponding to the arc.

[0110] Optionally, when the needle insertion posture determination module 130 is configured to determine the needle insertion posture angle value corresponding to the needle insertion tool based on the angle data corresponding to the needle insertion area image, the needle insertion posture determination module 130 is configured to:

[0111] Step S13011: Based on the surgical requirements corresponding to the eye surgery, the target needle insertion area is divided in the needle insertion area image.

[0112] In this step, with the target patient's eyeball in a non-rotating state, the fundus region of the target patient's eyeball is identified as the target needle insertion area in the needle insertion area image, based on the surgical scope and target site of the eyeball surgery.

[0113] Step S13012: Determine the optical axis center angle corresponding to the target needle insertion area as the needle insertion area center angle value corresponding to the needle insertion area image.

[0114] In this step, in specific implementation, firstly, the working area angle corresponding to the target needle insertion area is determined; then, based on the correspondence between the working area angle and the optical axis center angle, the optical axis center angle corresponding to the target needle insertion area is determined; finally, the optical axis center angle is determined as the needle insertion area center angle value corresponding to the needle insertion area image.

[0115] Optionally, based on the corneal specifications of the eyeball, the working area angle corresponding to the target needle insertion area (fundus) is set to 60 degrees to avoid the problem of possible insertion deviation of the injection needle;

[0116] Optionally, the working area angle and the optical axis center angle correspond to the working area angle being twice the optical axis center angle, that is, the center angle value of the needle insertion area (optical axis center angle) is 30 degrees.

[0117] Step S13013: For the target needle insertion area, determine the needle insertion area limit angle value corresponding to the needle insertion area image using a preset inscribed angle relationship.

[0118] It should be noted that the needle insertion area limit angle value includes the maximum angle value and the minimum angle value of the needle insertion area; among them, the needle insertion area limit angle value defines the angle value of the nearest and farthest working areas of the accessible target needle insertion area.

[0119] In this step, a circular plane is established with the center of the eyeball included in the target needle insertion area as the center and the cornea of ​​the eyeball as the diameter. Within this circular plane, the maximum and minimum angle values ​​of the needle insertion area corresponding to the needle insertion area image are calculated using the inscribed angle theorem.

[0120] Step S13014: Based on the center angle value of the needle insertion area and the limit angle value of the needle insertion area, determine the needle insertion posture angle value corresponding to the needle insertion tool.

[0121] In this step, the center angle value, maximum angle value, and minimum angle value of the needle insertion area are input into the preset posture angle calculation tool to determine the needle insertion posture angle value corresponding to the needle insertion tool.

[0122] Step S1302: Based on the needle insertion area image, determine the tool position data corresponding to the needle insertion tool.

[0123] In this step, a planar coordinate system is established in the needle insertion area image with the target patient's eyeball as the center point; then, the planar coordinate data corresponding to the needle insertion tool is determined; finally, based on the parameter matrix corresponding to the second camera device and the planar coordinate data, the tool position data corresponding to the needle insertion tool is determined.

[0124] Optionally, when the needle insertion posture determination module 130 is configured to determine the tool position data corresponding to the needle insertion tool based on the needle insertion area image, the needle insertion posture determination module 130 is configured to:

[0125] Step S13021: Using the eyeball of the target patient as the center point, and according to the preset horizontal and vertical axis directions, establish a planar coordinate system in the needle insertion area image.

[0126] In this step, based on the eyeball posture and position data corresponding to the target patient's eyeball, a planar coordinate system is established in the needle insertion area image with the target patient's eyeball as the center point and according to the preset horizontal and vertical axis directions.

[0127] Optionally, the planar coordinate system is calibrated; specifically, based on the preset relationship between the second camera device and the end effector installed on the trolley robot, and according to the eyeball posture position parameters corresponding to the target patient's eyeball, the end effector (needle tip) is determined to be on the planar coordinate system to align the surgical area.

[0128] Step S13022: Determine the plane coordinate data corresponding to the needle insertion tool in the plane coordinate system.

[0129] In this step, the puncture point corresponding to the injection needle included in the needle insertion tool in the target needle insertion area is determined; then, the position coordinates corresponding to the puncture point are determined in the plane coordinate system; and the position coordinates are determined as the plane coordinate data corresponding to the needle insertion tool.

[0130] Step S13023: Based on the parameter matrix corresponding to the second camera device and the planar coordinate data, determine the tool position data corresponding to the needle insertion tool.

[0131] In this step, in specific implementation, firstly, the parameter matrix corresponding to the second camera device is obtained; then, the planar coordinate data corresponding to the needle insertion tool is transformed using the parameter matrix, and the planar coordinate data is transformed into three-dimensional coordinate data; finally, the three-dimensional coordinate data is determined as the tool position data corresponding to the needle insertion tool.

[0132] Optionally, the parameter matrix corresponding to the second camera device includes a camera intrinsic parameter matrix and a camera extrinsic parameter matrix; wherein, the camera extrinsic parameter matrix consists of a set rotation matrix and translation vector position parameters, and the translation vector position parameters refer to the position parameters of the translation vector corresponding to the center point position coordinates in the eye pose position parameters in the planar coordinate system.

[0133] In this way, by assuming that the four-degree-of-freedom robot is perpendicular to the head direction of the target patient, the problem of the five-degree-of-freedom posture of the injection needle is solved; by controlling the movement of the control cart robot and the tool position data corresponding to the needle insertion tool, the three-dimensional coordinate data of the injection needle is calculated using the parameter matrix transformation method, so as to control the control cart robot to translate in the space of the operating room according to the tool position data of the injection needle.

[0134] Step S1303: Determine the needle insertion posture angle value and the tool position data as the needle insertion posture position data corresponding to the needle insertion tool.

[0135] In this step, the determined needle insertion posture angle value and the tool position data representing the corresponding three-dimensional coordinate data of the needle insertion tool are determined as the needle insertion posture position data corresponding to the needle insertion tool.

[0136] The surgical needle insertion control module 140 is configured to control the needle insertion tool to insert a needle into the eyeball of the target patient according to the needle insertion posture and position data.

[0137] In specific implementation, the surgical needle insertion control module 140 first lowers the height of the trolley robot to align the puncture card and injection needle; then, it removes the protective cover of the surgical instrument and starts the docking program of the needle insertion tool; finally, it controls the needle insertion tool to perform surgical needle insertion operation on the target patient's eyeball according to the needle insertion posture and position data.

[0138] Here, the height adjustment mechanism of the trolley robot controls the trolley robot to adjust its height according to the position and angle of the puncture card, ensuring that the injection tool composed of the puncture card and the injection needle contacts the eyeball and is aligned with the target injection area.

[0139] The ocular surgery needle insertion control device provided in this application adjusts the surgical posture and position of the trolley robot based on the eye image corresponding to the target patient's eye acquired by the first camera device. Furthermore, it determines the needle insertion posture and position of the needle insertion tool based on the needle insertion area image corresponding to the needle insertion tool acquired by the second camera device, and controls the needle insertion tool to perform surgical needle insertion on the target patient's eye. This achieves precise optimization of robot utilization by minimizing target approach motion in the preoperative stage and maintaining maximum robot mobility during the surgical process, thereby improving the accuracy and efficiency of ocular surgery needle insertion and ultimately enhancing the flexibility of ocular surgery operations.

[0140] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing device embodiments, and will not be repeated here.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0144] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0145] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims. Industrial applicability

[0146] In summary, the embodiments of this application provide a control device for needle insertion in ocular surgery, which improves the accuracy and efficiency of needle insertion in ocular surgery, thereby increasing the flexibility of ocular surgical operations.

Claims

1. A control device for eye surgery needle insertion, characterized by, The application is applied to a trolley robot, wherein the trolley robot is installed with a first camera device, a second camera device and a needle insertion tool, and the control device comprises: a first image acquisition module configured to control the first camera device to align with an eyeball of a target patient and acquire an eyeball image corresponding to the eyeball of the target patient; a second image acquisition module configured to adjust a pose position parameter of the trolley robot to correspond to an eyeball pose position parameter corresponding to the eyeball image and acquire an eyeball region image corresponding to the needle insertion tool collected by the second camera device; a needle insertion pose determination module configured to determine needle insertion pose position data corresponding to the needle insertion tool according to the eyeball region image; a surgical needle insertion control module configured to control the needle insertion tool to perform needle insertion on the eyeball of the target patient according to the needle insertion pose position data.

2. The control device according to claim 1, characterized by When the first image acquisition module is configured to control the first camera device to align with the eyeball of the target patient, the first image acquisition module is configured to: control the first camera device to face the eyeball of the target patient and acquire a camera field of view image collected by the first camera device; perform edge detection on the camera field of view image and extract eyeball contour data determined by the edge detection in the camera field of view image; determine whether the eyeball contour data meets eyeball model standards corresponding to the eyeball of the target patient pre-labeled; if the eyeball contour data meets the eyeball model standards, it is determined that the first camera device aligns with the eyeball of the target patient.

3. The control device of claim 2, wherein The first image acquisition module is further configured to: if the eyeball contour data does not meet the eyeball model standards, obtain deviation data between the camera field of view image and the eyeball of the target patient by using a preset visual servo controller based on the eyeball contour data; based on the deviation data, adjust the orientation of the first camera device by using a preset feedback controller until the eyeball contour data meets the eyeball model standards.

4. The control device according to any one of claims 1 to 3, characterized by When the second image acquisition module is configured to determine the eyeball pose position parameter corresponding to the eyeball image, the second image acquisition module is configured to: segment the eyeball image to obtain an eyeball sub-image segmented from the eyeball image; perform edge detection on the eyeball sub-image to determine contour position data corresponding to the eyeball of the target patient; based on the contour position data, perform principal axis rotation calculation and ellipse fitting calculation on the eyeball sub-image respectively to obtain eyeball pose data corresponding to the eyeball of the target patient; based on the contour position data and the eyeball pose data and pose prediction data output by a preset detection model according to the eyeball image, determine the eyeball pose position parameter corresponding to the eyeball image.

5. The control device according to any one of claims 1 to 4, characterized by When the needle insertion pose determination module is configured to determine the needle insertion pose position data corresponding to the needle insertion tool according to the eyeball region image, the needle insertion pose determination module is configured to: determine a needle insertion pose angle value corresponding to the needle insertion tool based on angle data corresponding to the eyeball region image; determine tool position data corresponding to the needle insertion tool based on the eyeball region image. The needle insertion pose angle value and the tool position data are determined as the needle insertion tool corresponding needle insertion pose position data.

6. The control device of claim 5, wherein When the needle insertion pose determination module is configured to determine the needle insertion tool corresponding needle insertion pose angle value based on the needle insertion region image corresponding angle data, the needle insertion pose determination module is configured to: According to the surgical requirements corresponding to the eye surgery, a target needle insertion region is divided in the needle insertion region image; The optical axis center angle corresponding to the target needle insertion region is determined as the needle insertion region image corresponding needle insertion region center angle value; For the target needle insertion region, the preset inscribed angle relationship is used to determine the needle insertion region image corresponding needle insertion region limit angle value; Based on the needle insertion region center angle value and the needle insertion region limit angle value, the needle insertion tool corresponding needle insertion pose angle value is determined.

7. The control device of claim 5, wherein When the needle insertion pose determination module is configured to determine the needle insertion tool corresponding tool position data based on the needle insertion region image, the needle insertion pose determination module is configured to: Taking the eyeball of the target patient as a center point, and according to a preset horizontal axis direction and a vertical axis direction, a plane coordinate system is established in the needle insertion region image; The plane coordinate data corresponding to the needle insertion tool is determined in the plane coordinate system; Based on the parameter matrix corresponding to the second camera device and the plane coordinate data, the tool position data corresponding to the needle insertion tool is determined.

Images