OCT-based vitreous injection data processing method, robot, device and medium

Abstract

The present application relates to an OCT-based vitreous injection data processing method and robot, device and medium, the vitreous injection robot comprising an OCT module and a vitreous injection robot module, the OCT module is used for image guidance, provides binocular stereoscopic microscopic image and OCT image of scanning area, the vitreous injection robot module includes mechanical arm and end effector, and the vitreous injection robot is used for ocular surface puncture and intraocular drug injection.The present application improves the puncture accuracy and injection accuracy of vitreous injection, reduces iatrogenic injury and subsequent injury caused by tissue-needle tip space relative position uncertainty.The introduction of OCT technology can not only assist binocular camera to carry out more accurate limbus positioning, but also can realize real-time imaging and tracking to intraocular tissue, lesion and needle tip, provide feedback for puncture and drug injection, realize automatic injection of drug at predetermined position and fixed angle without causing injury.

Description

Technical Field

[0001] This invention relates to the field of optical coherence tomography (OCT), and particularly to a method, robot, equipment, and medium for processing vitreous injection data based on OCT. Background Technology

[0002] Optical coherence tomography (OCT) is a high-resolution, high-sensitivity, non-contact three-dimensional imaging method. It performs tomographic imaging of biological tissues by detecting the phase delay and intensity of the backscattered echoes, and is often referred to as "optical biopsy." OCT boasts an axial resolution of 1-15 micrometers, far exceeding traditional imaging methods such as ultrasound, MRI, and X-rays, and has been widely applied in dermatology, dentistry, respiratory medicine, gastroenterology, and oncology. In ophthalmology, OCT can provide real-time imaging of intraocular tissues, lesions, and surgical instruments, making it particularly suitable for preoperative path planning, intraoperative real-time navigation, and postoperative outcome evaluation. Along with surgical microscopes, it is considered the "gold standard" of ophthalmic surgery.

[0003] Due to the non-regenerative nature of retinal and corneal cells, once a patient suffers from an irreversible blinding disease, they inevitably undergo four treatment stages: ocular surface medication, laser cryotherapy, surgery, and intravitreal injection of small molecule inhibitors. Especially for mid-to-late-stage optometric blindness, surgery combined with intravitreal injection is the preferred, and often the only, definitively effective, treatment. The advantages of intravitreal injection include low dosage, minimal systemic impact from local application, and direct action. The types of diseases treated and the number of cases treated are rapidly increasing. The first intravitreal injection occurred in 1911, when a doctor injected air into the vitreous humor to repair a detached retina. Subsequently, with the advent of antibiotics, antiviral drugs, antifungal drugs, and anti-VEGF drugs, intravitreal injection began to be widely used in the treatment and research of various ophthalmic diseases, including age-related macular degeneration, retinal vein occlusion macular edema, diabetic macular edema, and endophthalmitis. During vitreous injection, the needle needs to enter the inferior temporal quadrant of the eyeball at a 45-60° angle within a circular area 3-4 mm from the limbus, and inject the medication to a depth of 4-6 mm. This requires extremely high injection precision. Therefore, to improve surgical accuracy and safety and reduce surgical time, the introduction of vitreous injection robots is imperative.

[0004] In ophthalmic surgery, doctors, with the help of assistants, puncture the sclera and inject medication into the vitreous humor to complete the treatment, with each injection taking only a few minutes. However, with the increase in the number of patients, manual injection is difficult to consistently meet the precision and stability requirements of vitreous injection. In contrast, vitreous injection robots have advantages such as high precision, fine operation, and micro-force sensing, which can reduce the burden on doctors, perform faster and more stable vitreous injections, and reduce secondary damage during surgery. Edwards et al. first demonstrated the clinical feasibility of robot-assisted ophthalmic surgery in their article (First-in-human study of the safety and viability of intraocular robotic surgery, Nat Biomed Eng, 2:649-656, (2018)). Compared with traditional manual surgery, the PRECEYES ophthalmic surgery robot compensates for the surgeon's hand tremors while peeling away the retina / inner limiting membrane. Through the control system, it provides smooth, tremor-free, and precise position control and force scaling, while meeting the accuracy and safety requirements of ophthalmic surgery, and significantly reducing secondary damage caused by the surgery. In their article (Assistive device for efficient intravitreal injections, Ophthalmic Surgery, Lasers and Imaging Retina, vol. 47, no. 8, pp. 752-762, (2016)), Franziska et al. demonstrated a vitreous injection-assisted robot. After the doctor roughly positions the patient's head, the robot can accurately identify the limbus, automatically locate the injection area, and complete the injection at the specified depth. In an in vitro vitreous injection experiment in pig eyes, the system precisely injected the drug into a safe injection area without causing damage to the lens or retina. Yunming et al. proposed a four-degree-of-freedom automated intravitreal injection robot in their paper (Design and analysis of arobot for automated intravitreal injection, IEEE International Conference on Robotics and Biomimetics, pp. 1849-1854, (2022)). This robot can autonomously control the needle to tilt, press, puncture, and inject drugs. It employs a bilinear drive mechanism to achieve needle rotation around the injection port, and incorporates easily replaceable grippers to suppress involuntary eye movements. In preliminary tests on pig eyes, the maximum motion error was less than 0.03 mm, meeting the precision requirements for intravitreal injection.However, current vitreous injection robots rely solely on binocular microscopic images of the eye's surface for navigation, failing to acquire the intraocular needle tip pose and injection location. This lack of depth information and uncertainty regarding the relative position of the tissue and needle tip pose significant risks. If the injection site is too close to the limbus, the needle tip may pierce the lens, causing extrinsic cataracts; if the injection site is too far from the limbus, the needle tip may damage the vortex veins, leading to complications such as intraocular hemorrhage. Summary of the Invention

[0005] In order to achieve the above-mentioned objectives and other advantages of the present invention, a first objective of the present invention is to provide a method for processing intravitreal injection data based on OCT, comprising the following steps:

[0006] The OCT module and the vitreous injection robot module were calibrated, and data transmission was established.

[0007] Identifying the limbus using ocular surface microscopic images and anterior segment OCT images;

[0008] Using the identified limbus as a reference, a circular area is set on the surface of the eyeball; wherein, the center of the circle is set as the pupil, the distance between the inner ring and the limbus is set as a first preset distance, and the distance between the outer ring and the limbus is set as a second preset distance, and the first preset distance is less than the second preset distance;

[0009] After determining the annular region, the annular region is divided into four quadrants: upper, lower, nasal, and temporal, according to the position of the eyeball. Only the lower temporal quadrant is retained as the puncture area. The puncture position is randomly generated within the puncture area, and the puncture angle is set to a random value within the range of 45° to 60°.

[0010] The preset puncture position and puncture angle are transformed from the microscope coordinate system to the robotic arm coordinate system, and the transformed pose information is transmitted to the robotic arm.

[0011] The pose of the needle tip is measured using OCT images and compared with preset values;

[0012] Determine whether the three-dimensional error of the puncture position is less than the preset position error and whether the puncture angle error is less than the preset angle error;

[0013] If yes, it is determined that the condition for advancing the syringe has been met; otherwise, it is determined that the condition for controlling the syringe to return along its original path has been met.

[0014] The insertion depth and angle of the needle are measured in real time using OCT images of the anterior segment;

[0015] When the measurement results meet the condition that the distance between the needle tip and the eye surface is within the preset distance range, it is determined that the condition for controlling the syringe to stop advancing has been met, and an injection command is sent to the robotic arm.

[0016] Furthermore, the calibration of the OCT module and the vitreous injection robot module includes the following steps:

[0017] The robotic arm is calibrated using the TCP method, with the fixed reference point being the syringe needle tip on the plane and the robotic arm reference point being the syringe needle tip on the end effector, thus obtaining the coordinates of the needle tip reference point in the base coordinate system.

[0018] The syringe needle tip is tracked using an OCT module to obtain the coordinates of the needle tip reference point in the OCT coordinate system;

[0019] By solving the rigidity variation relationship between reference point pairs using the Kabsch algorithm, hand-eye calibration of the OCT-integrated vitreous injection robot can be achieved.

[0020] Furthermore, the calibration of the robotic arm using the TCP method includes the following steps:

[0021] Manipulate the robotic arm to make the syringe needle tip on the end effector contact the syringe needle tip on the plane in several different postures. Record the rotation matrix and translation vector between the base coordinate system of the robotic arm and the coordinate system of the end effector respectively, and substitute them into formula (1) to solve the relative position between the syringe needle tip and the robotic arm.

[0022] R BEi ·t EiT +t BEi =t BT (i=1,2,...,n) (1)

[0023] Among them, R BEi Let t be the rotation matrix between the robot arm's base coordinate system and the end effector's coordinate system. EiT Let t be the translation vector between the end effector coordinate system of the robotic arm and the syringe needle tip. BEi Let t be the translation vector between the robot arm's base coordinate system and the end effector's coordinate system. BT Let be the translation vector between the robot arm's base coordinate system and the syringe needle tip;

[0024] The solution t is obtained using the least squares method. EiT The position and pose information of the syringe needle tip in the coordinate system of the robotic arm are obtained to complete the calibration of the robotic arm.

[0025] Furthermore, the step of using the OCT module to track the syringe needle tip and obtain the coordinates of the needle tip reference point in the OCT coordinate system includes the following steps:

[0026] The syringe needle tip is manipulated to move within a set space, at least three sets of needle tip coordinates are recorded, and the needle tip at the recorded point is scanned synchronously using an OCT module.

[0027] Three-dimensional reconstruction was performed on the OCT scan data. The volume data was sliced ​​in the x, y, and z directions. The xz slice and yz slice were read sequentially. The slice containing high-brightness points was selected. The high-brightness point with the largest z coordinate was selected from the slices and the coordinates of the high-brightness point were recorded to obtain the coordinates of the needle tip in the OCT coordinate system.

[0028] Furthermore, the movement of the syringe needle tip within the set space is described as controlling the robotic arm in various directions (x, y, z) and various angles (R) within the set space. x ,R y ,R z (To exercise.)

[0029] Furthermore, the method of solving the rigidity variation relationship between reference point pairs using the Kabsch algorithm includes the following steps:

[0030] After obtaining the reference point coordinates of the needle tip in the base coordinate system and the OCT coordinate system, the reference point coordinates are normalized, and the center point C of the reference point set of the robot arm is obtained using formula (2). Rob and the center point C of the OCT reference point set OCT ;

[0031]

[0032]

[0033] Using two center points C Rob and C OCT Center the two sets of points to eliminate the influence of the translation vector t;

[0034]

[0035]

[0036] After obtaining the two centered point sets, the covariance matrix H between the reference point sets is calculated using formula (6);

[0037]

[0038] The optimal rotation matrix R is obtained by SVD decomposition, and then the translation vector t is solved by inverse solution of the rotation matrix R to complete the calibration.

[0039] Furthermore, the identification of the limbus via ocular surface microscopic images and anterior segment OCT images includes the following steps:

[0040] In the ocular surface microscopic image, edge detection algorithm and ellipse detection algorithm are used to perform preliminary localization of the limbus.

[0041] In the anterior segment OCT image, the limbus is precisely located by identifying the anterior and posterior boundaries of the limbus; wherein, the anterior boundary is defined by the line connecting the end of the anterior elastic layer of the cornea to the end of the posterior elastic layer of the cornea, and the posterior boundary is defined by the tangent line on the ocular surface originating from the scleral protrusion.

[0042] Furthermore, the preliminary localization of the limbus using edge detection and ellipse detection algorithms in the ocular surface microscopic image includes the following steps:

[0043] The ocular surface microscopic image is preprocessed by converting it from RGB space to HSV space and performing histogram equalization, binarization, opening and closing operations and Gaussian filtering.

[0044] The initial edges are obtained by using the Canny operator for edge extraction.

[0045] Edge detection algorithms and arc determination conditions are used to remove unacceptable edges, and elliptical arcs that meet the requirements are selected by setting constraints to achieve limbal identification. The arc determination conditions include ellipse integrity, ellipse edge point number ratio coefficient, and ellipse sign. The constraints include absolute size of major and minor axes, relative size of major and minor axes, and axis position.

[0046] Furthermore, the measurement of the needle tip pose using OCT images includes the following steps:

[0047] In response to a click operation at the needle tip, obtain the needle tip's position information;

[0048] The angle information of the needle tip is obtained by selecting any point along the tangent of the cornea, including the needle body and the needle tip.

[0049] A second objective of the present invention is to provide an electronic device comprising: a memory having program code stored thereon; and a processor connected to the memory, wherein when the program code is executed by the processor, a method for processing intravitreal injection data based on OCT is implemented.

[0050] A third objective of this invention is to provide a computer-readable storage medium having program instructions stored thereon, which, when executed, implement a method for processing intravitreal injection data based on OCT.

[0051] A fourth objective of this invention is to provide an OCT-based vitreous injection robot for implementing the above-described method, comprising an OCT module and a vitreous injection robot module. The OCT module is used for image guidance, providing binocular stereomicroscopic images and OCT images of the scanned area. The vitreous injection robot module includes a robotic arm and an end effector. The vitreous injection robot is used for ocular surface puncture and intraocular drug injection.

[0052] Compared with the prior art, the beneficial effects of the present invention are:

[0053] This invention combines optical coherence tomography (OCT) with a vitreous injection robot, improving the puncture and injection accuracy of vitreous injections and reducing iatrogenic and subsequent damage caused by uncertainties in the relative spatial position of tissue and needle tip. The introduction of OCT technology not only assists binocular cameras in more precise limbal localization but also enables real-time imaging and tracking of intraocular tissues, lesions, and needle tips, providing feedback for puncture and drug injection, and achieving automated drug injection at predetermined locations and angles without causing damage.

[0054] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0055] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0056] Figure 1 This is a schematic diagram of the OCT-based vitreous injection robot structure in Example 1;

[0057] Figure 2 This is a flowchart of the OCT-based vitreous injection data processing method in Example 2;

[0058] Figure 3 This is a flowchart illustrating the calibration process of the OCT module and the vitreous injection robot module in Example 2.

[0059] Figure 4 This is a schematic diagram of the limbus cornea in the anterior segment OCT image of Example 2;

[0060] Figure 5 This is a schematic diagram of the annular region division in Example 2;

[0061] Figure 6 This is a schematic diagram of the cube frame of Example 2;

[0062] Figure 7 This is a three-dimensional reconstruction and slice image of the needle tip from Example 2;

[0063] Figure 8 This is a schematic diagram of the electronic device in Example 3;

[0064] Figure 9This is a schematic diagram of the storage medium in Example 4.

[0065] In the diagram: 1. OCT module; 2. Vitreous injection robot module. Detailed Implementation

[0066] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0067] Example 1

[0068] This embodiment combines optical coherence tomography (OCT) with a vitreous injection robot to form an OCT-based vitreous injection robot, such as... Figure 1 As shown, the system includes an OCT module 1 and a vitreous injection robot module 2. The OCT module 1 is an OCT navigation system used for image guidance, providing binocular stereomicroscopic images and OCT images of the scanning area. The maximum scanning area of ​​the OCT image is 10.24mm × 10.24mm × 13.7mm, the resolution of the binocular stereomicroscopic image is 1920 × 1080, the horizontal resolution of the OCT image is 9.8μm, and the depth resolution is 5.7μm. The vitreous injection robot module includes a robotic arm and an end effector. The robotic arm has a repeatability accuracy of 0.01mm. The vitreous injection robot is used for ocular surface puncture and intraocular drug injection.

[0069] This embodiment adds an OCT imaging device that shares the same path as the binocular camera. The binocular camera, OCT imaging device, and robotic arm together constitute the entire vitreous injection robot. The main function of the OCT module is to visualize intraocular surgical instruments, eye tissues, and lesions. When the needle tip is outside the eyeball, the OCT image can assist the binocular camera image in more accurately locating the limbus. When the needle tip is inside the eyeball, the OCT image can provide needle tip position / direction information that the binocular camera cannot provide, realizing fully automatic and high-precision vitreous injection.

[0070] In existing technologies, vitreous injection robots typically use microscopic images of the ocular surface for navigation. However, during scleral puncture and drug injection, the needle tip position and angle cannot be observed, requiring intervention from the surgeon or pre-setting parameters based on experience. Intervention diminishes the robot's advantages; semi-automatic injection is inferior to fully automatic injection in terms of precision and efficiency. Pre-set parameters are often too conservative; to avoid damage to the anterior chamber angle and lens, the robot is positioned far from the lesion, requiring higher drug dosages and concentrations, which can cause some damage to the eye. The emergence of Optical Coherence Tomography (OCT) has precisely addressed this deficiency. OCT is a high-resolution, non-contact tomographic imaging technique based on low-coherence interferometry. It adds depth information to non-quantitative, vectorized stereoscopic visual information, making it particularly suitable for delicate ophthalmic microsurgery. Real-time acquisition of the pose and spatial relative position information of ocular tissues and surgical instruments gives vitreous injection robots better clinical outcomes and lower surgical risks. For OCT-integrated vitreous injection robots, the high-precision calibration of the robot module and the OCT module determines the success of the vitreous injection. During the injection, the syringe, guided by the OCT, enters vertically into the inferotemporal quadrant of the eye within an area 3.5-4 mm from the limbus, injecting the medication 4-6 mm into the eye. To ensure the precision and stability of the procedure, both the robot module and the OCT module must meet high-precision spatial constraints throughout the entire process; otherwise, it may lead to conjunctival hemorrhage, conjunctival scarring, severe pain, or even traumatic cataracts.

[0071] For a detailed description of the vitreous injection data processing method of the OCT-based vitreous injection robot, please refer to the corresponding description in the following method embodiments, which will not be repeated here.

[0072] Example 2

[0073] Example 1 provides a method for processing intravitreal injection data using an OCT-based intravitreal injection robot, such as... Figure 2 As shown, it includes the following steps:

[0074] After calibrating the OCT module and the vitreous injection robot module and establishing data transmission, automatic injection into the vitreous cavity can be performed.

[0075] Calibration in OCT-integrated intravitreal injection robots involves establishing a mathematical model to solve for the three-dimensional spatial relationship between the robot and the imaging system. After obtaining the homogeneous transformation matrix between them, visual information is used as feedback to plan and control various operations. Based on the positional relationship between the imaging system and the robot module, calibration can be divided into eye-to-hand calibration and eye-in-hand calibration. In eye-to-hand calibration, the imaging module is fixed, while the robot module can move freely; in eye-in-hand calibration, the imaging module moves along with the robot module. Due to the limited space in the operating room and the need for the robot module to contact the surgical area, the eye-to-hand calibration method is typically chosen. Zhang et al. proposed a hand-eye calibration method based on a circular mesh calibration plate in their paper (A computationally efficient method for hand-eye calibration, Int J CARS 12, 1775-1787, (2017)). This method uses dual quaternions to represent the rigid transformation between the robotic arm and the stereo laparoscope, and simultaneously recovers the real and dual parts of the dual quaternions through a two-step iterative method to estimate the rotation matrix and translation vector. In experiments using the Da Vinci Experimental Toolkit (DVRK), the calibration results converged in only three iterations. Kenji et al. proposed a hand-eye calibration method based on minimizing reprojection error in their paper (General Hand-eye calibration based on reprojection error minimization, IEEE ROBOTICS AND AUTOMATION LETTERS, (2019)). This method simultaneously estimates the hand-eye transformation and the pose of the calibration object under the condition of minimizing the reprojection error of the calibration object. It can also adapt to different camera models by changing the projection model. In the calibration experiment of the pinhole camera, the root mean square error of reprojection was 1.619 pixels, and the reconstruction accuracy error was 1.379 mm. 2In the calibration experiment of the X-ray camera, the root mean square error of reprojection was 7.118 pixels. Jesus et al. proposed a hand-to-hand calibration method based on a binocular camera in their paper (A low-cost stereovision system for eye-to-hand calibration, IEEE International Autumn Meeting on Power, Electronics and computing, (2022)). The method uses a binocular camera to track a small ball on the robot's end effector and solves the rigidity change between the robot and the binocular camera by capturing the motion trajectory of the ball's center. In the simulation experiment, the calibration error was 3.313 mm. In the above hand-to-hand calibration, either a calibration object is placed or it has obvious morphological features. The artificial markers or natural features are converted into reference point pairs through image recognition, thereby obtaining the rigidity transformation relationship between the robot and the imaging system. However, for the OCT-integrated vitreous injection robot, it is difficult to set artificial markers in the surgical scene, and the structural features of the eyeball are relatively complex, making it impossible to perform unified recognition and tracking.

[0076] This embodiment uses the needle tip in the vitreous injection scenario as a reference point. A robotic arm is used for calibration to obtain the coordinates of the needle tip reference point in the base coordinate system. OCT is used to track the needle tip to obtain the coordinates of the needle tip reference point in the OCT coordinate system. Then, the Kabsch algorithm is used to solve for the rigidity change relationship between reference point pairs, achieving hand-eye calibration of the OCT-integrated vitreous injection robot. Specifically, as... Figure 3 As shown, the calibration of the OCT module and the vitreous injection robot module includes the following steps:

[0077] The robotic arm is calibrated using the TCP method, with the fixed reference point being the syringe needle tip on a plane, and the robotic arm reference point being the syringe needle tip on the end effector. The coordinates of the needle tip reference point in the base coordinate system are obtained. Specifically, the calibration of the robotic arm using the TCP method includes the following steps:

[0078] Using a teach pendant to manipulate the robotic arm, the syringe needle tip on the end effector is made to contact the syringe needle tip on the plane in several different postures. In this embodiment, the syringe needle tip on the end effector is made to contact the syringe needle tip on the plane in four different postures. The rotation matrix and translation vector between the base coordinate system of the robotic arm and the coordinate system of the end effector are recorded respectively, and they are substituted into formula (1) to solve the relative position between the syringe needle tip and the robotic arm.

[0079] R BEi ·t EiT +t BEi =t BT (i=1,2,3,4) (1)

[0080] Among them, R BEi Let t be the rotation matrix between the robot arm's base coordinate system and the end effector's coordinate system. EiT Let t be the translation vector between the end effector coordinate system of the robotic arm and the syringe needle tip. BEi Let t be the translation vector between the robot arm's base coordinate system and the end effector's coordinate system. BT Let be the translation vector between the robot arm's base coordinate system and the syringe needle tip;

[0081] The solution t is obtained using the least squares method. EiT The position and pose information of the syringe needle tip in the coordinate system of the robotic arm are obtained to complete the calibration of the robotic arm.

[0082] The syringe needle tip is tracked using an OCT module to obtain the coordinates of the needle tip reference point in the OCT coordinate system;

[0083] Because the OCT imaging area is limited, subsequent calibration work needs to be performed within a designated space. This embodiment uses a customized cubic frame, such as... Figure 6 As shown, its inner frame is 12mm×12mm×12mm and its outer frame is 14mm×14mm×14mm, covering the OCT scanning area (10.24mm×10.24mm×10.24mm).

[0084] Specifically, using the OCT module to track the syringe needle tip and obtain the coordinates of the needle tip reference point in the OCT coordinate system includes the following steps:

[0085] Using a teach pendant, manipulate the syringe needle tip to move freely within a designated space, specifically within a cube-shaped frame. Record at least three sets of needle tip coordinates and use an OCT module to synchronously scan the needle tip at the recorded points. It is important to note that during the movement, it is best to move the robotic arm at various angles (R) in all directions (x, y, z). x ,R y ,R z All of them are in motion, so that the calibration results obtained have smaller errors.

[0086] As the needle tip moves within the cube frame, OCT is used to simultaneously scan the needle tip at the recording point.

[0087] Three-dimensional reconstruction was performed on the OCT scan data. The volume data was sliced ​​along the x, y, and z directions. The tip 3D reconstruction image and the slice image are shown below. Figure 7 As shown, the xz and yz slices are read sequentially, and slices containing high-brightness points are filtered out. Then, the high-brightness point with the largest z-coordinate is selected from these slices, and its coordinates are recorded to obtain the coordinates of the needle tip in the OCT coordinate system.

[0088] The Kabsch algorithm is used to solve the rigidity variation relationship between reference point pairs, enabling hand-eye calibration of an OCT-integrated vitreous injection robot. The specific steps include:

[0089] After obtaining the reference point coordinates of the needle tip in the base coordinate system and the OCT coordinate system, the reference point coordinates are normalized, and the center point C of the reference point set of the robot arm is obtained using formula (2). Rob and the center point C of the OCT reference point set OCT ;

[0090]

[0091]

[0092] To calculate the rotation matrix R, we use two center points C. Rob and C OCT Center the two sets of points to eliminate the influence of the translation vector t;

[0093]

[0094]

[0095] After obtaining the two centered point sets, the covariance matrix H between the reference point sets is calculated using formula (6);

[0096]

[0097] The optimal rotation matrix R is obtained by SVD decomposition, and then the translation vector t is solved by inverse solution of the rotation matrix R to complete the calibration.

[0098] After obtaining the rigidity transformation matrix between the robotic arm coordinate system and the OCT coordinate system, the accuracy of the above calibration method was verified by sampling multiple times within a cube frame. The average three-dimensional coordinate error was 0.1101 mm, the RMSE was 0.2397, and the average angle error was 0.822°, which meets the requirements for vitreous injection.

[0099] This embodiment uses the needle tip in the vitreous injection scenario as a reference point, avoiding the identification of natural features and the placement of manual calibration objects. The coordinates of the reference point in two coordinate systems are obtained through robotic arm calibration and OCT scanning, and then the Kabsch algorithm is used to solve the rigid transformation between the coordinate systems. The calibration spatial error is 0.1101 mm, and the angular error is 0.822°, meeting the high precision requirements of vitreous injection.

[0100] Identifying the limbus using ocular surface microscopy and anterior segment OCT images; specifically including the following steps:

[0101] In ocular surface microscopic images, edge detection and ellipse detection algorithms are used for preliminary localization of the limbus. The limbus is the transitional region between the cornea and sclera; viewed from outside the eyeball, it is the boundary between the iris and the white of the eye. It has two distinct morphological features: a clear boundary between the cornea and sclera, and a circular shape. Since the eyeball is not a perfect sphere and deforms during movement, strictly speaking, the limbus is an ellipse. Therefore, edge detection and ellipse detection algorithms can be used to identify the limbus.

[0102] Preprocessing is performed on the ocular surface microscopic images by converting them from RGB space to HSV space, and then performing histogram equalization, binarization, opening and closing operations, and Gaussian filtering.

[0103] The initial edges are obtained by using the Canny operator for edge extraction.

[0104] Edge detection algorithms and arc determination criteria are used to remove edges that do not meet the requirements. Elliptical arcs that meet the requirements are selected by setting constraints to achieve the identification of limbus. The arc determination criteria include ellipse integrity, ellipse edge point number ratio coefficient, and ellipse sign. The constraints include absolute size of major and minor axes, relative size of major and minor axes, and axis position.

[0105] In anterior segment OCT images, the limbus is precisely located by identifying its anterior and posterior boundaries. The anterior boundary is defined by the line connecting the endpoint of Bowman's layer to the endpoint of Descemet's membrane (the basement membrane of the corneal endothelial cells). The posterior boundary is defined by a tangent line on the ocular surface originating from the scleral ridge. The limbus in the OCT image is shown as follows: Figure 4 As shown, Figure 4 In the middle section, BM refers to the Bowman layer, DM refers to the Descemet's membrane, and SS refers to the scleral protuberance.

[0106] Using the identified limbus as a reference, a circular area is set on the surface of the eyeball; wherein, the center of the circle is set as the pupil, the distance between the inner ring and the limbus is set as a first preset distance, such as setting the first preset distance to 3mm, and the distance between the outer ring and the limbus is set as a second preset distance, such as setting the first preset distance to 4mm, the first preset distance is less than the second preset distance;

[0107] After determining the annular region, it is divided into four quadrants—superior, inferior, nasal, and temporal—based on eye position (left / right eye), as detailed below. Figure 5 As shown. Only the inferotemporal quadrant region is retained (i.e. Figure 5 The lower temporal side marked in the middle is used as the puncture area. The puncture position p1 is randomly generated within the puncture area, and the puncture angle θ1 (the angle between the needle tip and the ocular surface) is set to a random value within the range of 45° to 60°.

[0108] The preset puncture position and angle are transformed from the microscope coordinate system to the robotic arm coordinate system. The transformed pose information is then transmitted to the robotic arm, enabling it to control the syringe to perform puncture at the preset position and a fixed angle. After calibrating the robotic arm and OCT in the above calibration steps, a transformation matrix between the robotic arm coordinate system and the OCT coordinate system is obtained. This transformation matrix can be used to achieve coordinate system transformation of position / angle, from the microscope coordinate system (OCT coordinate system) to the robotic arm coordinate system.

[0109] After the puncture is completed, the position of the needle tip is measured using OCT images and compared with preset values; specifically, the following steps are included:

[0110] The OCT module's real-time imaging software has a click-to-position function and a three-point angle calculation function. After the needle tip enters the eye, by clicking on the needle tip position, the OCT module responds to the click operation of the needle tip position and obtains the position information p2 of the needle tip; the angle information θ2 of the needle tip is obtained by selecting any point on the needle body, the needle tip, and along the corneal tangent direction.

[0111] Determine whether the three-dimensional error of the puncture position is less than the preset position error and whether the puncture angle error is less than the preset angle error; in this embodiment, the preset position error is set to 0.1mm and the preset angle error is set to 1°.

[0112] If so, the condition for injecting the syringe is met, and the syringe is injected.

[0113] Otherwise, if the condition for controlling the syringe to return along its original path is met, the syringe will be controlled to return along its original path.

[0114] Once the needle tip enters the eyeball, the insertion depth p3 and angle θ3 are measured in real time using anterior segment OCT images;

[0115] When the measurement results meet the condition that the distance between the needle tip and the eye surface is within the preset distance range, such as 4-6 mm, it is determined that the condition for controlling the syringe to stop advancing has been met. The syringe stops advancing, sends an injection command to the robotic arm, and after the injection is completed, the syringe returns along the original path.

[0116] This invention improves the puncture and injection accuracy of intraocular injections, reducing iatrogenic and subsequent damage caused by the uncertainty of the relative position of tissue and needle tip. The introduction of OCT technology not only assists binocular cameras in more precise limbal localization, but also enables real-time imaging and tracking of intraocular tissues, lesions, and needle tips, providing feedback for puncture and drug injection, and achieving automatic drug injection at predetermined locations and fixed angles without causing damage.

[0117] Example 3

[0118] An electronic device 200, such as Figure 8 As shown, the method includes, but is not limited to: a memory 201 storing program code; and a processor 202 connected to the memory, which, when the program code is executed by the processor, implements the OCT-based vitreous injection data processing method. For a detailed description of the method, please refer to the corresponding description in the above method embodiments, which will not be repeated here.

[0119] Example 4

[0120] A computer-readable storage medium, such as Figure 9 As shown, it stores program instructions, which, when executed, implement an OCT-based vitreous injection data processing method. For a detailed description of the method, please refer to the corresponding description in the above method embodiments; it will not be repeated here.

[0121] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0122] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0123] The above are merely embodiments of this specification and are not intended to limit the scope of one or more embodiments of this specification. Various modifications and variations can be made to one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of one or more embodiments of this specification.

Claims

1. An OCT-based intravitreal injection robot, characterized in that: It includes an OCT module and a vitreous injection robot module. The OCT module is used for image guidance, providing binocular stereomicroscopic images and OCT images of the scanned area. The vitreous injection robot module includes a robotic arm and an end effector. The vitreous injection robot is used for ocular surface puncture and intraocular drug injection. The intravitreal injection robot is configured to perform an OCT-based intravitreal injection data processing method, which includes the following steps: The OCT module and the vitreous injection robot module were calibrated, and data transmission was established. Identifying the limbus using ocular surface microscopic images and anterior segment OCT images; Using the identified limbus as a reference, a circular area is set on the surface of the eyeball; wherein, the center of the circle is set as the pupil, the distance between the inner ring and the limbus is set as a first preset distance, and the distance between the outer ring and the limbus is set as a second preset distance, and the first preset distance is less than the second preset distance; After determining the annular region, the annular region is divided into four quadrants: upper, lower, nasal, and temporal, according to the position of the eyeball. Only the lower temporal quadrant is retained as the puncture area. The puncture position is randomly generated within the puncture area, and the puncture angle is set to a random value within the range of 45° to 60°. The preset puncture position and puncture angle are transformed from the microscope coordinate system to the robotic arm coordinate system, and the transformed pose information is transmitted to the robotic arm. The pose of the needle tip is measured using OCT images and compared with preset values; Determine whether the three-dimensional error of the puncture position is less than the preset position error and whether the puncture angle error is less than the preset angle error; If yes, it is determined that the condition for advancing the syringe has been met; otherwise, it is determined that the condition for controlling the syringe to return along its original path has been met. The insertion depth and angle of the needle are measured in real time using OCT images of the anterior segment; When the measurement results meet the condition that the distance between the needle tip and the eye surface is within the preset distance range, it is determined that the condition for controlling the syringe to stop advancing has been met, and an injection command is sent to the robotic arm.

2. The OCT-based vitreous injection robot as described in claim 1, characterized in that, The calibration of the OCT module and the intravitreal injection robot module includes the following steps: The robotic arm is calibrated using the TCP method, with the fixed reference point being the syringe needle tip on the plane and the robotic arm reference point being the syringe needle tip on the end effector, thus obtaining the coordinates of the needle tip reference point in the base coordinate system. The syringe needle tip is tracked using an OCT module to obtain the coordinates of the needle tip reference point in the OCT coordinate system; By solving the rigidity variation relationship between reference point pairs using the Kabsch algorithm, hand-eye calibration of the OCT-integrated vitreous injection robot can be achieved.

3. The OCT-based vitreous injection robot as described in claim 2, characterized in that, The calibration of the robotic arm using the TCP method includes the following steps: Manipulate the robotic arm so that the syringe needle tip on the end effector contacts the syringe needle tip on the plane in several different postures. Record the rotation matrix and translation vector between the base coordinate system of the robotic arm and the coordinate system of the end effector respectively, and substitute them into formula (1) to solve the relative position between the syringe needle tip and the robotic arm. （1） in, Let be the rotation matrix between the robot arm's base coordinate system and the end effector's coordinate system. Let be the translation vector between the coordinate system of the robotic arm's end effector and the syringe needle tip. Let be the translation vector between the robot arm's base coordinate system and the end effector's coordinate system. Let be the translation vector between the robot arm's base coordinate system and the syringe needle tip; Solving using the least squares method yields The position and pose information of the syringe needle tip in the coordinate system of the robotic arm are obtained to complete the calibration of the robotic arm.

4. The OCT-based vitreous injection robot as described in claim 3, characterized in that, The process of using an OCT module to track the syringe needle tip and obtain the coordinates of the needle tip reference point in the OCT coordinate system includes the following steps: The syringe needle tip is manipulated to move within a set space, at least three sets of needle tip coordinates are recorded, and the needle tip at the recorded point is scanned synchronously using an OCT module. Three-dimensional reconstruction was performed on the OCT scan data. The volume data was sliced ​​in the x, y, and z directions. The xz slice and yz slice were read sequentially. The slice containing high-brightness points was selected. The high-brightness point with the largest z coordinate was selected from the slices and the coordinates of the high-brightness point were recorded to obtain the coordinates of the needle tip in the OCT coordinate system.

5. The OCT-based intravitreal injection robot as described in claim 4, characterized in that: The control of the syringe needle tip to move within a set space is equivalent to controlling the robotic arm in various directions within that set space. From all angles Engage in exercise.

6. The OCT-based intravitreal injection robot as described in claim 4, characterized in that, The method of solving the rigidity variation relationship between reference point pairs using the Kabsch algorithm includes the following steps: After obtaining the reference point coordinates of the needle tip in the base coordinate system and the OCT coordinate system of the robotic arm, the reference point coordinates are normalized, and the center point of the set of reference points of the robotic arm is obtained using formula (2). and the center point of the OCT reference point set ; （2） （3） Using two center points and Center the two sets of points to eliminate the influence of the translation vector t; （4） （5） After obtaining the two centered point sets, the covariance matrix H between the reference point sets is calculated using formula (6); （6） The optimal rotation matrix R is obtained by SVD decomposition, and then the translation vector t is solved by inverse solution of the rotation matrix R to complete the calibration.

7. The OCT-based vitreous injection robot as described in claim 1, characterized in that, The identification of the limbus using ocular surface microscopic images and anterior segment OCT images includes the following steps: In the ocular surface microscopic image, edge detection algorithm and ellipse detection algorithm are used to perform preliminary localization of the limbus. In the anterior segment OCT image, the limbus is precisely located by identifying the anterior and posterior boundaries of the limbus; wherein, the anterior boundary is defined by the line connecting the end of the anterior elastic layer of the cornea to the end of the posterior elastic layer of the cornea, and the posterior boundary is defined by the tangent line on the ocular surface originating from the scleral protrusion.

8. The OCT-based intravitreal injection robot as described in claim 7, characterized in that, The preliminary localization of the limbus using edge detection and ellipse detection algorithms in the ocular surface microscopic image includes the following steps: The ocular surface microscopic image is preprocessed by converting it from RGB space to HSV space and performing histogram equalization, binarization, opening and closing operations and Gaussian filtering. The initial edges are obtained by using the Canny operator for edge extraction. Edge detection algorithms and arc determination conditions are used to remove unacceptable edges, and elliptical arcs that meet the requirements are selected by setting constraints to achieve limbal identification. The arc determination conditions include ellipse integrity, ellipse edge point number ratio coefficient, and ellipse sign. The constraints include absolute size of major and minor axes, relative size of major and minor axes, and axis position.

9. The OCT-based vitreous injection robot as described in claim 1, characterized in that, The measurement of the needle tip pose using OCT images includes the following steps: In response to a click operation at the needle tip, obtain the needle tip's position information; The angle information of the needle tip is obtained by selecting any point along the tangent of the cornea, including the needle body and the needle tip.

Images