Apparatus and method for delivering treatment or detection light to ocular tissue of patient

Abstract

An apparatus and method for delivering treatment or detection light to ocular tissue of a patient, where the apparatus includes a light source device, a camera device, an actuation device, and a controller. The controller is coupled with the light source device, the camera device and the actuating device, and is configured to control the camera device to collect at least two images of the eyes of the patient at different angles; at least extracting iris information of the patient based on the at least two images at different angles; judging the eye state of the patient at least based on the iris information; and controlling the light source device to deliver or not deliver the treatment light or the detection light to the eye tissue of the patient based on the eye state of the patient, and / or controlling the actuating device to move the light source device.

Description

Technical Field

[0001] The present application relates to the field of ophthalmic treatment, and more particularly, to an apparatus and method for delivering therapeutic light or detection light to an ocular tissue of a patient. Background Art

[0002] Using lasers or other light of different wavelengths to irradiate the patient's eyes to treat eye diseases is a common type of ophthalmic treatment. For example, LASIK surgery uses excimer lasers to reshape the corneal curvature to correct myopia, hyperopia and astigmatism; retinal laser coagulation uses green or yellow lasers to coagulate the lesion area of ​​the retina to treat diabetic retinopathy or retinal tears; and YAG laser posterior capsulotomy uses infrared lasers to destroy posterior capsule opacities after cataract surgery. In the above-listed ophthalmic treatments, in order to achieve better treatment of the diseased tissue or the area to be operated on and effectively avoid damage to healthy tissue, it is usually necessary to judge and monitor the patient's eye condition (such as whether the treatment light is aimed at the part of the eyeball to be treated) before or during treatment. However, since this type of treatment is usually accompanied by local anesthesia of the patient's eyes and the use of devices such as eyelid openers, the focus of judging the patient's eye condition is mainly on limited situations such as whether the patient's eyeball has micro-movement.

[0003] In addition, there are some light irradiation treatment scenarios in which the patient's eyes are in a non-anesthetized state. For example, the Chinese patent application with publication number CN113924143A discloses a device for treating cataracts by emitting light of a specific wavelength to the affected eye through an LED light source. The device uses light energy to treat the patient's cataracts, which usually takes 20 minutes each time, and the patient's eyes are in a non-anesthetized state throughout the process. In this case, due to physiological or other pathological factors, as well as the influence of attention diversion, the patient's eyes may experience shaking, tremors, blinking, closing eyes, etc., thereby affecting the final treatment effect or causing damage to healthy tissue. In addition, due to the long treatment time of the device, the use of the traditional method of emitting detection light to the target position of the eye and processing its reflected light signal to obtain target position information may lead to the risk of excessive light energy absorption by the eye. At the same time, since the detection light may interfere with the treatment light, the final treatment effect may not meet expectations.

[0004] Therefore, it is necessary to improve existing ophthalmic treatment equipment and methods to enhance the ability to detect the patient's eye condition. Summary of the invention

[0005] One object of the present application is to provide a device and method for delivering therapeutic light or detection light to a patient's eye tissue, which has an improved ability to detect the patient's eye condition, thereby better achieving treatment and detection effects while avoiding damage to the normal tissue of the patient's eye.

[0006] In one aspect of the present application, a device for delivering therapeutic light or detection light to a patient's eye tissue is provided, the device comprising: a light source device, configured to generate therapeutic light or detection light; a camera device, configured to capture images of the patient's eye; an actuator device, configured to at least move the light source device; and a controller, coupled to the light source device, the camera device and the actuator device, and configured to: control the camera device to capture at least two images of the patient's eye at different angles; extract at least the patient's iris information based on the at least two images at different angles; determine the patient's eye condition based at least on the iris information; and control the light source device to deliver or not deliver the therapeutic light or the detection light to the patient's eye tissue based on the patient's eye condition, and / or control the actuator device to move the light source device.

[0007] In another aspect of the present application, a method for delivering therapeutic light or detection light to a patient's eye tissue is provided, the method comprising: controlling a camera device to capture at least two images of the patient's eye at different angles; extracting at least the patient's iris information based on the at least two images at different angles; judging the patient's eye condition based at least on the iris information; and controlling a light source device to deliver or not deliver the therapeutic light or the detection light to the patient's eye tissue based on the patient's eye condition, and / or controlling an actuator device to move the light source device.

[0008] In another aspect of the present application, a non-volatile computer-readable storage medium is provided, characterized in that a computer program is stored on the non-volatile computer-readable storage medium, and when the computer program is executed by a processor, the above-mentioned method for delivering therapeutic light or detection light to the patient's eye tissue is implemented.

[0009] The above is an overview of the present application, which may be simplified, summarized, and omitted in detail. Therefore, those skilled in the art should recognize that this section is only illustrative and is not intended to limit the scope of the present application in any way. This summary section is neither intended to determine the key features or essential features of the claimed subject matter, nor is it intended to be used as an auxiliary means to determine the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features of the present application will be more fully understood through the following description and the attached claims in conjunction with the accompanying drawings. It is understood that these drawings only depict several embodiments of the present application and should not be considered as limiting the scope of the present application. By using the accompanying drawings, the present application will be more clearly and detailed.

[0011] Figure 1 A block diagram of an apparatus for delivering therapeutic light or detection light to an eye tissue of a patient according to one embodiment of the present application is shown.

[0012] FIG. 2A to FIG. 2C A stereoscopic view, a front view and a top view of a treatment module according to an embodiment of the present application are respectively shown.

[0013] Figure 3A and Figure 3B Patient eye images before and after image processing according to an embodiment of the present application are respectively shown.

[0014] FIG. 4A to FIG. 4G A schematic diagram of extracting iris contour information from a patient's eye image according to an embodiment of the present application is shown.

[0015] Figure 5A and Figure 5B A schematic diagram showing a patient's eyes being open according to an embodiment of the present application.

[0016] Fig. 6A and Figure 6B A schematic diagram showing a patient's eyes in a squinting state according to an embodiment of the present application.

[0017] Fig. 7A and Figure 7B A schematic diagram showing a patient's eyes being in a closed state according to an embodiment of the present application.

[0018] Figure 8 A schematic diagram of determining the distance between the center of a light outlet of a light source device and a preset light outlet center position according to an embodiment of the present application is shown.

[0019] Fig. 9 A flow chart of a method for delivering therapeutic light or detection light to an eye tissue of a patient according to one embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] In the following detailed description, reference is made to the accompanying drawings which form a part thereof. In the drawings, similar symbols generally represent similar components unless the context indicates otherwise. The illustrative embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be employed and other changes may be made without departing from the spirit or scope of the subject matter of the present application. It is understood that various configurations, substitutions, combinations, and designs of various aspects of the present application generally described in this application and illustrated in the drawings may be made, all of which explicitly constitute a part of the present application.

[0021] refer to Figure 1 , which shows a block diagram of an ophthalmic treatment device 100 for delivering treatment light or detection light to the patient's eye tissue according to an embodiment of the present application. Figure 1 As shown, the ophthalmic treatment device 100 includes a controller 110 , and a light source device 120 , a camera device 130 , an actuator device 140 , a memory 150 , and a user interface device 160 that are communicatively connected to the controller 110 .

[0022] The light source device 120 is configured to generate therapeutic light or detection light. The light source device 120 can be a laser light source, or a non-laser light source, such as an LED light source. In some embodiments, the ophthalmic treatment device 100 is configured to deliver therapeutic light with a preset wavelength band to the patient's lens to treat cataract disease in the patient's eye. In some embodiments, the light source device 120 is configured to emit therapeutic light with a wavelength range of 400nm to 570nm to the patient's lens to treat the patient's eye. In some embodiments, the wavelength of the therapeutic light of the light source device 120 is in the range of 410nm to 420nm. In some embodiments, the light source device 120 is configured to send detection light to the patient's eye tissue, for example, emitting light with a wavelength in the range of 350nm to 410nm, preferably in the range of 360nm to 370nm, and more preferably at a wavelength of 365nm to excite fluorescent light in the cataract. It can be understood that in different embodiments, for different eye diseases, the therapeutic light or detection light generated by the light source device 120 can have different wavelengths, and the present application is not limited to the above example wavelengths.

[0023] The camera 130 is configured to capture images of the patient's eyes, so that the controller 110 can determine the patient's eye status based on these images. Figure 1In the example, the camera device 130 and the light source device 120 are arranged together to form the treatment module 102 of the ophthalmic treatment device 100. The actuator 140 is configured to move at least the light source device 120. When the light source device 120 and the camera device 130 are arranged together, the actuator 140 can be mechanically coupled to the treatment module 102 to drive the treatment module 102 to move, so that the positions of the light source device 120 and the camera device 130 and the patient's eyes can be adjusted simultaneously, thereby achieving alignment of specific areas of the patient's eyes. In some embodiments, the actuator 140 can adopt a three-axis drive system to drive the treatment module 102 to move along the X-axis, the Y-axis and / or the Z-axis. For example, the actuator 140 may include three independent drive units (such as a stepper motor or a servo motor) to achieve driving of the treatment module 102 in three different dimensions. However, the present application is not limited to the above examples. In some other embodiments, the light source device 120 and the camera device 130 may be provided separately, and the actuating device 140 may actuate the light source device 120 and the camera device 130 respectively.

[0024] The memory 150 is used to store relevant data or program instructions of the ophthalmic treatment device 100, which includes but is not limited to a random access memory (RAM) for temporary storage of information, or a flash memory, hard disk, removable disk, read-only memory (ROM) for permanent storage of information. The user interface device 160 may include any device capable of interacting with a user, including but not limited to a display, a touch screen interface, a keyboard, an audio input or output device, a mouse, a joystick or a button, etc. The controller 110 is coupled to the light source device 120, the camera device 130, the actuator device 140, the memory 150 and / or the user interface device 160, and can execute the program stored in the memory 150 or receive instructions from the user interface device 160 to complete tasks or operations, for example, according to a specific treatment plan, the light source device 120, the camera device 130, the actuator device 140, the memory 150 and / or the user interface device 160 are controlled or operated to deliver the treatment light or the detection light to the patient's eye tissue. The controller 110 may include a general or dedicated processor, a controller, a field programmable gate array (FPGA), etc. In an embodiment of the present invention, the controller 110 is capable of controlling the camera device 130 to capture at least two images of the patient's eyes at different angles; extracting at least the patient's iris information based on the at least two images at different angles; judging the patient's eye condition based at least on the iris information; and controlling the light source device 120 to deliver or not deliver therapeutic light or detection light to the patient's eye tissue based on the patient's eye condition, and / or controlling the actuator device 140 to move the light source device 120.

[0025] refer to FIG. 2A to FIG. 2C, which respectively show a stereoscopic view, a front view and a top view of the treatment module 102 in an embodiment of the present application. In this embodiment, the camera device 130 includes a first camera 130-1 and a second camera 130-2, and the first camera 130-1 and the second camera are installed at a preset angle relative to the light source device 120 so as to obtain images of the eye 200 of the patient to be treated from different angles. Figure 2B As shown, the first camera 130-1 and the second camera 130-2 are symmetrically arranged on both sides of the light outlet of the light source device 120 along the radial direction of the light source device 120. Figure 2C As shown, the optical axes of the first camera 130-1 and the second camera 130-2 intersect with the axis of the light source device 120 at the same point, and the angle θ between the optical axes of the first camera 130-1 and the second camera 130-2 is greater than 0 degrees and less than 120 degrees, preferably 40 to 80 degrees. In some embodiments, during the treatment process, the patient's eyes will reflect light with a wavelength of 360nm-530nm when continuously receiving treatment light or detection light. In order to reduce the impact of the reflected light on the visual judgment of the camera's real-time monitoring, corresponding filters can also be set in the first camera 130-1 and the second camera 130-2.

[0026] It should be noted that although in 2A to Figure 2C In the embodiment, the first camera 130-1 and the second camera 130-2 are arranged on both sides of the light outlet of the light source device 120, but the present application is not limited to this. In some other embodiments, the first camera 130-1 and the second camera 130-2 may also be arranged at other positions where the patient's eye images can be captured at different angles. In some other embodiments, the camera device 130 may include three or more cameras to capture three or more images of the patient's eyes at different angles. For example, in addition to the first camera and the second camera arranged on both sides of the light source device 120, the camera device 130 may also include a third camera or more cameras arranged at other positions of the light source device 120 (for example, coaxial with the light source device 120). In some other embodiments, the camera device 130 may include only one camera, and the controller 110 may control the movement of the camera to capture the patient's eye images at different angles.

[0027] The following combination Figure 1 The ophthalmic treatment device 100 and FIG. 2A to FIG. 2CThe treatment module 102 with two cameras is shown to provide a detailed description of the device for delivering treatment light or detection light to the patient's eye tissue of the present application. It can be understood that the technical solution of the present application is also applicable to devices with one, three or more cameras, which can also improve the ability to detect the patient's eye status after appropriate adjustment and deformation, thereby better achieving the treatment effect while avoiding damage to the normal tissue of the patient's eye.

[0028] refer to Figure 3A , the controller 110 controls the first camera 130-1 to capture a first image 310 of the patient's eye 200 from a first angle, and controls the second camera 130-2 to be controlled to capture a second image 320 of the patient's eye 200 from a second angle different from the first angle. After receiving the first image 310 and the second image 320, the controller 110 extracts first eye information a1 and first iris information b1 from the first image 310; and extracts second eye information a2 and second iris information b2 from the second image 320.

[0029] In some embodiments, the first eye information a1 and the second eye information a2 may include: whether the characteristic information of the eye, the contour information of the eye, and / or other information of the eye are extracted from the first image 310 and the second image 320. The first iris information b1 and the second iris information b2 may include: whether the characteristic information, contour information, shape information, gradient information, color information, or texture information of the iris are extracted from the first image 310 and the second image 320. Specifically, since there is a very obvious color difference between the iris and the surrounding eye tissue, these differences constitute the edge features of the iris; in image processing, after preprocessing such as image enhancement, the contour of the iris edge can be extracted by using threshold segmentation and contour detection to obtain the contour information of the iris. The overall shape of the iris may vary from person to person, but it is basically close to a circle, an ellipse, etc.; in image processing, after preprocessing such as image enhancement, the circle or arc in the image can be detected by using methods such as Hough circle detection, and then the circle that fits the iris edge can be selected from the detected multiple circles by setting a screening strategy, so as to obtain the shape information of the iris. The gradient value reflects the rate of change of pixel intensity in the image; the grayscale gradient from the center to the edge of the iris, as well as the sudden change at the edge, are all information that can be used; for example, in an image, the gradient can be calculated for each pixel, and the gradient contains the amplitude and direction; for eye images, the closer to the center of the eyeball, the lower the grayscale value, and there will be more gradient lines intersecting at that point. The determination of the center position is to find the point where the straight lines in the gradient direction intersect the most, so as to obtain the center position information of the iris. The color information of the iris can include basic color, color distribution and color intensity, among which the basic color is usually divided into several main categories such as blue, brown, green, etc.; color distribution refers to the distribution of color on the iris, such as spots, stripes or radial patterns; color intensity refers to the depth of color, such as the difference between dark brown and light brown. The fine texture of the iris surface can be captured by high-resolution images; in addition, the distribution of blood vessels in the iris, although not the main basis for iris recognition, can also be used as an auxiliary feature in some cases; the texture information of the iris is a common feature used for auxiliary detection. Some examples of iris information are given above, but the present application is not limited thereto. According to different application scenarios, iris information may also include position information, size information, semantic information, etc.

[0030] The controller 110 extracts the first eye information a1 and the first iris information b1 from the first image 310 and extracts the second eye information a2 and the second iris information b2 from the second image 320 by an image processing method or by a machine learning or deep learning algorithm.

[0031] The technical solution of the present application is described below by obtaining eye information and iris information through an image processing method.

[0032] In some embodiments, after acquiring the first image 310 and the second image 320, the first image 310 and the second image 320 may be processed first, and then the eye information and the iris contour information may be extracted. For example, the first image 310 and the second image 320 may be subjected to gamma correction and perspective transformation correction.

[0033] As a specific example, the gamma correction process may include the following steps S11-S13. In step S11, the first image 310 and the second image 320 are respectively grayed using the weighted average method to form a grayscale image. In step S12, each pixel in the grayscale image is filtered. For example, each pixel in the grayscale image is processed by spatial proximity and pixel value similarity. In step S13, an adaptive gamma correction method is used to process the filtered grayscale image. For example, the grayscale image is filtered by a Gaussian filter to obtain a template, wherein the kernel of the Gaussian filter is larger than 3*3 pixels; then all elements in the template are scaled and nonlinearly mapped; the value of each element in the template processed by scaling and nonlinearly mapping is regarded as a gamma value; and according to the multiple gamma values ​​obtained, the filtered grayscale image is gamma corrected pixel by pixel. Reference Figure 3B , which shows the first image 310 and the second image 320 after gamma correction. It can be seen that by considering the value of each element in the template processed by scale scaling and function linear mapping as a gamma value, and performing pixel-by-pixel gamma correction on the grayscale image after filtering through the obtained multiple gamma values, the low brightness area of ​​the image can be effectively enhanced, the high brightness area can be suppressed, and the overall brightness of the image can be balanced.

[0034] Since the first camera 130-1 and the second camera 130-2 are installed at a non-vertical angle, the first image 310 and the second image 320 captured will have perspective distortion, that is, objects in the distance in the image appear smaller than those in the near. In order to make the first image 310 and the second image 320 look like they are taken from the front, a perspective transformation can be applied to correct the image, which can make subsequent target detection more accurate. The perspective transformation correction can be used before or after any one of the above-mentioned gamma correction steps S11-S13. As a specific example, the perspective transformation correction may include the following steps S21-S23. In step S21, according to the installation angles of the first camera 130-1 and the second camera 130-2, the expected values ​​of the four corner points of the image after the perspective transformation are calculated. In step S22, the four corner points of the image and the expected values ​​of the four corner points obtained in step S21 are used to calculate the perspective transformation matrix using the getPerspectiveTransform function in the OpenCV library. In step S23, the OpenCV function warpPerspective is used to perform perspective transformation and output the corrected image. The input of the warpPerspective function includes the original image, the perspective transformation matrix, and the desired output image size. The first image 310 and the second image 320 corrected by perspective transformation can better reflect the geometric relationship of the real world, which helps to improve the accuracy of the calibration and three-dimensional coordinate calculation of the first camera 130-1 and the second camera 130-2.

[0035] The above describes the image processing of the first image 310 and the second image 320 by taking gamma correction and perspective transformation correction as examples. It can be understood that the present application is not limited thereto, and in some other embodiments, other processing methods can be used to process the first image 310 and the second image 320, as long as they are helpful in extracting eye information and iris information.

[0036] Combine the following FIG. 4A to FIG. 4G , an example process of extracting the first iris contour information b1 from the first image 310 or extracting the second iris contour information b2 from the second image 320 is described.

[0037] First, the original image (eg, Figure 4A As shown) to take the eye area and obtain the eye area image (as shown Figure 4B shown).

[0038] Next, the local threshold segmentation method is applied to the eye region image to segment the image into foreground and background, and the image after local threshold segmentation is obtained (such as Figure 4CSpecifically, the gray value of each pixel in the window is compared with the local threshold; if the pixel gray value is greater than the local threshold, the pixel is marked as foreground (target); if the pixel gray value is less than or equal to the local threshold, the pixel is marked as background (non-target).

[0039] Next, morphological operations are performed on the segmented image, such as corrosion and expansion, filling holes and connecting adjacent segmented areas, to remove noise and further refine the shape of the iris, and obtain a morphologically processed image (such as Figure 4D shown).

[0040] Next, use the findContours() function in OpenCV to find all the contours in the image, and obtain the external contours by specifying cv2.RETR_EXTERNAL to obtain the image after contour detection (such as Figure 4E shown).

[0041] Next, the contours in the image after contour detection also include the contours of the eyelids and part of the eyebrows, so it is necessary to set filtering conditions (such as the area and length of the contour) to filter out the contour that is most likely to be the iris, and obtain the image after the contour is filtered (such as Figure 4F As shown), the first iris contour information b1 (or the second iris contour information b2) is obtained.

[0042] In addition, after extracting the iris contour information b1, the maximum circumscribed rectangle of the iris contour can be further obtained (such as Figure 4G As shown), the center coordinates of the largest circumscribed rectangle are calculated as the coordinates of the iris center point, which are subsequently used to determine the patient's eye condition. It should be noted that the present application uses the coordinates of the iris center point as the coordinates of the pupil center to achieve the delivery of therapeutic light to the pupil.

[0043] In some embodiments, extracting the first eye information a1 from the first image 310 or extracting the second eye information a2 from the second image 320 may include the following steps. First, extract the eye area from the first image 310 (or the second image 320) through a Haar cascade classifier, and then crop the eye area from the original image; then, apply the image processing method in OpenCV to further refine the extraction of eye features. Haar cascade is a feature-based object detection method that achieves target detection by cascading features in an image. It uses Haar features as basic image features, which can capture the grayscale changes in different areas of the image, thereby constructing a cascade classifier. The method first slides a window of a specific size in the image, then calculates the values ​​of multiple Haar features in each window, and finally uses a cascade classifier to determine whether these windows contain target objects. This method has been widely used in fields such as face detection due to its advantages in computational efficiency and accuracy.

[0044] After extracting the first eye information a1 and the first iris contour information b1 from the first image 310 and extracting the second eye information a2 and the second iris contour information b2 from the second image 320, the controller 110 can judge the patient's eye state based on the first eye information a1 and the second eye information a2, and the first iris contour information b1 and the second iris contour information b2.

[0045] Specifically, refer to Figure 5A and 5B , when the first iris outline information b1 and the second iris outline information b2 indicate that the iris outline exists, and the height-to-width ratios of the first circumscribed rectangle 510 and the second circumscribed rectangle 520 corresponding to the first iris outline information b1 and the second iris outline information b2 are not less than a preset value, the controller 110 determines that the patient's eyes are in an open state. In some embodiments, the preset value is between 0.2 and 0.6. Preferably, the preset value is 0.5.

[0046] refer to Fig. 6A and 6B When the first iris outline information b1 and the second iris outline information b2 indicate that an iris outline exists, and the aspect ratios of the first circumscribed rectangle 610 and the second circumscribed rectangle 620 corresponding to the first iris outline information b1 and the second iris outline information b2 are both smaller than a preset value, the controller 110 determines that the patient's eyes are in a squinting state.

[0047] refer to Fig. 7A and 7BWhen at least one of the first eye information a1 and the second eye information a2 indicates that the eye exists, and at least one of the first iris outline information b1 and the second iris outline information b2 indicates that the iris outline does not exist, the controller 110 determines that the patient's eyes are in a closed state.

[0048] In addition, when the first iris contour information b1 and the second iris contour information b2 indicate the existence of an iris contour, the distance between the center of the light outlet of the light source device 120 and the preset light outlet center position is determined. When the distance is greater than a preset threshold, it is judged that the patient's eyes are in an incorrectly aligned state; when the distance is less than or equal to the preset threshold, it is judged that the patient's eyes are in a correctly aligned state.

[0049] In some embodiments, the controller 110 may determine the distance between the center of the light outlet of the light source device 120 and the preset light outlet center position by the following steps. Figure 8 First, a three-dimensional coordinate system (X, Y, Z) is established with the center point of the iris of the patient's eye as the origin O, and the preset center position P of the light outlet is located on the coordinate axis X of the three-dimensional coordinate system. The distance between the preset center position P of the light outlet and the origin O of the three-dimensional coordinate system is between 20 mm and 150 mm. Next, determine the coordinate value of the center of the light outlet of the light source device 120 in the three-dimensional coordinate system. Since in a specific application, the relative position relationship and distance between the first camera 130-1 and the second camera 130-2 and the light source device 120 are determined, the binocular ranging algorithm can be used based on the images of different angles captured by the first camera 130-1 and the second camera 130-2 to determine the coordinate value of the center of the light outlet of the light source device 120 in the three-dimensional coordinate system. Next, the distance r between the center of the light outlet of the light source device 120 and the preset center position P of the light outlet is calculated based on the coordinate value. The distance r can be expressed as Wherein, x, y and z are the difference between the center point of the light outlet and the preset center position P of the light outlet in the three-dimensional coordinate system. When the distance r is greater than the preset threshold, it is judged that the patient's eyes are in an incorrect alignment state; when the distance r is less than or equal to the preset threshold, it is judged that the patient's eyes are in a correct alignment state. The preset threshold is between 0.1mm and 1mm, preferably 0.5mm. Continue to refer to Figure 8, in the three-dimensional coordinate system (X, Y, Z), the preset light outlet center position P is taken as the sphere center. If the center point of the light outlet is within a sphere with P as the sphere center and a preset threshold value (e.g., 0.5mm) as the radius (i.e., r<0.5mm), the patient's eyes are judged to be in a correctly aligned state; otherwise, the patient's eyes are judged to be in an incorrectly aligned state. Since the actuator (e.g., a three-axis drive system) moves according to the difference between the three-dimensional coordinates of the center point of the light outlet and the three-dimensional coordinates of the preset light outlet center position P, when the first iris contour information b1 and the second iris contour information b2 are extracted, the alignment state is set when r<0.5mm, thereby eliminating the jitter of the three-axis drive system.

[0050] In other embodiments, when the first eye information a1 and the first iris contour information b1 indicate that the feature information of the eye and the feature information of the iris cannot be extracted in the first image 310, or when the second eye information a2 and the second iris contour information b2 indicate that the feature information of the eye and the feature information of the iris cannot be extracted in the second image 320, it is determined that the state of the patient's eyes is not detected.

[0051] The above describes the specific steps of using an image processing method to obtain eye information and iris information, and to determine the eye status of a patient, using a specific example. However, the present application is not limited thereto, and in some other embodiments, other image processing methods may be used to obtain eye information and iris information and / or determine the eye status of a patient according to specific needs.

[0052] For example, the center point of the iris can be determined based on the direction information of the gradient, which includes the following steps: a. Gradient calculation: grayscale and edge detection are performed on the original image to obtain the gradient value and direction of each pixel; b. Gradient direction analysis: in an iris image, the gradient direction of pixels near the center of the iris usually points to the center of the iris; this is because the grayscale difference between the edge of the pupil and the iris is large, resulting in the gradient direction mostly pointing to the center of the iris; c. Center point positioning: count all gradient directions and find the point where the gradient directions intersect the most, which is the center of the iris.

[0053] In other examples, the center point of the iris can be determined based on the Hough transform, which may include the following steps: a. Edge detection: using methods such as Canny edge detection to detect edges in the image; b. Hough circle transform: applying the Hough circle transform to detect circular contours in the image; the Hough circle transform can detect circular objects in the image, and can effectively detect them even if the boundaries are incomplete; c. Center of circle positioning: the center coordinates returned by the Hough circle transform are the estimated position of the iris center.

[0054] In some other examples, the center point of the iris can be determined based on threshold segmentation, which may include the following steps: a. Threshold segmentation: selecting a suitable threshold according to the grayscale value range of the image, and dividing the image into two parts: foreground (iris) and background (the rest); b. Morphological operation: using morphological operations such as dilation and erosion to remove noise and refine contours; c. Centroid calculation: calculating the centroid of the segmented foreground area (iris), which is the iris center.

[0055] In some other examples, multiple methods may be combined to determine the iris center point. For example, the results of multiple methods such as gradient direction, Hough transform, and threshold segmentation may be combined to determine the final iris center position using weighted average or other fusion strategies.

[0056] The above embodiments describe an embodiment in which the controller 110 uses an image processing method to obtain iris information and judge the patient's eye state. However, the present application is not limited thereto. In some other embodiments, the controller 110 may also use machine learning or deep learning algorithms to extract the patient's iris information from an image and / or judge the patient's eye state.

[0057] Iris detection algorithms based on machine learning or deep learning algorithms can use the powerful capabilities of models such as convolutional neural networks (CNNs) to detect and locate the center of the iris in an image. The following introduces the input information, output information, and common models when using machine learning or deep learning algorithms to obtain iris information.

[0058] The input information can include an eye image. Specifically, the input information can be a color image or a black and white image, and the size can be adjusted according to the requirements of the model. For example, if the model requires an input size of 3×640×640 pixels, the input image needs to be scaled to this size. Before being input into the network, the image may also need to be preprocessed such as standardization or normalization, such as dividing by 255 so that the pixel value is between 0 and 1, or subtracting the mean and dividing by the standard deviation to standardize the image data.

[0059] The output information may include the coordinates of the center of the iris. The output information of the model is usually the coordinates of the center of the iris circumscribed rectangle (x, y) and the height and width (h, w), where (x, y) is the position in the input image coordinate system. The output information of the model may also include a confidence score. For example, the model will also output a confidence score, indicating the credibility of the model's prediction results.

[0060] In some embodiments, models such as target detection models and semantic segmentation models can be used to detect and locate the iris center in the image. The target detection model is usually used to locate specific objects in the image and give the position coordinates of these objects. The target detection model may include a yolo series model (e.g., yolov3, yolov4, yolov5, ..., yolov8, yolov9, yolov10, yolox, etc.), an rcnn series model (e.g., Fast R-CNN, Faster R-CNN, Mask R-CNN, etc.), and other detection models (e.g., SSD, CenterNet, etc.). The semantic segmentation model can divide the image into different areas and assign a category label to each pixel. For iris detection tasks, this type of model can be used to segment the iris area and locate the iris center by calculating the segmentation results. The semantic segmentation model may include U-Net, DeepLab, FCN, PSPNet, etc.

[0061] After obtaining the patient's eye state (i.e., open state, squinting state, closed state, improperly aligned state, or state in which the patient's eyes are not detected), the controller 110 can also control the light source device 120 to deliver or not deliver therapeutic light or detection light to the patient's eye tissue based on the patient's eye state, and / or control the actuator device to move the light source device 120.

[0062] Specifically, in the case where it is judged that the patient's eyes are in an open state, the controller 110 controls the light source device 120 to deliver therapeutic light or detection light. In the case where it is judged that the patient's eyes are in a squinting state, the controller 110 controls the light source device 120 to deliver therapeutic light, and after the squinting state lasts for a first preset time length (e.g., 10 seconds), a first prompt message is output. In the case where it is judged that the patient's eyes are in a closed state, the light source device 120 is controlled to deliver therapeutic light, and after the closed state lasts for a second preset time length (e.g., 6 seconds), a second prompt message is output. In addition, in the case where it is judged that the patient's eyes are in an improperly aligned state, the controller 110 controls the light source device 120 to deliver therapeutic light and controls the actuator 140 to move the light source device 120. In the case where the state of the patient's eyes is not detected, if the state of the patient's eyes is not detected for a third preset time length (e.g., 6 seconds), the controller 110 controls the light source device 120 to stop delivering therapeutic light or detection light; further, in some examples, the controller 110 can also control the actuator 140 to return to the preset point position.

[0063] In some embodiments, when it is determined that the patient's eyes are in an incorrect alignment state, if the patient's eyes are in a correct alignment state by moving the light source device 120, the light source device 120 is continuously turned on to deliver the therapeutic light or the detection light; if the incorrect alignment state continues for more than another preset time length (for example, 6 seconds) although the light source device 120 is moved, the controller 110 outputs a third prompt message and controls the light source device 120 to stop delivering the therapeutic light or the detection light. In some embodiments, since the first camera 130-1 and the second camera 130-2 may have tearing and delay in the captured images during the movement, errors may be generated in the calculation of the three-dimensional coordinates, causing the actuator 140 (for example, a three-axis drive system) to sway and follow the eyeball seriously, which violates the purpose of eliminating jitter, so the image sampling may not be performed during the movement of the light source device 120.

[0064] In some embodiments, the radiation power of the therapeutic light or detection light delivered in the open state, squinting state, closed eye state, and incorrect alignment state is the same, which is conducive to simplifying the operation of the light source device 120. In other embodiments, the radiation power of the therapeutic light or detection light delivered in the open state, squinting state, closed eye state, and incorrect alignment state may also be different or may vary. For example, when the patient is detected to be in a squinting state or a closed eye state, it may be because the patient is not adapted to the stimulation of the therapeutic light or the detection light, and the light source device 120 may be controlled to deliver the therapeutic light or the detection light with a reduced radiation power; after the patient's squinting state or the closed eye state continues for a fourth preset time length (for example, 6 seconds), the patient has adapted to the therapeutic light or the detection light, and the light source device 120 may be controlled to restore the original radiation power to continue delivering the therapeutic light or the detection light. For another example, when it is determined that the patient's eyes are in a squinting state or a closed eye state, the radiation power of the light source device 120 may be controlled to change in a step-by-step manner. Specifically, when it is determined that the patient's eyes are in a squinting state or a closed state, the radiation power of the light source device 120 is controlled to be reduced from a first preset power to a second preset power; and after the radiation power of the light source device 120 is reduced to the second preset power, if the controller 110 determines that the patient's eyes are still in a squinting state or a closed state, the radiation power of the light source device 120 is controlled to be reduced from the second preset power to a third preset power.

[0065] For another example, when it is determined that the patient's eyes are in an open state, the radiation power of the light source device 120 may also be controlled to change in a stepwise manner. Specifically, when it is determined that the patient's eyes are in an open state, the radiation power of the light source device 120 is controlled to increase from the first preset power to the fourth preset power; and after the radiation power of the light source device 120 is increased to the fourth preset power, if the controller 110 determines that the patient's eyes are still in an open state, the radiation power of the light source device 120 is controlled to increase from the fourth preset power to the fifth preset power.

[0066] Further, the controller 110 may provide the user with the aforementioned first, second, and third prompt information via the user interface device 160 to inform the user of the current working state of the ophthalmic treatment device 100, for example, by providing text information through a screen, or providing sound information through an audio playback device, etc. Specifically, after determining that the patient's eyes are in a squinting state and last for a first preset time length, the controller 110 outputs a first prompt information (for example, "eyelid occlusion, please adjust") via the user interface device 160 to prompt the user with information about eyelid occlusion. After determining that the patient's eyes are in a closed state and last for a second preset time length, the second prompt information (for example, "eyelid occlusion, please adjust") is output via the user interface device to prompt the user with information about eyelid occlusion. After determining that the patient's eyes are in an incorrectly aligned state and last for a third preset time length, the third prompt information (for example, "eye alignment is not completed, please adjust") is output via the user interface device to prompt the user with information that the patient's eye alignment is not completed.

[0067] The present application also provides a method for delivering therapeutic light or detection light to an ocular tissue of a patient. In some embodiments, Fig. 9 As shown, the method 900 for delivering therapeutic light or detection light to the patient's eye tissue includes the following steps 910-940. In step 910, the camera device is controlled to capture at least two images of the patient's eye at different angles. In step 920, at least the iris information of the patient is extracted based on the at least two images at different angles. In step 930, the patient's eye state is judged based at least on the iris information. In step 940, based on the patient's eye state, the light source device is controlled to deliver or not deliver the therapeutic light or the detection light to the patient's eye tissue, and / or the actuator device is controlled to move the light source device. The method 900 can be performed by the device 100 for delivering therapeutic light or detection light to the patient's eye tissue described above, and more details can be found in the above combined with the attached. Figures 1 to 8 It is necessary to explain that the above method 900 can be executed multiple times periodically when the device 100 is used to treat the patient's eyes, so that the state of the patient's eyes can be continuously monitored and corresponding treatment can be performed.

[0068] The present application also provides a non-volatile computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method for delivering therapeutic light or detection light to the patient's eye tissue in the above embodiment is executed. In some embodiments, the non-volatile computer-readable storage medium can be a flash memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of non-volatile computer-readable storage medium known in the art.

[0069] It should be noted that although several modules or submodules of the method and apparatus for delivering therapeutic light or detection light to the patient's ocular tissue are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present application, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules to be embodied.

[0070] Those of ordinary skill in the art can understand and implement other changes to the disclosed embodiments by studying the specification, the disclosed content, the drawings and the appended claims. In the claims, the word "comprising" does not exclude other elements and steps, and the words "a" and "an" do not exclude the plurality. In the actual application of this application, a part may perform the functions of multiple technical features cited in the claims. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A device for delivering therapeutic light or detection light to an ocular tissue of a patient, characterized in that The device comprises: a light source device configured to generate therapeutic light or detection light; An imaging device configured to capture images of the patient's eyes; an actuating device configured to move at least the light source device; and A controller is coupled to the light source device, the camera device and the actuation device, and is configured to: Controlling the camera device to capture at least two images of the patient's eyes at different angles; Extracting at least the iris information of the patient based on the at least two images at different angles; determining the patient's eye condition based at least on the iris information; and Based on the eye condition of the patient, the light source device is controlled to deliver or not deliver the therapeutic light or the detection light to the eye tissue of the patient, and / or the actuator device is controlled to move the light source device.

2. The device according to claim 1, characterized in that The eye tissue is a lens, the therapeutic light has a preset wavelength band, and the device is configured to deliver the therapeutic light to the lens of the patient to treat cataract disease.

3. The device according to claim 1, characterized in that The iris information includes contour information, shape information, gradient information, color information, or texture information.

4. The device according to claim 1, characterized in that The camera device includes a first camera and a second camera, the first camera and the second camera are installed at a preset angle relative to the light source device, and the controller controls the camera device to collect at least two images of the patient's eyes at different angles, including: Controlling the first camera to capture a first image of the patient's eye from a first angle; and The second camera is controlled to capture a second image of the patient's eye from a second angle different from the first angle.

5. The device according to claim 4, characterized in that The first camera and the second camera are installed on both sides of the light outlet of the light source device, and the angle between the optical axes of the first camera and the second camera is greater than 0 degree and less than 120 degrees.

6. The device according to claim 1, characterized in that The controller extracts at least the iris information of the patient based on the at least two images at different angles, including: Extracting first eye information and first iris contour information from a first image of the at least two images at different angles; and The second eye information and the second iris contour information are extracted from a second image of the at least two images at different angles.

7. The device according to claim 6, characterized in that The controller determines the eye state of the patient based at least on the iris information, including: When the first iris outline information and the second iris outline information indicate that an iris outline exists, and the aspect ratios of the first circumscribed rectangle and the second circumscribed rectangle corresponding to the first iris outline information and the second iris outline information are not less than a preset value, it is determined that the patient's eyes are in an open state; When the first iris outline information and the second iris outline information indicate that the iris outline exists, and the aspect ratios of the first circumscribed rectangle and the second circumscribed rectangle corresponding to the first iris outline information and the second iris outline information are both smaller than a preset value, it is determined that the patient's eyes are in a squinting state; When at least one of the first eye information and the second eye information indicates that an eye exists, and at least one of the first iris outline information and the second iris outline information indicates that the iris outline does not exist, determining that the patient's eyes are in a closed state; In a case where the first iris outline information and the second iris outline information indicate that the iris outline exists, determining a distance between a center of a light outlet of the light source device and a preset light outlet center position, and when the distance is greater than a preset threshold, determining that the patient's eyes are in a misaligned state; or When the first eye information and the first iris contour information indicate that the feature information of the eye and the feature information of the iris cannot be extracted in the first image, or when the second eye information and the second iris contour information indicate that the feature information of the eye and the feature information of the iris cannot be extracted in the second image, it is determined that the patient's eye is not detected.

8. The device according to claim 7, characterized in that The preset value is between 0.2 and 0.

6.

9. The device according to claim 7, characterized in that The controller determines the distance between the center of the light outlet of the light source device and the preset light outlet center position, including: The three-dimensional coordinate system is established with the center point of the iris of the patient's eye as the origin, and the preset center position of the light outlet is located on a coordinate axis of the three-dimensional coordinate system; Determining the coordinate value of the center of the light outlet of the light source device in the three-dimensional coordinate system; and The distance between the center of the light outlet of the light source device and the preset light outlet center position is calculated based on the coordinate value.

10. The device according to claim 9, characterized in that The distance between the preset light outlet center position and the origin of the three-dimensional coordinate system is between 20 mm and 100 mm, and the preset threshold is between 0.1 mm to 1mm.

11. The device according to claim 7, characterized in that The controller controls the light source device to deliver or not deliver the therapeutic light or the detection light to the eye tissue of the patient based on the eye state of the patient, and / or controls the actuator device to move the light source device, including: In case that it is determined that the patient's eyes are in the open state, controlling the light source device to deliver the treatment light or the detection light; In the case where it is determined that the patient's eyes are in the squinting state, the light source device is controlled to deliver the therapeutic light, and after the squinting state continues for a first preset time length, a first prompt message is output; in the case where it is determined that the patient's eyes are in the closed-eye state, the light source device is controlled to deliver the therapeutic light, and after the closed-eye state continues for a second preset time length, a second prompt message is output; in the case where it is determined that the patient's eyes are in the improperly aligned state, the light source device is controlled to deliver the therapeutic light and the actuator device is controlled to move the light source device; or In the case of the state where the patient's eyes are not detected, if the state where the patient's eyes are not detected continues for a third preset time length, the light source device is controlled to stop delivering the treatment light or the detection light.

12. The device according to claim 1, characterized in that The controller controls the light source device to deliver or not deliver the therapeutic light to the patient's eye tissue based on the patient's eye state, comprising: In case it is determined that the patient's eyes are in a squinting state or an eye-closing state, controlling the light source device to deliver the therapeutic light with a reduced radiation power; and After the squinting state or the eye-closing state continues for a fourth preset time length, the light source device is controlled to restore the original radiation power to continue delivering the therapeutic light.

13. The device according to claim 1, characterized in that The controller controls the light source device to deliver or not deliver the therapeutic light to the patient's eye tissue based on the patient's eye state, comprising: When it is determined that the patient's eyes are in a squinting state or a closed eye state, controlling the radiation power of the light source device to decrease from a first preset power to a second preset power; After the radiation power of the light source device is reduced to the second preset power, if the controller determines that the patient's eyes are still in the squinting state or the closed eye state, the radiation power of the light source device is controlled to be reduced from the second preset power to the third preset power.

14. The device according to claim 1, characterized in that The controller controls the light source device to deliver or not deliver the therapeutic light to the patient's eye tissue based on the patient's eye state, comprising: When it is determined that the patient's eyes are in an open state, controlling the radiation power of the light source device to increase from a first preset power to a fourth preset power; and After the radiation power of the light source device is increased to the fourth preset power, if the controller determines that the patient's eyes are still in the open state, the radiation power of the light source device is controlled to increase from the fourth preset power to the fifth preset power.

15. The device according to claim 11, characterized in that The device further includes a user interface device, and the controller outputs the first prompt information and the second prompt information to a user via the user interface device, wherein the first prompt information and the second prompt information prompt the user of eyelid occlusion.

16. A method for delivering therapeutic light or detection light to an ocular tissue of a patient, characterized in that: The method comprises: Controlling a camera device to capture at least two images of the patient's eyes at different angles; Extracting at least the iris information of the patient based on the at least two images at different angles; determining the patient's eye condition based at least on the iris information; and Based on the eye condition of the patient, the light source device is controlled to deliver or not deliver the therapeutic light or the detection light to the eye tissue of the patient, and / or the actuator device is controlled to move the light source device.

17. The method according to claim 16, characterized in that The eye tissue is a lens, the therapeutic light has a preset wavelength band, and the method is used to deliver the therapeutic light to the lens of the patient to treat cataract disease.

18. The method according to claim 16, characterized in that The iris information includes contour information, shape information, gradient information, color information, or texture information.

19. The method according to claim 16, characterized in that Extracting at least the iris information of the patient based on the at least two images at different angles comprises: Extracting first eye information and first iris contour information from a first image of the at least two images at different angles; and The second eye information and the second iris contour information are extracted from a second image of the at least two images at different angles.

20. The method according to claim 19, characterized in that Determining the eye condition of the patient based at least on the iris information includes: When the first iris outline information and the second iris outline information indicate that an iris outline exists, and the aspect ratios of the first circumscribed rectangle and the second circumscribed rectangle corresponding to the first iris outline information and the second iris outline information are not less than a preset value, it is determined that the patient's eyes are in an open state; When the first iris outline information and the second iris outline information indicate that the iris outline exists, and the aspect ratios of the first circumscribed rectangle and the second circumscribed rectangle corresponding to the first iris outline information and the second iris outline information are both smaller than a preset value, it is determined that the patient's eyes are in a squinting state; When at least one of the first eye information and the second eye information indicates that an eye exists, and at least one of the first iris outline information and the second iris outline information indicates that the iris outline does not exist, determining that the patient's eyes are in a closed state; In a case where the first iris outline information and the second iris outline information indicate that the iris outline exists, determining a distance between a center of a light outlet of the light source device and a preset light outlet center position, and when the distance is greater than a preset threshold, determining that the patient's eyes are in a misaligned state; or When the first eye information and the first iris contour information indicate that the feature information of the eye and the feature information of the iris cannot be extracted in the first image, or when the second eye information and the second iris contour information indicate that the feature information of the eye and the feature information of the iris cannot be extracted in the second image, it is determined that the patient's eye is not detected.

21. The method according to claim 20, characterized in that Determining the distance between the center of the light outlet of the light source device and a preset light outlet center position includes: The three-dimensional coordinate system is established with the center point of the iris of the patient's eye as the origin, and the preset center position of the light outlet is located on a coordinate axis of the three-dimensional coordinate system; Determining the coordinate value of the center of the light outlet of the light source device in the three-dimensional coordinate system; and The distance between the center of the light outlet of the light source device and the preset light outlet center position is calculated based on the coordinate value.

22. The method according to claim 20, characterized in that Controlling the light source device to deliver or not deliver the therapeutic light or the detection light to the patient's eye tissue based on the patient's eye state, and / or controlling the actuator device to move the light source device comprises: In case that it is determined that the patient's eyes are in the open state, controlling the light source device to deliver the treatment light or the detection light; In the case where it is determined that the patient's eyes are in the squinting state, the light source device is controlled to deliver the therapeutic light, and after the squinting state continues for a first preset time length, a first prompt message is output; in the case where it is determined that the patient's eyes are in the closed-eye state, the light source device is controlled to deliver the therapeutic light, and after the closed-eye state continues for a second preset time length, a second prompt message is output; in the case where it is determined that the patient's eyes are in the improperly aligned state, the light source device is controlled to deliver the therapeutic light and the actuator device is controlled to move the light source device; or In the case of the state where the patient's eyes are not detected, if the state where the patient's eyes are not detected continues for a third preset time length, the light source device is controlled to stop delivering the treatment light or the detection light.

23. The method according to claim 16, characterized in that Controlling the light source device to deliver or not deliver the therapeutic light to the patient's eye tissue based on the patient's eye state comprises: In case it is determined that the patient's eyes are in a squinting state or an eye-closing state, controlling the light source device to deliver the therapeutic light with a reduced radiation power; and After the squinting state or the eye-closing state continues for a fourth preset time length, the light source device is controlled to restore the original radiation power to continue delivering the therapeutic light.

24. The method according to claim 16, characterized in that Controlling the light source device to deliver or not deliver the therapeutic light to the patient's eye tissue based on the patient's eye state comprises: When it is determined that the patient's eyes are in a squinting state or a closed eye state, controlling the radiation power of the light source device to decrease from a first preset power to a second preset power; and After the radiation power of the light source device is reduced to the second preset power, if the patient's eyes are still in the squinting state or the closed eye state, the radiation power of the light source device is controlled to be reduced from the second preset power to a third preset power.

25. The method according to claim 16, characterized in that Controlling the light source device to deliver or not deliver the therapeutic light to the patient's eye tissue based on the patient's eye state comprises: When it is determined that the patient's eyes are in the open state, controlling the radiation power of the light source device to increase from a first preset power to a fourth preset power; and After the radiation power of the light source device is increased to the fourth preset power, if the patient's eyes are still in the open state, the radiation power of the light source device is controlled to increase from the fourth preset power to a fifth preset power.

26. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium has a computer program stored thereon, which, when executed by a processor, implements the method for delivering therapeutic light to an eye tissue of a patient as described in any one of claims 16-25.

Images