A slit lamp self-adjustment control method based on machine vision and slit lamp

Abstract

The present invention provides a slit lamp self-adjustment control method based on machine vision, which includes the steps of 1) using a camera to collect eye images; 2) comparing the collected eye images with standard data in a database, determining the eye lesion area, and marking the lesion area; 3) establishing a two-dimensional coordinate system with the center of the pupil as the coordinate origin, and obtaining the coordinates of the lesion area; 4) generating control data according to the coordinates of the lesion area, controlling the slit lamp to obtain the best shooting angle, determining the type of eye lesions according to the lesion image of the marked lesion area, and generating the data of the filtering mode, the width of the projection light source, and the irradiation angle required for the slit lamp to shoot the lesion. The present invention can quickly capture the eye lesion area and identify the lesion category, provide a basis for the operation of the slit lamp, and ensure the slit lamp imaging effect to facilitate doctors to diagnose diseases.

Description

Technical Field

[0001] The invention relates to a slit lamp self-adjustment control method based on machine vision and a slit lamp. Background Art

[0002] Patients with ophthalmic diseases need to undergo a comprehensive and in-depth examination of their eyes during ophthalmic treatment in order to obtain an accurate analysis and treat the patients in a manner that best meets their needs. The most commonly used ophthalmic examination is a slit lamp microscope, which consists of an illumination system and a binocular microscope. It can not only observe superficial lesions very clearly, but also adjust the focus and light source width to make an "optical section" so that lesions in deep tissues can also be clearly displayed. However, the existing technology still has the following problems:

[0003] 1) It cannot assist doctors in determining the location of the lesion and adjusting the microscope so that the slit lamp is aimed at the lesion;

[0004] 2) The adjustment of the slit lamp parameters needs to be done manually, and the lesion type cannot be determined based on the lesion area image and the slit lamp parameters cannot be adjusted based on the lesion image. Summary of the invention

[0005] In view of the defects of the prior art, the present invention provides a slit lamp self-adjustment control method and a slit lamp based on machine vision, which can quickly capture the eye lesion area and identify the lesion type, provide a basis for the operation of the slit lamp, and ensure the slit lamp imaging effect to facilitate doctors to diagnose diseases.

[0006] In order to achieve the above object, the present invention provides a slit lamp self-adjustment control method based on machine vision, which comprises the following steps:

[0007] 1) Using a camera device to collect eye images;

[0008] 2) comparing the collected eye image with the standard data in the database, determining the eye lesion area, and marking the lesion area;

[0009] 3) Establishing a two-dimensional coordinate system with the center of the pupil as the origin to obtain the coordinates of the lesion area;

[0010] 4) Generate control data according to the coordinates of the lesion area, control the slit lamp to obtain the best shooting angle, determine the type of eye lesions according to the lesion image of the marked lesion area, and generate data on the filtering method, width of the projection light source, and irradiation angle required for the slit lamp to photograph the lesion;

[0011] The process of determining the eye lesion type of the lesion image of the marked lesion area is as follows:

[0012] Eye images of various types of eye diseases are selected, and specific information on the lesion image in each lesion area is extracted as identification information through an algorithm, and then a determination matrix P (P1, P2...Pn) that can determine the lesion image corresponding to each type of eye disease is generated and stored, wherein P1 represents one type of disease identification information, P2 represents the second type of disease identification information, and Pn represents the nth type of disease identification information. The lesion image of the marked lesion area is compared one by one with the determination matrix to determine the type of eye disease corresponding to the lesion image of the marked lesion area.

[0013] As another specific embodiment of the present invention, data extraction is performed on the eye image collected in step 1) to divide the collected eye image into an eye canthus image, an eye pupil image, an eyelid image and an eyelid area image.

[0014] As another specific implementation of the present invention, in step 2), the collected eye image is first binarized, a contrast threshold is preset, and the area where the grayscale change exceeds the preset contrast threshold is marked as the lesion area.

[0015] As another specific embodiment of the present invention, the process of generating manipulation data according to the coordinates of the lesion area in step 4) is:

[0016] A two-dimensional coordinate set f(x, y) of the lesion area is established according to the acquired coordinates of the lesion area;

[0017] According to the relationship between the position and shooting angle of the microscope and the lesion area, a coordinate set adjustment matrix Si (Si1, Si2, Si3, Si4, i=1, 2...n) is established, where Si1 represents the coordinate area, Si2 represents the X-axis coordinate adjusted by the microscope, Si3 represents the Y-axis coordinate adjusted by the microscope, and Si4 represents the adjustment angle of the microscope;

[0018] Determine whether the area represented by the two-dimensional coordinate set f(x, y) of the lesion area exists in the coordinate area Si1. If so, call the microscope to adjust the X-axis coordinate Si2, adjust the Y-axis coordinate Si3 and adjust the microscope angle Si4, and prompt that the microscope position should be adjusted to the X-axis coordinate Si1 and the Y-axis coordinate Si2, and the angle should be adjusted to the microscope adjustment angle Si4.

[0019] As another specific embodiment of the present invention, the specific information on the lesion image in the lesion area in step 4) includes contour features and chromaticity features.

[0020] As another specific implementation scheme of the present invention, when determining the type of eye lesions according to the lesion image of the marked lesion area in step 4), the lesion image is divided into a lesion edge area and a lesion center area, and the contour features and chromaticity features of the lesion edge area and the lesion center area are compared with the i-th (i=1, 2...n) type of disease identification information in the determination matrix P, and the matching areas and non-matching areas are recorded to determine the matching rate Sz of the lesion image:

[0021]

[0022] Where S1 represents the contour feature matching rate of the lesion edge area, S2 represents the contour feature matching rate of the lesion middle area, S3 represents the chromaticity matching rate of the lesion edge area, S4 represents the chromaticity matching rate of the lesion center area, α1 is the weight parameter of S1 to Sz, α2 is the weight parameter of S2 to Sz, α3 is the weight parameter of S3 to Sz, and α4 is the weight parameter of S4 to Sz;

[0023] Compare the matching rate Sz of the lesion image with the set matching rate parameter S of the eye disease image:

[0024] When Sz>S, it is determined that the eye disease image and the eye disease represented by the i-th type of disease identification information are the same eye disease;

[0025] When Sz≤S, it is determined that the eye disease image and the eye disease represented by the i-th type of disease identification information are different eye diseases.

[0026] As another specific embodiment of the present invention, the process of generating the data of the filtering mode, the width of the projection light source, and the irradiation angle required for photographing the lesion with a slit lamp in step 4) is as follows:

[0027] Based on the relationship between the type of eye lesions and the initial filtering method, the width of the initial projection light source and the initial irradiation angle, a control matrix Ki (Ki1, Ki2, Ki3, Ki4, i=1, 2...n) is established, where Ki1 represents the type of eye lesions, Ki2 represents the initial filtering method, Ki3 represents the width of the initial projection light source, and Ki4 represents the initial irradiation angle. Once the eye lesion type Ki1 is determined, the initial filtering method, the width of the initial projection light source and the initial irradiation angle can be determined.

[0028] As another specific implementation of the present invention, it further includes step 5) collecting the eye image again, and correcting the width and illumination angle of the projection light source based on the clarity and chromaticity of the re-collected eye image.

[0029] As another specific implementation of the present invention, the camera device in step 1) includes a wide-angle camera and an infrared camera.

[0030] The present invention also provides a slit lamp that implements the above-mentioned slit lamp self-adjustment control method based on machine vision.

[0031] The present invention has the following beneficial effects:

[0032] The present invention captures image information through a camera device, which is convenient for capturing the eye lesion area. At the same time, the moving coordinates and the irradiation angle when the microscope is aimed at the lesion area are calculated, which is convenient for doctors to quickly view the lesion area and determine the type of disease according to a preset algorithm. The slit lamp's filtering mode, the width of the projection light source and the irradiation angle are adjusted accordingly according to the disease type to obtain higher image quality, which is convenient for doctors to quickly align and correct the slit lamp to the lesion area.

[0033] In addition, after the present invention identifies the lesion area, the operating data of the slit lamp is adjusted accordingly according to the type of the lesion, thereby ensuring that the slit lamp imaging effect is convenient for doctors to diagnose diseases.

[0034] The present invention will be further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic diagram of the process of the present invention;

[0036] Figure 2 It is a schematic diagram of the structure of a slit lamp that can be used in the present invention;

[0037] Figure 3 It is a photo taken by the present invention through a wide-angle camera;

[0038] Figure 4 It is a photo taken by the present invention through an infrared camera;

[0039] Figure 5 It is a photograph showing the lesion area of ​​the present invention. DETAILED DESCRIPTION

[0040] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0042] Example 1

[0043] This embodiment provides a slit lamp self-adjustment control method based on machine vision, such as Figure 1-2 As shown, the slit lamp has an illumination system and a microscope system. This embodiment does not limit the specific structure of the slit lamp, and is intended to control multiple parameters of the slit lamp (irradiation angle, light source width, filtering method, etc.) to obtain the best use effect.

[0044] It includes the following steps:

[0045] 1) Using a camera device to collect eye images;

[0046] The camera device includes a wide-angle camera and an infrared camera, and the wide-angle camera and the infrared camera are used to collect eye images. Figure 3-4 As shown ( Figure 3 The colors to be displayed in the image are processed in grayscale), wherein the wide-angle camera and the infrared camera work simultaneously, the wide-angle camera receives visible light to determine the lesions that are easily observed under visible light, and the infrared camera receives infrared light to determine the lesions that are easily observed under infrared light.

[0047] 2) Compare the collected eye image with the standard data in the database to determine the eye lesion area, and mark the lesion area;

[0048] The standard database is established based on the healthy eye database, and the eye lesion area is obtained through image comparison technology analysis, such as Figure 5 As shown, the boxed area is the observed lesion area of ​​the suspected lesion.

[0049] Specifically, after acquiring the collected eye image, the collected eye image is first binarized, and the lesion area is determined by presetting a contrast threshold in the processor. Specifically, the area where the grayscale change exceeds the preset contrast threshold is marked as the lesion area.

[0050] 3) Establishing a two-dimensional coordinate system with the center of the pupil as the origin to obtain the coordinates of the lesion area;

[0051] like Figure 5 As shown, according to the location of the suspected lesion in the photo, the (X, Y) coordinates of the lesion area can be obtained, and according to the position difference of the images taken by the wide-angle camera and the infrared camera for the same part, the (Z) coordinate of the lesion area can be determined.

[0052] 4) Generate control data according to the coordinates of the lesion area, control the slit lamp to obtain the best shooting angle, determine the type of eye lesions according to the lesion image of the marked lesion area, and generate data on the filtering method, width of the projection light source, and irradiation angle required for the slit lamp to photograph the lesion;

[0053] The process of generating manipulation data according to the coordinates of the lesion area is as follows:

[0054] A two-dimensional coordinate set f(x, y) of the lesion area is established according to the acquired coordinates of the lesion area;

[0055] According to the relationship between the position and shooting angle of the microscope and the lesion area, a coordinate set adjustment matrix Si (Si1, Si2, Si3, Si4, i=1, 2...n) is established, where Si1 represents the coordinate area, Si2 represents the X-axis coordinate adjusted by the microscope, Si3 represents the Y-axis coordinate adjusted by the microscope, and Si4 represents the adjustment angle of the microscope;

[0056] Determine whether the area represented by the two-dimensional coordinate set f(x, y) of the lesion area exists in the coordinate area Si1. If so, call the microscope to adjust the X-axis coordinate Si2, adjust the Y-axis coordinate Si3 and adjust the microscope angle Si4, and prompt that the microscope position should be adjusted to the X-axis coordinate Si1 and the Y-axis coordinate Si2, and the angle should be adjusted to the microscope adjustment angle Si4.

[0057] That is to say, once the coordinates of the lesion area are obtained, the X-axis coordinates, Y-axis coordinates and adjustment angle data of the microscope system corresponding to the coordinates of the lesion area can be obtained through the relevant data pre-set in the processor (i.e., matrix Si).

[0058] The process of determining the eye lesion type of the lesion image of the marked lesion area is as follows:

[0059] Eye images of various types of eye diseases are selected, and specific information on the lesion image in each lesion area is extracted as identification information through the Ai algorithm, and then a determination matrix P (P1, P2...Pn) that can determine the lesion image corresponding to each type of eye disease is generated and stored, wherein P1 represents one type of disease identification information, P2 represents the second type of disease identification information, and Pn represents the nth type of disease identification information. The lesion image of the marked lesion area is compared one by one with the determination matrix to determine the type of eye disease corresponding to the lesion image of the marked lesion area.

[0060] Specifically, the process of generating the decision matrix P can be:

[0061] First, select an eye lesion image of a type of eye disease, extract the characteristic information of the lesion area in the eye lesion image through the Ai algorithm, generate and store identification information that can determine the type of disease;

[0062] The above process is repeated to generate identification information of various types of lesion areas, and then generate a decision matrix P corresponding to the lesion images of various eye diseases.

[0063] The process of generating the data of the filtering mode, the width of the projection light source, and the irradiation angle required for the slit lamp to photograph the lesion is as follows:

[0064] Based on the relationship between the type of eye lesions and the initial filtering method, the width of the initial projection light source and the initial irradiation angle, a control matrix Ki (Ki1, Ki2, Ki3, Ki4, i=1, 2...n) is established, where Ki1 represents the type of eye lesions, Ki2 represents the initial filtering method, Ki3 represents the width of the initial projection light source, and Ki4 represents the initial irradiation angle. Once the eye lesion type Ki1 is determined, the initial filtering method, the width of the initial projection light source and the initial irradiation angle can be determined.

[0065] For example, when the disease is determined to be type 1 disease, the processor can determine that the initial filtering mode is K12, the initial projection light source width is K13, and the initial irradiation angle is K14;

[0066] A specific process of the Ai algorithm is as follows:

[0067] Transformer and Unet are integrated into one model to achieve anterior segment image segmentation and lesion recognition. The encoder module of U-net uses vision transformer for encoding. Transformer encodes tokenized image blocks from convolutional neural network (CNN) feature maps into input sequences for extracting global context. The decoder upsamples the encoded features and then combines them with high-resolution CNN feature maps to achieve precise positioning.

[0068] The specific information on the lesion image in the lesion area used in this embodiment is the contour features and the chromaticity features. In order to facilitate the image comparison processing, the eye image collected in step 1) is subjected to data extraction and the collected eye image is divided into eye corner image, eye pupil image, eyelid image and eyelid image. The image information of the eye lesion area in each type of image is processed by the controller, and the directionality in the image processing process is stronger.

[0069] Furthermore, when determining the type of eye lesions according to the lesion image of the marked lesion area in step 4), the lesion image is divided into a lesion edge area and a lesion center area, and the contour features and chromaticity features of the lesion edge area and the lesion center area are compared with the i-th (i=1, 2...n) type of disease identification information in the determination matrix P, and the matching areas and non-matching areas are recorded to determine the matching rate Sz of the lesion image:

[0070]

[0071] Where S1 represents the contour feature matching rate of the lesion edge area, S2 represents the contour feature matching rate of the lesion middle area, S3 represents the chromaticity matching rate of the lesion edge area, S4 represents the chromaticity matching rate of the lesion center area, α1 is the weight parameter of S1 to Sz, α2 is the weight parameter of S2 to Sz, α3 is the weight parameter of S3 to Sz, and α4 is the weight parameter of S4 to Sz;

[0072] Compare the matching rate Sz of the lesion image with the set matching rate parameter S of the eye disease image:

[0073] When Sz>S, it is determined that the eye disease image and the eye disease represented by the i-th type of disease identification information are the same eye disease;

[0074] When Sz≤S, it is determined that the eye disease image and the eye disease represented by the i-th type of disease identification information are different eye diseases.

[0075] In this embodiment, after determining the initial filtering mode, the initial projection light source width and the initial irradiation angle, the method further includes:

[0076] Step 5): The eye image is collected again, and the width and illumination angle of the projection light source are corrected based on the clarity and chromaticity of the eye image collected again. A specific correction process is:

[0077] Establish the brightness matrix C0 and the light source width adjustment parameter matrix D0;

[0078] For the brightness matrix C0, C0 (C1, C2, C3, C4), C1 is the first preset brightness parameter, C2 is the second preset brightness parameter, C3 is the third preset brightness parameter, and C4 is the fourth preset brightness parameter, and each preset brightness parameter increases in sequence;

[0079] For the light source width adjustment parameter matrix D0, D0 (D1, D2, D3, D4), where D1 is the first preset light source width adjustment parameter, D2 is the second preset light source width adjustment parameter, D3 is the third preset light source width adjustment parameter, and D4 is the fourth preset light source width adjustment parameter;

[0080] The processor detects the brightness Ci of the i-th area of ​​the eye image (the i-th area can be any area, and in most cases it refers to the lesion area or the area suspected of having a lesion) and compares Ci with the parameters in the C0 matrix:

[0081] When Ci≤C1, it is determined that the brightness of the i-th area is insufficient and D1 is selected from the D0 matrix as the light source width adjustment parameter;

[0082] When C1<Ci≤C2, it is determined that the brightness of the i-th area is insufficient and D2 is selected from the D0 matrix as the light source width adjustment parameter;

[0083] When C2<Ci≤C3, it is determined that the brightness of the i-th area is in the standard state;

[0084] When C3<Ci≤C4, it is determined that the brightness of the i-th area is too high and D3 is selected from the D0 matrix as the light source width adjustment parameter;

[0085] When Ci>C4, it is determined that the brightness of the i-th region is too high and D4 is selected from the D0 matrix as the light source width adjustment parameter;

[0086] When it is determined that the brightness of the i-th area is not in the standard state (the standard state is the state of C2<Ci≤C3), the width of the light source of the i-th area is adjusted to Ci';

[0087] When it is determined that the width of the light source in the i-th region is insufficient, Ci'=Ci+(C3-Ci)×Dj, j=1, 2;

[0088] When it is determined that the width of the light source in the i-th region is too high, Ci'=Ci-(Ci-C2)×Dr, r=3, 4;

[0089] When the width of the light source in the i-th region is adjusted to Ci', compare Ci' with the parameters in the C0 matrix:

[0090] When C2<Ci'≤C3, it is determined that the brightness of the i-th area is in the standard state;

[0091] When Ci' is not in C2-C3, the above operation is repeated until C2<Ci'≤C3.

[0092] Although the present invention is disclosed as above with preferred embodiments, it is not intended to limit the scope of the present invention. Any person skilled in the art can make some improvements without departing from the scope of the present invention, that is, all equivalent improvements made according to the present invention should be covered by the scope of the present invention.

Claims

1. A slit lamp self-adjustment control method based on machine vision, comprising the following steps: 1) Using a camera device to collect eye images; 2) Compare the collected eye images with the standard data in the database to determine the eye lesion area and mark the lesion area; 3) Establish a two-dimensional coordinate system with the center of the pupil as the origin to obtain the coordinates of the lesion area; 4) Generate control data according to the coordinates of the lesion area, control the slit lamp to obtain the best shooting angle, determine the type of eye lesions according to the lesion image of the marked lesion area, and generate data on the filtering method, the width of the projection light source, and the irradiation angle required for the slit lamp to shoot the lesion; the specific information on the lesion image in the lesion area in step 4) includes contour features and chromaticity features; when determining the type of eye lesions according to the lesion image of the marked lesion area in step 4), the lesion image is divided into a lesion edge area and a lesion center area, and the contour features and chromaticity features of the lesion edge area and the lesion center area are compared with the i-th disease identification information in the determination matrix P, i=1, 2···n, and the matching areas and non-matching areas are recorded to determine the matching rate Sz of the lesion image: Where S1 represents the contour feature matching rate of the lesion edge area, S2 represents the contour feature matching rate of the lesion middle area, S3 represents the chromaticity matching rate of the lesion edge area, S4 represents the chromaticity matching rate of the lesion center area, 𝛼1 is the weight parameter of S1 to Sz, 𝛼2 is the weight parameter of S2 to Sz, 𝛼3 is the weight parameter of S3 to Sz, and 𝛼4 is the weight parameter of S4 to Sz; Compare the matching rate Sz of the lesion image with the set matching rate parameter S of the eye disease image: When Sz>S, it is determined that the eye disease image and the eye disease represented by the i-th type of disease identification information are the same eye disease; When Sz≤S, it is determined that the eye disease image and the eye disease represented by the i-th type of disease identification information are different eye diseases; The process of determining the eye lesion type of the lesion image of the marked lesion area is as follows: Eye images of various types of eye diseases are selected, and specific information on the lesion image in each lesion area is extracted as identification information through an algorithm, and then a determination matrix P that can determine the lesion image corresponding to each type of eye disease is generated and stored. P includes P1, P2...Pn, where P1 represents one type of disease identification information, P2 represents the second type of disease identification information, and Pn represents the nth type of disease identification information. The lesion image of the marked lesion area is compared one by one with the determination matrix to determine the type of eye disease corresponding to the lesion image of the marked lesion area.

2. The slit lamp self-adjustment control method based on machine vision according to claim 1, wherein: Data extraction is performed on the eye image collected in step 1) to divide the collected eye image into an eye canthus image, an eye pupil image, an eyelid image and an eyelid area image.

3. The slit lamp self-adjustment control method based on machine vision according to claim 1, wherein: In step 2), the collected eye image is first binarized, a contrast threshold is preset, and the area where the grayscale change exceeds the preset contrast threshold is marked as the lesion area.

4. The slit lamp self-adjustment control method based on machine vision according to claim 1, wherein: The process of generating manipulation data according to the coordinates of the lesion area in step 4) is as follows: Establish a two-dimensional coordinate set f (x, y) of the lesion area according to the acquired coordinates of the lesion area; According to the relationship between the position and shooting angle of the microscope and the lesion area, a coordinate set adjustment matrix Si is established, where Si includes Si1, Si2, Si3, and Si4, wherein Si1 represents the coordinate area, Si2 represents the X-axis coordinate adjusted by the microscope, Si3 represents the Y-axis coordinate adjusted by the microscope, and Si4 represents the adjustment angle of the microscope; Determine whether the area represented by the two-dimensional coordinate set f (x, y) of the lesion area exists in the coordinate area Si1. If so, call the microscope to adjust the X-axis coordinate Si2, adjust the Y-axis coordinate Si3 and adjust the microscope angle Si4, and prompt that the microscope position should be adjusted to the X-axis coordinate Si1 and the Y-axis coordinate Si2, and the angle should be adjusted to the microscope adjustment angle Si4.

5. The slit lamp self-adjustment control method based on machine vision according to claim 1, wherein: The process of generating the data of the filter mode, the width of the projection light source, and the irradiation angle required for photographing the lesion with a slit lamp in step 4) is as follows: Based on the relationship between the type of eye lesions and the initial filtering method, the width of the initial projection light source and the initial irradiation angle, a control matrix Ki is established. Ki includes Ki1, Ki2, Ki3 and Ki4, where Ki1 represents the type of eye lesions, Ki2 represents the initial filtering method, Ki3 represents the width of the initial projection light source and Ki4 represents the initial irradiation angle. Once the eye lesion type Ki1 is determined, the initial filtering method, the width of the initial projection light source and the initial irradiation angle can be determined.

6. The slit lamp self-adjustment control method based on machine vision according to claim 5, wherein: The method further includes step 5) collecting the eye image again, and correcting the width and illumination angle of the projection light source based on the clarity and chromaticity of the re-collected eye image.

7. The slit lamp self-adjustment control method based on machine vision according to claim 1, wherein: The camera device in step 1) includes a wide-angle camera and an infrared camera.

8. A slit lamp that implements the slit lamp self-adjustment control method based on machine vision as described in any one of claims 1 to 7.

Images