Training method of keratitis automatic identification model for low-quality slit lamp image

Abstract

The invention provides a training method of a keratitis automatic identification model for a low-quality slit lamp image, and relates to the technical field of artificial intelligence, and the method comprises the steps: collecting a plurality of first slit lamp image samples; for each first slit lamp image sample, using an image quality degradation algorithm to generate a second slit lamp image sample, and pairing the first slit lamp image sample and the second slit lamp image sample to form a training data pair; based on each training data pair, training the initial slit lamp image quality enhancement sub-model to obtain a slit lamp image quality enhancement sub-model; acquiring a plurality of third slit lamp image samples; respectively inputting each third slit lamp image sample into a slit lamp image quality enhancement sub-model to obtain a fourth slit lamp image sample corresponding to each third slit lamp image sample output by the slit lamp image quality enhancement sub-model; and based on each fourth slit lamp image sample, training the initial keratitis identification sub-model to obtain a keratitis identification sub-model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a training method for an automatic keratitis recognition model for low-quality slit lamp images. Background Art

[0002] Keratitis is one of the leading causes of blindness worldwide. Its occurrence and development pose a serious threat to human vision health and quality of life. The traditional method of relying on manual diagnosis by ophthalmologists has disadvantages such as time-consuming, labor-intensive and subjective differences, which limits the popularity and accessibility of keratitis diagnosis in resource-scarce and remote areas. With the continuous accumulation of deep learning technology and medical imaging, automatic diagnosis of keratitis based on artificial intelligence algorithms has become possible, and has promoted the rapid development of technology in this field.

[0003] Slit lamp images taken by a slit lamp microscope are the main medium for diagnosing keratitis. Existing automatic diagnosis technology for keratitis relies on high-quality slit lamp image data. Before model training, ophthalmologists inspect and annotate the collected slit lamp images, screen out low-quality images such as motion blur, defocus blur, and exposure distortion, and then analyze the high-quality slit lamp image data after excluding the low-quality images.

[0004] Prior art 1 is a solution for preventing corneal blindness caused by keratitis using artificial intelligence. The solution realizes automatic diagnosis of keratitis based on a deep convolutional neural network (CNN), and verifies the effectiveness of the algorithm on high-quality slit lamp images from three different hospitals and a smartphone image.

[0005] Prior art 2 is a clinical image grading algorithm for infectious keratitis based on a deep sequential feature learning model (Deep sequential feature learning in clinical image classification of infectious keratitis). It uses CNN to extract high-level features of randomly cropped slit lamp image blocks, and inputs them into a time series model to mine the correlation between features of different image blocks to achieve grading of bacterial, fungal and viral keratitis.

[0006] Prior art 3 is an automatic diagnosis of keratitis using object localization combined with cost-sensitive deep attention convolutional neural network. The constructed target localization algorithm crops the corneal and conjunctival area from the slit lamp image, and inputs the cropped area as the region of interest into the cost-sensitive deep attention convolutional neural network to realize automatic diagnosis of keratitis.

[0007] Prior art 4 (patent number CN117877103A) is an intelligent screening method for keratitis based on deep meta-learning. It collects corneal photographs with labeled corneal classifications, builds a corneal-assisted screening model based on the deep learning classification algorithm ConvNeXt, inputs slit lamp corneal and conjunctival area images into the corneal-assisted screening model, extracts ocular surface features of keratitis in the corneal and conjunctival areas, and trains the corneal-assisted screening model; inputs Canton slit lamp corneal photographs and mobile device corneal photographs into the trained corneal-assisted screening model, combines meta-learning strategies, extracts ocular surface features of keratitis in corneal photographs, trains the corneal-assisted screening model again, and updates the model by backpropagation according to the training results until convergence.

[0008] However, the above-mentioned existing methods are all based on the training of automatic diagnosis models based on high-quality slit lamp images, so that the automatic diagnosis model does not learn the features of low-quality images, and the deployed model cannot diagnose low-quality images. However, in the actual clinical examination process, there may be factors such as improper operation of ophthalmologists, non-cooperation of patients, or interference from the external environment, which leads to the inevitable existence of low-quality slit lamp images. The existing methods cannot accurately identify low-quality slit lamp images, which limits the application and promotion of existing methods in actual clinical practice. Their generalization ability is generally low, which in turn reduces their diagnostic accuracy in real clinical applications. In addition, low-quality slit lamp images can also lead to misdiagnosis by doctors or increase multiple redundant sampling of patients, increasing the burden on the medical system. If low-quality slit lamp images are included in the training data set of the model, although it helps the model to identify low-quality slit lamp images, since low-quality slit lamp images contain a lot of noise and unrecognizable features, they will inevitably interfere with the model's recognition effect on high-quality images, thereby reducing the overall performance of the system. Therefore, how to accurately and effectively identify the corneal state corresponding to low-quality slit lamp images is a problem that needs to be solved urgently. Summary of the invention

[0009] In view of the problems existing in the prior art, an embodiment of the present invention provides a training method for an automatic keratitis recognition model for low-quality slit lamp images.

[0010] In a first aspect, the present invention provides a training method for a keratitis automatic recognition model for low-quality slit lamp images, the keratitis automatic recognition model comprising: a target slit lamp image quality enhancement sub-model for repairing noise in low-quality slit lamp images, and a target keratitis recognition sub-model for identifying a corneal state corresponding to the slit lamp image;

[0011] The training method comprises:

[0012] Collecting a plurality of first slit lamp image samples that have been labeled as high-quality image quality categories; for each of the first slit lamp image samples, using an image quality degradation algorithm to generate a second slit lamp image sample, and pairing the first slit lamp image sample with the second slit lamp image sample to form a training data pair, wherein the image quality category of the second slit lamp image sample is lower than that of the first slit lamp image sample;

[0013] Based on each of the training data pairs, the initial slit lamp image quality enhancer model is trained until convergence to obtain the target slit lamp image quality enhancer model; wherein the initial slit lamp image quality enhancer model includes a SCUNet generation model formed by fusing a Swin-Conv module and a U-Net framework;

[0014] Collect a plurality of third slit lamp image samples that have been marked as low-quality image quality categories; input each of the third slit lamp image samples into the target slit lamp image quality enhancement sub-model, respectively, to obtain a fourth slit lamp image sample corresponding to each of the third slit lamp image samples output by the target slit lamp image quality enhancement sub-model, wherein the image quality category of the fourth slit lamp image sample is higher than that of the third slit lamp image sample;

[0015] Based on each of the fourth slit lamp image samples, the initial keratitis recognition sub-model is trained until convergence to obtain the target keratitis recognition sub-model; wherein the initial keratitis recognition sub-model is a dense convolutional neural network fused with a Gaussian context transformer GCT module.

[0016] Optionally, for each of the first slit lamp image samples, generating a second slit lamp image sample using an image quality degradation algorithm comprises:

[0017] Based on the image quality degradation algorithm, performing at least one of motion blur processing, defocus blur processing and exposure distortion processing on each of the first slit lamp image samples to obtain a second slit lamp image sample;

[0018] The motion blur processing is implemented by the following formula (1):

[0019]

[0020] The defocus blur processing is implemented by the following formula (2):

[0021]

[0022] The exposure distortion processing is achieved by the following formula (3):

[0023] I exposure (x, y) = α·I(x, y) + β (3)

[0024] Wherein, I(x, y) represents the first slit lamp image sample; I motion (x, y), I defocus (x, y) and I exposure (x, y) respectively represent the second slit lamp image sample generated by using the image quality degradation algorithm on the first slit lamp image sample; (x, y) represent the coordinates of the first slit lamp image sample; K(x, y) represents the motion blur kernel; (i, j) represents the discrete coordinates iterated to the motion blur kernel; (k, l) represents the standard deviation of defocus blur; α and β represent the adjustment coefficients of contrast and brightness.

[0025] Optionally, the initial slit lamp image quality enhancer model also includes a discriminator;

[0026] The initial slit lamp image quality enhancer model is trained based on each of the training data pairs until convergence to obtain the target slit lamp image quality enhancer model, including:

[0027] Step A: Based on each of the training data pairs, the SCUNet generation model is trained so that the SCUNet generation model learns the mapping relationship between the second slit lamp image sample and the first slit lamp image sample from each of the training data pairs, and generates a fifth slit lamp image sample with enhanced image quality corresponding to each of the second slit lamp image samples based on the mapping relationship;

[0028] Step B: inputting the first slit lamp image sample and the fifth slit lamp image sample into the discriminator, so that the discriminator distinguishes the difference between the fifth slit lamp image sample and the corresponding first slit lamp image sample, and adjusts the model parameters of the SCUNet generation model based on the difference;

[0029] Step C: looping through steps AB until the SCUNet generated model converges to obtain the target slit lamp image quality enhancement sub-model.

[0030] Optionally, the SCUNet generation model is formed by fusing the Swin-Conv module and the U-Net framework;

[0031] The Swin-Conv module includes a first convolutional layer, a second convolutional layer, a Swin-Transformer module and a residual convolutional module, and the first convolutional layer is residually connected to the second convolutional layer;

[0032] The U-Net framework includes a strided convolution module, a transposed convolution module, a plurality of Swin-Conv modules and a convolution layer, and the strided convolution module is residually connected with the transposed convolution module.

[0033] Optionally, the training of the initial keratitis recognition sub-model based on each of the fourth slit lamp image samples until convergence to obtain the target keratitis recognition sub-model comprises:

[0034] Inputting each of the fourth slit lamp image samples into the initial keratitis recognition sub-model for iterative training until the loss value of the target loss function reaches a preset threshold, thereby obtaining the target keratitis recognition sub-model;

[0035] The objective loss function is expressed by the following formula (4):

[0036]

[0037] Among them, loss(P, Q, W) represents the loss value, P i Indicates that the initial keratitis recognition sub-model outputs the probability of the i-th corneal state category, Q i represents the probability that the true label of the fourth slit lamp image sample is the i-corneal state category, W i Represents the error weight set for the i-th corneal condition category.

[0038] Optionally, the method further comprises:

[0039] Inputting each of the first slit lamp image samples into the initial keratitis identification sub-model for iterative training until the loss value of the target loss function reaches a preset threshold, thereby obtaining the initial weight parameters of the target keratitis identification sub-model;

[0040] The initial keratitis recognition sub-model is trained based on each of the fourth slit lamp image samples until convergence to obtain the target keratitis recognition sub-model, including:

[0041] Loading the initial weight parameters obtained through training based on each of the first slit lamp image samples into the initial keratitis recognition sub-model;

[0042] Based on each of the fourth slit lamp image samples, the initial keratitis identification sub-model is trained for a second time until the loss value of the target loss function reaches a preset threshold, thereby obtaining the target keratitis identification sub-model.

[0043] In a second aspect, the present invention further provides a method for automatically identifying keratitis in low-quality slit lamp images, comprising:

[0044] Acquire slit lamp images of the patient's eyes to be identified;

[0045] In the case that the slit lamp image to be identified is of a low-quality image quality category, the slit lamp image to be identified is input into a target slit lamp image quality enhancement sub-model to obtain an enhanced slit lamp image output by the target slit lamp image quality enhancement sub-model; the target slit lamp image quality enhancement sub-model is used to repair noise existing in the low-quality slit lamp image;

[0046] Inputting the enhanced slit lamp image into a target keratitis recognition sub-model to obtain a corneal state corresponding to the slit lamp image to be recognized output by the target keratitis recognition sub-model; the target keratitis recognition sub-model is used to recognize the corneal state corresponding to the slit lamp image;

[0047] The target slit lamp image quality enhancement sub-model and the target keratitis recognition sub-model are trained by the training method of the automatic keratitis recognition model for low-quality slit lamp images as described in the first aspect.

[0048] Optionally, after acquiring the slit lamp image to be identified of the patient's eye, the method further comprises:

[0049] Inputting the slit lamp image to be identified into an image quality classification model to obtain an image quality category corresponding to the slit lamp image to be identified output by the image quality classification model;

[0050] When the image quality category is high quality, the slit lamp image to be identified is input into the target keratitis identification sub-model to obtain the corneal state corresponding to the slit lamp image to be identified.

[0051] In a third aspect, the present invention provides a training device for a keratitis automatic recognition model for low-quality slit lamp images, wherein the keratitis automatic recognition model comprises: a target slit lamp image quality enhancement sub-model for repairing noise in low-quality slit lamp images, and a target keratitis recognition sub-model for recognizing a corneal state corresponding to the slit lamp image;

[0052] The training device comprises:

[0053] A first acquisition module is used to acquire a plurality of first slit lamp image samples that have been marked as high-quality image quality categories; for each of the first slit lamp image samples, use an image quality degradation algorithm to generate a second slit lamp image sample, and pair the first slit lamp image sample with the second slit lamp image sample to form a training data pair, wherein the image quality category of the second slit lamp image sample is lower than that of the first slit lamp image sample;

[0054] A first training module is used to train the initial slit lamp image quality enhancer model based on each of the training data pairs until convergence, so as to obtain the target slit lamp image quality enhancer model; wherein the initial slit lamp image quality enhancer model includes a SCUNet generation model formed by fusing a Swin-Conv module and a U-Net framework;

[0055] A second acquisition module is used to acquire a plurality of third slit lamp image samples that have been marked as low-quality image quality categories; each of the third slit lamp image samples is input into the target slit lamp image quality enhancement sub-model, and a fourth slit lamp image sample corresponding to each of the third slit lamp image samples output by the target slit lamp image quality enhancement sub-model is obtained, wherein the image quality category of the fourth slit lamp image sample is higher than that of the third slit lamp image sample;

[0056] The second training module is used to train the initial keratitis recognition sub-model based on each of the fourth slit lamp image samples until convergence to obtain the target keratitis recognition sub-model; wherein the initial keratitis recognition sub-model is a dense convolutional neural network fused with a Gaussian context transformer GCT module.

[0057] In a fourth aspect, the present invention provides an automatic keratitis identification device for low-quality slit lamp images, characterized in that it comprises:

[0058] A third acquisition module is used to acquire a slit lamp image of the patient's eye to be identified;

[0059] A slit lamp image enhancement module is used for inputting the slit lamp image to be identified into a target slit lamp image quality enhancement sub-model when the slit lamp image to be identified is of a low-quality image quality category, so as to obtain an enhanced slit lamp image output by the target slit lamp image quality enhancement sub-model; the target slit lamp image quality enhancement sub-model is used for repairing noise existing in the low-quality slit lamp image;

[0060] A first corneal state recognition module is used to input the enhanced slit lamp image into a target keratitis recognition sub-model to obtain a corneal state corresponding to the slit lamp image to be recognized output by the target keratitis recognition sub-model; the target keratitis recognition sub-model is used to recognize the corneal state corresponding to the slit lamp image;

[0061] Among them, the target slit lamp image quality enhancement sub-model and the target keratitis recognition sub-model are trained by the training method of the keratitis automatic recognition model for low-quality slit lamp images as described in the first aspect above.

[0062] In a fifth aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for training a keratitis automatic recognition model for low-quality slit lamp images as described in the first aspect above, or the method for automatically recognizing keratitis for low-quality slit lamp images as described in the second aspect above is implemented.

[0063] In a sixth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method for an automatic keratitis recognition model for low-quality slit lamp images as described in the first aspect above, or the automatic keratitis recognition method for low-quality slit lamp images as described in the second aspect above.

[0064] In the seventh aspect, the present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the training method of the automatic keratitis recognition model for low-quality slit lamp images as described in the first aspect above, or the automatic keratitis recognition method for low-quality slit lamp images as described in the second aspect above.

[0065] The present invention provides a training method for an automatic keratitis recognition model for low-quality slit lamp images. By collecting multiple first slit lamp image samples that have been labeled as high-quality image quality categories, and using an image quality degradation algorithm to process each first slit lamp image sample, a variety of slit lamp image degradation conditions corresponding to the first slit lamp image sample can be simulated, thereby creating a second slit lamp image sample of a wide range of low-quality image quality categories covering real clinical scenarios, ensuring the diversity of low-quality slit lamp image samples; the first slit lamp image sample and the corresponding second slit lamp image sample are paired to form a training data pair, and based on the multiple training data pairs, an initial slit lamp image quality enhancer model is trained until convergence, which can improve the robustness of model training and help the initial slit lamp image quality enhancer model learn in training that the first slit lamp image sample in each training data pair degrades to the second slit lamp image Various complex noise and degradation conditions of the samples are taken into consideration, so that the model can enhance and restore the slit lamp images of low image quality to slit lamp images of high image quality in a real clinical environment; after obtaining the trained target slit lamp image quality enhancement sub-model, multiple third slit lamp image samples marked as low quality image quality categories are input into the target slit lamp image quality enhancement sub-model to obtain the fourth slit lamp image samples after the image quality of each third slit lamp image sample is enhanced; based on each fourth slit lamp image sample, the initial keratitis recognition sub-model is trained until convergence, so that the initial keratitis recognition sub-model can fully learn the feature information in the fourth slit lamp image sample enhanced and restored by the target slit lamp image quality enhancement sub-model, so that in practical applications, the corneal state corresponding to the slit lamp image enhanced and restored by the target slit lamp image quality enhancement sub-model can be accurately and effectively identified.

[0066] Furthermore, since the slit-lamp image samples of the low-quality image quality category do not directly participate in the training of the initial keratitis recognition sub-model, the trained target keratitis recognition sub-model can not only realize the accurate recognition of slit-lamp images of different low-quality image quality categories in a real clinical environment, but also ensure the recognition effect of slit-lamp images of the high-quality image quality category. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0068] Figure 1It is one of the flow charts of the training method of the automatic keratitis recognition model for low-quality slit lamp images provided by the present invention;

[0069] Figure 2 It is a structural diagram of the SCUNet generation model provided by the present invention;

[0070] Figure 3 It is a schematic diagram of the structure of the discriminator provided by the present invention;

[0071] Figure 4 It is a schematic diagram of the structure of the GCT-DenseNet121 model provided by the present invention;

[0072] Figure 5 It is a logical schematic diagram of extracting global features of slit lamp images based on the GCT module provided by the present invention;

[0073] Figure 6 This is the second flow chart of the training method of the automatic keratitis recognition model for low-quality slit lamp images provided by the present invention;

[0074] Figure 7 It is a flow chart of the automatic keratitis recognition method for low-quality slit lamp images provided by the present invention;

[0075] Figure 8 It is a structural schematic diagram of a training device for an automatic keratitis recognition model for low-quality slit lamp images provided by the present invention;

[0076] Fig. 9 It is a schematic structural diagram of a keratitis automatic identification device for low-quality slit lamp images provided by the present invention;

[0077] Fig.10 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0078] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] In order to facilitate a clearer understanding of the embodiments of the present application, the relevant knowledge is first introduced as follows.

[0080] Eye diseases such as keratitis, conjunctivitis, eyelid diseases, and iris diseases are one of the leading causes of blindness worldwide. Their occurrence and development pose a serious threat to human vision health and quality of life. If not diagnosed in time, eye diseases such as keratitis will cause corneal ulcers, corneal scars, and even permanent vision damage or blindness. The traditional method of relying on ophthalmologists to manually diagnose eye diseases (for ease of description, keratitis is used as an example below) has the disadvantages of being time-consuming, labor-intensive, and highly subjective, which limits the popularity and accessibility of keratitis diagnosis in resource-scarce and remote areas. The continuous accumulation of deep learning technology and medical imaging has made it possible to automatically identify keratitis based on artificial intelligence algorithms, and has promoted the rapid development of technology in this field.

[0081] However, the automatic keratitis recognition methods based on artificial intelligence algorithms are all trained on high-quality slit lamp images. Although these automatic diagnosis models achieved satisfactory recognition performance for keratitis diagnosis in the experimental stage, they did not learn the features of low-quality images, and the deployed models were unable to diagnose low-quality images.

[0082] In the actual clinical examination process, low-quality slit lamp images are inevitably present due to improper operation of ophthalmologists, non-cooperation of patients, and interference from the external environment. These low-quality images limit the application and promotion of existing artificial intelligence methods in actual clinical practice, and their generalization ability is generally low, which in turn reduces their diagnostic accuracy in real clinical applications. Low-quality slit lamp images can also lead to misdiagnosis by doctors or increase multiple redundant sampling of patients, increasing the burden on the medical system.

[0083] In addition, if low-quality slit lamp images are directly used in the training of the automatic diagnosis model, although it helps to identify low-quality images, since low-quality images contain a lot of noise and unrecognizable features, it will inevitably interfere with the model's recognition effect on high-quality images, thereby reducing the overall performance of the system.

[0084] In view of the above problems, there is an urgent need for an automatic keratitis recognition model that is compatible with low-quality slit-lamp images, which can be used to accurately and effectively identify the corneal state corresponding to the low-quality slit-lamp images.

[0085] Combine the following Figures 1 to 5 The training method of the automatic keratitis recognition model for low-quality slit lamp images provided by the present invention is described in detail. Figure 1 This is one of the flow charts of the training method of the automatic keratitis recognition model for low-quality slit lamp images provided by the present invention, see Figure 1As shown, the keratitis automatic recognition model includes: a target slit lamp image quality enhancement sub-model for repairing noise in low-quality slit lamp images, and a target keratitis recognition sub-model for identifying the corneal state corresponding to the slit lamp image.

[0086] The training method comprises steps 101 to 105, wherein:

[0087] Step 101: Collect a plurality of first slit lamp image samples that have been labeled as high-quality image quality categories; for each of the first slit lamp image samples, use an image quality degradation algorithm to generate a second slit lamp image sample, and pair the first slit lamp image sample and the second slit lamp image sample to form a training data pair, wherein the image quality category of the second slit lamp image sample is lower than that of the first slit lamp image sample.

[0088] It should be noted that the executor of the present invention can be any electronic device that can realize the training of the automatic recognition model of keratitis, for example, it can be any one of a smart phone, a smart watch, a desktop computer, a laptop computer, etc.

[0089] The training method of the keratitis automatic recognition model for low-quality slit lamp images provided by the present invention is mainly used to train a keratitis recognition model for low-quality slit lamp images, aiming to improve the performance of keratitis recognition by improving the quality of slit lamp images. That is, the keratitis automatic recognition model is mainly used to recognize the corneal state of keratitis in the low-quality slit lamp images of patients, and the corneal state includes: normal cornea, keratitis and other corneal abnormalities. The keratitis recognition model includes a target slit lamp image quality enhancement sub-model and a target keratitis recognition sub-model.

[0090] Specifically, the target slit lamp image quality enhancer model is also called the front-end model, and the front-end model includes a Swin-Conv-UNet (SCUNet) model. Among them, the SCUNet model integrates the Swin-Conv (SC) module in the Swin Transformer (SwinT) model into the U-Net framework to form a SCUNet model for enhancing (denoising) low-quality slit lamp images. That is, the front-end model can enhance low-quality slit lamp images, repair noise such as motion blur, defocus blur, and exposure distortion in low-quality slit lamp images, improve the quality of slit lamp images, and then generate high-quality slit lamp images.

[0091] The target keratitis recognition sub-model is also called the back-end model. The back-end model integrates the Gaussian Context Transformer (GCT) module into the densely connected network (DenseNet12) to form a GCT-DenseNet12 model, which is used to identify the type of eye disease in the high-quality slit lamp images output by the front-end model. That is, the back-end model can extract global features from the enhanced high-quality slit lamp images, so as to accurately identify the keratitis status (such as normal cornea, keratitis, other keratitis, etc.) in the high-quality slit lamp images.

[0092] In an embodiment of the present invention, the first slit lamp image sample is a high-quality slit lamp image of the corneal area of ​​the patient's eye, and its function is to generate a second slit lamp image sample (i.e., a low-quality slit lamp image sample) corresponding to the first slit lamp image sample.

[0093] In practical applications, it is necessary to use high-quality slit lamp image samples and corresponding low-quality slit lamp image samples to form training data pairs, and use the training data pairs to train the initial slit lamp image quality enhancer model to obtain the target slit lamp image quality enhancer model. However, in actual clinical practice, high- and low-quality slit lamp image sample pairs are very scarce, and sufficient high- and low-quality slit lamp image sample pairs are not retained and annotated in clinical practice, resulting in the inability to fully train the model. Therefore, in the process of training the initial slit lamp image quality enhancer model, in order to address the problem of lack of high- and low-quality slit lamp image sample pairs, it is necessary to process the first slit lamp image sample corresponding to the high-quality image quality category based on the image quality degradation algorithm, randomly generate second slit lamp image samples of various low-quality image quality categories (i.e., simulate the generation of low-quality slit lamp images with motion blur, defocus blur, and exposure distortion that appear in real clinical scenes), and pair the generated second slit lamp image samples with the first slit lamp image samples to form training data pairs.

[0094] In the embodiment of the present invention, the image quality degradation algorithm uses a random generation strategy to ensure the diversity of low-quality slit lamp images and the robustness of the model. By simulating a variety of image degradation conditions (motion blur, defocus blur, exposure distortion), the image quality degradation algorithm creates a wide range of training data sets covering real clinical scenarios, which helps the initial slit lamp image quality enhancer model to learn and adapt to various complex noise and degradation conditions during training, and improves the generalization ability and reliability of the model in practical applications. Ultimately, the target slit lamp image quality enhancer model can cope with different low-quality slit lamp images in a clinical environment, ensuring the stability and accuracy of diagnosis.

[0095] Step 102: Based on each of the training data pairs, the initial slit lamp image quality enhancer model is trained until convergence to obtain the target slit lamp image quality enhancer model; wherein the initial slit lamp image quality enhancer model includes a SCUNet generation model formed by fusing a Swin-Conv module and a U-Net framework.

[0096] In an embodiment of the present invention, the initial slit lamp image quality enhancer model is trained based on multiple training data pairs, so that the model can learn the mapping relationship from the low-quality second slit lamp image samples to the high-quality first slit lamp image samples, thereby enhancing the clarity and contrast of the low-quality slit lamp images.

[0097] Specifically, the initial slit lamp image quality enhancer model integrates the SC module into the U-Net framework. Among them, the SC module combines the traditional convolution and the global attention mechanism of the Swin Transformer, so that the initial slit lamp image quality enhancer model can capture the global information and long-range dependencies in the image, improve the denoising effect, and effectively remove motion blur, defocus blur and exposure distortion noise. Through the above method, the limitations of related technologies in extracting global information when processing low-quality slit lamp images are solved. The introduced SC module can enhance the model's ability to handle complex noise and improve the image restoration effect, especially in clinical low-quality slit lamp images with complex noise.

[0098] Step 103, collect multiple third slit lamp image samples that have been marked as low-quality image quality categories; input each of the third slit lamp image samples into the target slit lamp image quality enhancement sub-model, and obtain fourth slit lamp image samples corresponding to each of the third slit lamp image samples output by the target slit lamp image quality enhancement sub-model, wherein the image quality category of the fourth slit lamp image sample is higher than that of the third slit lamp image sample.

[0099] In the embodiment of the present invention, after the target slit lamp image quality enhancement sub-model is obtained through training, the initial keratitis recognition sub-model needs to be trained based on the high-quality fourth slit lamp image sample.

[0100] Since the high-quality fourth slit-lamp image sample is obtained after the target slit-lamp image quality enhancement sub-model performs image quality enhancement on the low-quality third slit-lamp image sample, the initial keratitis recognition sub-model is trained based on the fourth slit-lamp image sample so that the initial keratitis recognition sub-model and the target slit-lamp image quality enhancement sub-model can fully cooperate and learn with each other.

[0101] Step 104: Based on each of the fourth slit lamp image samples, train the initial keratitis recognition sub-model until convergence to obtain the target keratitis recognition sub-model; wherein the initial keratitis recognition sub-model is a dense convolutional neural network integrated with a Gaussian context transformer GCT module.

[0102] In practical applications, the target keratitis recognition sub-model (back-end model) is used to recognize the corneal state of the high-quality slit lamp image after the target slit lamp image quality enhancement sub-model has performed image quality enhancement.

[0103] The present invention provides a training method for an automatic keratitis recognition model for low-quality slit lamp images. By collecting multiple first slit lamp image samples that have been labeled as high-quality image quality categories, and using an image quality degradation algorithm to process each first slit lamp image sample, a variety of slit lamp image degradation conditions corresponding to the first slit lamp image sample can be simulated, thereby creating a second slit lamp image sample of a wide range of low-quality image quality categories covering real clinical scenarios, ensuring the diversity of low-quality slit lamp image samples; the first slit lamp image sample and the corresponding second slit lamp image sample are paired to form a training data pair, and based on the multiple training data pairs, an initial slit lamp image quality enhancer model is trained until convergence, which can improve the robustness of model training and help the initial slit lamp image quality enhancer model learn in training that the first slit lamp image sample in each training data pair degrades to the second slit lamp image Various complex noise and degradation conditions of the samples are taken into consideration, so that the model can enhance and restore the slit lamp images of low image quality to slit lamp images of high image quality in a real clinical environment; after obtaining the trained target slit lamp image quality enhancement sub-model, multiple third slit lamp image samples marked as low quality image quality categories are input into the target slit lamp image quality enhancement sub-model to obtain the fourth slit lamp image samples after the image quality of each third slit lamp image sample is enhanced; based on each fourth slit lamp image sample, the initial keratitis recognition sub-model is trained until convergence, so that the initial keratitis recognition sub-model can fully learn the feature information in the fourth slit lamp image sample enhanced and restored by the target slit lamp image quality enhancement sub-model, so that in practical applications, the corneal state corresponding to the slit lamp image enhanced and restored by the target slit lamp image quality enhancement sub-model can be accurately and effectively identified.

[0104] Furthermore, since the slit-lamp image samples of the low-quality image quality category do not directly participate in the training of the initial keratitis recognition sub-model, the trained target keratitis recognition sub-model can not only realize the accurate recognition of slit-lamp images of different low-quality image quality categories in a real clinical environment, but also ensure the recognition effect of slit-lamp images of the high-quality image quality category.

[0105] Optionally, for each first slit lamp image sample, a second slit lamp image sample is generated using an image quality degradation algorithm, which can be specifically achieved by the following steps:

[0106] Based on the image quality degradation algorithm, at least one of motion blur processing, defocus blur processing and exposure distortion processing is performed on each first slit lamp image sample to obtain a second slit lamp image sample.

[0107] In an embodiment of the present invention, training the initial slit lamp image quality enhancer model requires training data pairs including high-quality and low-quality slit lamp image samples, but the slit lamp image samples collected clinically lack low-quality slit lamp image samples corresponding to the high-quality slit lamp image samples.

[0108] Based on the above-mentioned problems, in order to simulate the situation in which low-quality slit lamp images containing noises such as motion blur, defocus blur and exposure distortion may be collected in real clinical scenarios, the embodiment of the present invention processes the first slit lamp image sample by using an image quality degradation algorithm, and randomly generates various low-quality second slit lamp image samples containing the above-mentioned various noises, thereby ensuring the diversity of training data pairs.

[0109] Specifically, the first slit lamp image sample is represented as I(x, y), and the second slit lamp image samples with motion blur, defocus blur and exposure distortion generated by the image quality degradation algorithm are I motion (x, y), I defocus (x, y) and I exposure (x, y).

[0110] Motion blur processing is achieved through the following formula (1):

[0111]

[0112] Defocus blur processing is achieved by the following formula (2):

[0113]

[0114] Exposure distortion processing is achieved through the following formula (3):

[0115] I exposur e(x, y)=α·I(x, y)+β (3)

[0116] Where, I(x, y) represents the first slit lamp image sample; I motion (x, y), I defocus (x, y) and I exposure(x, y) represent the second slit lamp image sample generated by the image quality degradation algorithm for the first slit lamp image sample; (x, y) represent the coordinates of the first slit lamp image sample; K(x, y) represents the motion blur kernel; (i, j) represents the discrete coordinates iterated to the motion blur kernel; (k, l) represents the standard deviation of the defocus blur; α and β represent the adjustment coefficients of contrast and brightness.

[0117] Figure 2 Schematic diagram of the structure of the SCUNet generation model provided by the present invention. Figure 2 As shown in the figure, the SCUNet generation model is formed by the fusion of the Swin-Conv module and the U-Net framework; wherein the Swin-Conv module includes the first convolutional layer, the second convolutional layer, the Swin-Transformer module and the residual convolutional module, and the first convolutional layer is residually connected with the second convolutional layer;

[0118] The U-Net framework includes a stride convolution module (SConv), a transpose convolution module (TConv), multiple Swin-Conv modules and convolution layers. The stride convolution module is residually connected to the transpose convolution module.

[0119] Specifically, the backbone module of the SCUNet model contains 4 scales, each of which has a residual connection between downsampling based on 2×2 strided convolution and upsampling based on 2×2 transposed convolution. From the first scale to the fourth scale, the number of channels per layer is 64, 128, 256, and 512, respectively. The SCUNet model uses 4 SC modules in each scale of downsampling and upsampling.

[0120] The SC module combines the traditional convolution operation with the non-local attention mechanism of the Swin Transformer to better capture the global information and long-range dependencies in the slit lamp image. The forward propagation method of each SC module is as follows: the input feature X is divided into two parts, X1 and X2, by a 1×1 convolution. The calculation process is as follows:

[0121] X1, X2=Split(Conv1×1(X)) (5)

[0122] Then, the SwinT module and the residual convolution (RConv) module are used to process the features X1 and X2 respectively to obtain the corresponding outputs Y1 and Y2, as shown in formula (6):

[0123] Y1, Y2=SwinT(X1), RConv(X2) (6)

[0124] Finally, Y1 and Y2 are concatenated and residually connected to the input to form the final output Z of the SC module, as shown in formula (7):

[0125] Z=Conv1×1(Concat(Y1, Y2))+x (7)

[0126] Optionally, the initial slit lamp image quality enhancer model includes a discriminator in addition to the SCUNet generation model. Based on each training data pair, the initial slit lamp image quality enhancer model is trained until convergence to obtain a target slit lamp image quality enhancer model, which can be specifically achieved by the following steps:

[0127] Step A: Based on each training data pair, the SCUNet generation model is trained so that the SCUNet generation model learns the mapping relationship between the second slit lamp image sample and the first slit lamp image sample from each training data pair, and generates a fifth slit lamp image sample with enhanced image quality corresponding to each second slit lamp image sample based on the mapping relationship.

[0128] In an embodiment of the present invention, the generator integrates the SC module into the U-Net framework to form a SCUNet generation model, and learns the mapping relationship between the low-quality second slit lamp image samples and the high-quality first slit lamp image samples from the training data pairs of high-quality and low-quality slit lamp images. Based on the above mapping relationship, the SCUNet generation model can remove noise such as motion blur, defocus blur, and exposure distortion in each second slit lamp image sample, enhance the clarity and contrast of each second slit lamp image sample, and then obtain the fifth slit lamp image sample with enhanced image quality.

[0129] Step B: Input the real first slit lamp image sample and the fifth slit lamp image sample with enhanced image quality into the discriminator, so that the discriminator distinguishes the difference between the fifth slit lamp image sample and the corresponding first slit lamp image sample, and adjusts the model parameters of the SCUNet generation model based on the difference.

[0130] In the front-end model, a discriminator corresponding to the SCUNet generative model is introduced, and the SCUNet generative model is trained using a generative adversarial learning strategy to improve the quality and authenticity of the fifth high-quality slit lamp image sample generated by the SCUNet generative model. The introduction of the discriminator enhances the stability of training and the visual quality of the generated slit lamp image. The discriminator compares the fifth slit lamp image sample with the first slit lamp image sample, and adjusts the parameters of the SCUNet generative model according to the adversarial loss, so that the output of the SCUNet generative model gradually approaches the real high-quality slit lamp image.

[0131] Figure 3 Schematic diagram of the structure of the discriminator provided by the present invention. Figure 3 As shown in the figure, the discriminator is composed of convolution layer, convolution layer, normalization layer, convolution layer, normalization layer and activation layer from input layer to output layer. The discriminator is used to distinguish the difference between the fifth slit lamp image sample generated by the SCUNet generation model and the first slit lamp image sample (i.e., the real slit lamp image sample). Based on the training data pairs of high-quality and low-quality slit lamp images, the SCUNet generation model and the discriminator are trained multiple times in adversarial training, and the adversarial loss is calculated to train the parameters of the SCUNet generation model, so that the SCUNet generation model is converged and stabilized.

[0132] The adversarial loss function of the SCUNet generative model is calculated as shown in formula (11), where formulas (8)-(10) are the calculation process of each loss component of the adversarial loss function.

[0133]

[0134] ssim _ loss = 1 - SSIM (Output, HQ) (10)

[0135] total_loss_G=adversarial_loss+α·loss_pixel+γ·ssim_loss (11)

[0136] Where HQ, LQ and Output represent the first slit lamp image sample, the second slit lamp image sample degraded to low quality and the fifth slit lamp image sample generated by the SCUNet generative model, respectively; N represents the number of training samples; D(·) represents the output of the discriminator; G(·) represents the output of the generator; SSIM(·) represents the structural similarity between the two images, α is the weight of the pixel-level loss, and γ is the weight of the SSIM loss.

[0137] After the SCUNet generative model is trained, the real low-quality slit-lamp images collected clinically are input into the SCUNet generative model, and the enhanced slit-lamp images can be obtained from its output.

[0138] Step C: Loop through steps AB until the SCUNet generated model converges and obtains the target slit lamp image quality enhancement sub-model.

[0139] In an embodiment of the present invention, after obtaining the target slit lamp image quality enhancement sub-model, it is necessary to input multiple low-quality third slit lamp image samples into the target slit lamp image quality enhancement sub-model to obtain fourth slit lamp image samples after the image quality of each third slit lamp image sample is enhanced; the initial keratitis recognition sub-model is trained using multiple fourth slit lamp image samples until convergence to obtain the target keratitis recognition sub-model (GCT-DenseNet121 model, also known as the back-end model).

[0140] Figure 4 Schematic diagram of the structure of the GCT-DenseNet121 model provided by the present invention. Figure 4 As shown, the GCT-DenseNet121 model includes at least one GCT module and a Densenet121 model, and the Densenet121 model includes a convolutional layer, a pooling layer, a normalization-activation-convolutional layer, a dense block, a pooling layer, a fully connected layer, and a softmax activation layer;

[0141] The GCT module, the normalization-activation-convolutional layer and the GCT module are sequentially connected between two adjacent dense blocks; the side of the first dense block that is not adjacent to other dense blocks is sequentially connected with the convolutional layer, the pooling layer and the GCT module starting from the side far away from the first dense block; the side of the second dense block that is not adjacent to other dense blocks is sequentially connected with the pooling layer, the fully connected layer and the softmax activation layer starting from the side close to the second dense block; the first dense block is the first dense block starting from the input end in the Densenet121 model, and the second dense block is the last dense block starting from the input end.

[0142] The present invention integrates the GCT module into the Densenet121 convolutional neural network and constructs a GCT-DenseNet121 back-end model for classifying high-quality slit lamp images after image quality enhancement and identifying corneal states (such as normal cornea, keratitis and other corneal abnormalities).

[0143] In an embodiment of the present invention, the target keratitis identification sub-model (back-end model) embeds the GCT module into the deep convolutional neural network DenseNet121, thereby improving the classification performance. Among them, the GCT module includes global context aggregation (GlobalContextAggregation, GCA), normalization (Normalization, Norm) and GaussianContext Excitation (GaussianContext Excitation, GCE) operations. It enhances the feature extraction capability of the back-end model through global average pooling and Gaussian excitation, so that the back-end model can simultaneously extract corneal states such as keratitis lesions and conjunctival hemorrhage and other complication features. The above method solves the problem of insufficient global feature extraction of images by traditional convolutional neural networks. The introduction of the GCT module improves the model's ability to identify complex lesions and complications of keratitis, and improves the accuracy of automatic diagnosis of keratitis.

[0144] Figure 5 This is a logical schematic diagram of extracting the global features of slit lamp images based on the GCT module provided by the present invention. Figure 5 As shown, each GCT module is used to extract the global features of the fourth slit lamp image sample. For each GCT module, the global features are obtained in the following way:

[0145] Step 1) Perform a global context aggregation (GCA) operation on the input features of the GCT module to obtain the first output features.

[0146] In an embodiment of the present invention, the GCA operation uses global average pooling (AVG) to aggregate global context information in the feature map space to obtain channel statistical information (ie, the first output feature Z), as specifically shown in formula (12).

[0147]

[0148] X is the input feature map of size C×H×W, where C is the number of channels and H and W are the spatial dimensions.

[0149] Step 2) normalize the first output feature to obtain the second output feature.

[0150] The Norm normalization operation is performed by introducing the average offset (z1μ) and the specific instance factor (σ) for standardization. The calculation process is shown in formulas (13)-(15).

[0151]

[0152] Here, ε is a small constant used to ensure the normalization operation Numerical stability.

[0153] Step 3) Perform Gaussian context excitation (GCE) operation on the second output feature to obtain a global feature.

[0154] The GCE operation uses a Gaussian function as context excitation, and the calculation process is shown in formula (16).

[0155]

[0156] Among them, the learnable parameter c represents the standard deviation of the Gaussian function, which controls the variation of channel attention activation. The larger the standard deviation, the smaller the difference in attention activation between channels.

[0157] Finally, the activated attention g is multiplied with the input feature x to obtain the output feature Y, as shown in formula (17).

[0158]

[0159] In a possible implementation, before the initial keratitis recognition sub-model is trained based on each fourth slit lamp image sample, the following operations need to be performed for the initial keratitis recognition sub-model:

[0160] Each first slit lamp image sample is input into the initial keratitis recognition sub-model for iterative training until the loss value of the target loss function reaches a preset threshold, thereby obtaining the initial weight parameters of the target keratitis recognition sub-model.

[0161] Accordingly, based on each fourth slit lamp image sample, the initial keratitis recognition sub-model is trained until convergence to obtain a target keratitis recognition sub-model, which can be specifically achieved by the following steps:

[0162] Step 1), loading the initial weight parameters obtained by training based on each first slit lamp image sample into the initial keratitis recognition sub-model;

[0163] Step 2), based on each fourth slit lamp image sample, the initial keratitis recognition sub-model is trained for a second iteration until the loss value of the target loss function reaches a preset threshold, thereby obtaining a target keratitis recognition sub-model.

[0164] In the above embodiment, each high-quality first slit lamp image sample is input into the initial keratitis recognition sub-model for iterative training until the loss value of the target loss function reaches a preset threshold value, and the initial weight parameter of the target keratitis recognition sub-model is obtained. This process uses a large number of high-quality slit lamp image samples to enable the keratitis recognition sub-model to obtain a better initial weight;

[0165] When the initial keratitis recognition sub-model can recognize the first slit lamp image sample with high quality, the third slit lamp image is input into the target slit lamp image quality enhancement sub-model to obtain an enhanced fourth slit lamp image sample output by the target slit lamp image quality enhancement sub-model;

[0166] The enhanced fourth slit lamp image sample is input into the initial keratitis recognition sub-model, and the initial weight parameters obtained by training the first slit lamp image sample are loaded for secondary iterative training until the loss value of the target loss function reaches the preset threshold, thereby obtaining the target keratitis recognition sub-model.

[0167] In the above implementation, the high-quality first slit lamp image sample is used to train the initial keratitis recognition sub-model, the low-quality third slit lamp image is enhanced by the front-end model to obtain the fourth slit lamp image with enhanced image quality, and then the fourth slit lamp image is used for transfer learning to load the initial keratitis recognition sub-model with initial weight parameters. In the above manner, better initial weight parameters are loaded in advance during the training of the initial keratitis recognition sub-model, and the real high-quality first slit lamp image sample, the real low-quality third slit lamp image sample and the fourth slit lamp image sample with enhanced image quality are used at the same time, thereby improving the accuracy of the target keratitis recognition sub-model.

[0168] Optionally, in one possible implementation, an optimizing cost-sensitive weights (OCSw) training strategy is introduced in the back-end model, that is, the back-end model integrates the deep convolutional neural network of the GCT module and OCSW to balance the error weights of different categories, thereby overcoming the data imbalance problem.

[0169] In the process of automatic recognition of keratitis for low-quality slit lamp images, the number of samples of the three categories of normal cornea, keratitis and other corneal abnormalities is unbalanced, and the number of images of keratitis and other corneal abnormalities is less than that of normal cornea. The model trained based on unbalanced data tends to be biased towards the category with a larger sample size during prediction, resulting in insufficient learning ability of the model for the category with a smaller sample size. To address this problem, the present invention adopts the OCSW strategy to assign a larger error weight to the category with a smaller sample size. Based on the OCSW strategy, the model can learn the features of different categories in a balanced manner, thereby solving the negative interference caused by the imbalance of sample data.

[0170] That is to say, when training the initial keratitis recognition sub-model, the OCSW training strategy is adopted, that is, based on each fourth slit lamp image sample, the initial keratitis recognition sub-model is trained until convergence to obtain the target keratitis recognition sub-model, which is specifically achieved by the following steps:

[0171] Inputting each fourth slit lamp image sample into the initial keratitis recognition sub-model for iterative training until the loss value of the target loss function reaches a preset threshold, thereby obtaining a target keratitis recognition sub-model;

[0172] The objective loss function is expressed by the following formula (4):

[0173]

[0174] Among them, loss(P, Q, W) represents the loss value, P i Indicates that the output of the initial keratitis recognition sub-model is the probability of the i-th corneal state category, Q i represents the probability that the true label of the fourth slit lamp image sample is the i-th corneal state category, W i Represents the error weight set for the i-th corneal condition category.

[0175] In the above implementation, the OCSW training strategy assigns a larger error weight to the category with a smaller sample size, so that the model can learn the characteristics of each category more evenly during training, avoiding bias towards the category with a larger sample size. In the case of unbalanced distribution of samples in different categories, the traditional model tends to favor the category with more samples. The OCSW strategy significantly improves the recognition ability of keratitis and other corneal abnormalities with a small sample size.

[0176] The present invention utilizes the SCUNet front-end model to remove noises such as motion blur, defocus blur, and exposure distortion in low-quality slit-lamp images, enhance the quality of low-quality slit-lamp images, restore the clarity and contrast of slit-lamp images, and help highlight features related to keratitis. The GCT-DenseNet121 back-end model integrates the GCT module and the OCSW training strategy. The GCT module can extract global features of keratitis lesions and conjunctival hemorrhagic complications, improve the accuracy of keratitis diagnosis, and the OCSW training strategy effectively handles the problem of unbalanced data between different categories, which is beneficial for the model to identify keratitis and other corneal abnormality categories with a small number of samples. Therefore, the present invention can be applied to the automatic diagnosis of keratitis in real clinical scenarios.

[0177] Figure 6 FIG2 is a flow chart of the training method of the automatic keratitis recognition model for low-quality slit lamp images provided by the present invention. Figure 6 As shown, the training method of the keratitis automatic recognition model for low-quality slit lamp images provided by the present invention comprises the following steps:

[0178] 1. Obtain the slit lamp image dataset.

[0179] 2. Ophthalmologists with rich clinical experience independently annotate each image in the slit lamp image dataset as a high-quality slit lamp image or a low-quality slit lamp image.

[0180] 3. Randomly divide the high-quality slit lamp images into 70% training set, 15% validation set and 15% test set.

[0181] 4. Based on the above training set, validation set and test set, randomly generate low-quality slit lamp images, and construct high- and low-quality slit lamp image pairs required to generate the front-end model.

[0182] Specifically, the high-quality slit-lamp images in the training set are processed using an image quality degradation algorithm to generate low-quality slit-lamp images, and high- and low-quality slit-lamp image pairs corresponding to the training set are constructed.

[0183] The high-quality slit-lamp images in the validation set were processed using an image quality degradation algorithm to generate low-quality slit-lamp images, and high- and low-quality slit-lamp image pairs corresponding to the validation set were constructed.

[0184] The high-quality slit-lamp images in the test set are processed using an image quality degradation algorithm to generate low-quality slit-lamp images, and high- and low-quality slit-lamp image pairs corresponding to the test set are constructed.

[0185] 5. Use the high- and low-quality slit lamp image pairs corresponding to the training set to train the front-end model; use the high- and low-quality slit lamp image pairs corresponding to the validation set to verify the front-end model; use the high- and low-quality slit lamp image pairs corresponding to the test set to test the front-end model to obtain the trained front-end model.

[0186] 6. Input the low-quality slit lamp images annotated in step 2 into the trained front-end model to obtain high-quality slit lamp images with enhanced image quality; randomly divide the high-quality slit lamp images with enhanced image quality to obtain 70% training set, 15% validation set and 15% test set.

[0187] 7. Based on the above training set, validation set and test set, train the backend model, verify the backend model and test the backend model to obtain a trained backend model.

[0188] Specifically, before using the training set to train the back-end model, it is also necessary to use the high-quality slit lamp images annotated in step 2 to pre-train the back-end model, so as to achieve data enhancement for the back-end model.

[0189] Then, the backend model is trained using the training set, verified using the validation set, and tested using the test set to obtain a trained backend model.

[0190] 8. In practical applications, low-quality slit lamp images are input into the trained front-end model for image quality enhancement to obtain high-quality slit lamp images; high-quality slit lamp images are input into the trained back-end model so that the back-end model can identify the type of keratitis in the slit lamp images.

[0191] 9. Test the generalization ability of the model.

[0192] Specifically, in practical applications, multiple low-quality slit lamp images can be used to perform step 8 in sequence, and the accuracy of the keratitis type recognition results can be judged to test the generalization ability of the model.

[0193] Figure 7 Schematic diagram of the process of the automatic keratitis recognition method for low-quality slit lamp images provided by the present invention. Figure 7 As shown, the specific steps include:

[0194] Step 701: Acquire a slit lamp image of the patient's eye to be identified.

[0195] Step 702: When the slit lamp image to be identified is of a low-quality image quality category, the slit lamp image to be identified is input into a target slit lamp image quality enhancement sub-model to obtain an enhanced slit lamp image output by the target slit lamp image quality enhancement sub-model; the target slit lamp image quality enhancement sub-model is used to repair noise existing in the low-quality slit lamp image;

[0196] Step 703: input the enhanced slit-lamp image into a target keratitis recognition sub-model to obtain a corneal state corresponding to the slit-lamp image to be recognized output by the target keratitis recognition sub-model; the target keratitis recognition sub-model is used to recognize the corneal state corresponding to the slit-lamp image.

[0197] Among them, the target slit lamp image quality enhancement sub-model and the target keratitis identification sub-model are obtained by Figures 1 to 6 The method is used to train the automatic keratitis recognition model for low-quality slit lamp images.

[0198] The present invention provides an automatic keratitis recognition method for low-quality slit-lamp images. By collecting a slit-lamp image to be recognized of a patient's eye, when the slit-lamp image to be recognized is of a low-quality image quality category, the slit-lamp image to be recognized is input into a target slit-lamp image quality enhancement sub-model, so that the target slit-lamp image quality enhancement model can enhance and restore the low-quality slit-lamp image to be recognized into an enhanced slit-lamp image in a real clinical environment; the enhanced slit-lamp image is input into a target keratitis recognition model, so that the target keratitis recognition model can fully recognize the feature information in the enhanced slit-lamp image, thereby realizing accurate recognition of the corneal states of different low-quality slit-lamp images in a real clinical environment, and also ensuring the recognition effect of high-quality slit-lamp images.

[0199] Optionally, after acquiring the slit lamp image to be identified of the patient's eye, it is also necessary to determine the image quality of the slit lamp image to be identified. Different processing procedures need to be performed for slit lamp images of different image qualities; specifically, the following steps are included:

[0200] Inputting the slit lamp image to be identified into the image quality classification model, and obtaining the image quality category corresponding to the slit lamp image to be identified output by the image quality classification model;

[0201] When the image quality category is high quality, the slit lamp image to be identified is input into the target keratitis identification sub-model to obtain the corneal state corresponding to the slit lamp image to be identified.

[0202] In the embodiment of the present invention, firstly, high-quality and low-quality slit lamp images are collected as training sample sets, and an image quality classification model is trained based on the training sample sets to automatically judge the image quality of the slit lamp images to be identified. In practical applications, ophthalmologists can also make manual judgments when collecting the slit lamp images to be identified.

[0203] If the image quality of the slit lamp image to be identified is a high-quality slit lamp image, there is no need to input the image quality of the slit lamp image to be identified into the target slit lamp image quality enhancement sub-model, but it is directly input into the target keratitis recognition model to identify the corneal state.

[0204] Through the above method, different recognition methods are performed for slit lamp images of different image qualities, which effectively improves the efficiency of corneal state recognition. At the same time, the recognition effect of high-quality slit lamp images and low-quality slit lamp images is also guaranteed.

[0205] The following describes the training device for the automatic keratitis recognition model for low-quality slit-lamp images provided by the present invention. The training device for the automatic keratitis recognition model for low-quality slit-lamp images described below and the training method for the automatic keratitis recognition model for low-quality slit-lamp images described above can be referenced to each other. Figure 8 is a schematic diagram of the structure of a training device for an automatic keratitis recognition model for low-quality slit lamp images provided by the present invention, such as Figure 8 As shown, the training device 800 for the automatic keratitis recognition model for low-quality slit lamp images includes: a first acquisition module 801, a first training module 802, a second acquisition module 803, and a second training module 804, wherein:

[0206] The first acquisition module 801 is used to acquire a plurality of first slit lamp image samples that have been marked as high-quality image quality categories; for each first slit lamp image sample, generate a second slit lamp image sample using an image quality degradation algorithm, and pair the first slit lamp image sample with the second slit lamp image sample to form a training data pair, wherein the image quality category of the second slit lamp image sample is lower than that of the first slit lamp image sample;

[0207] A first training module 802 is used to train the initial slit lamp image quality enhancer model based on each training data pair until convergence, so as to obtain a target slit lamp image quality enhancer model; wherein the initial slit lamp image quality enhancer model includes a SCUNet generation model formed by fusing a Swin-Conv module and a U-Net framework;

[0208] The second acquisition module 803 is used to acquire a plurality of third slit lamp image samples that have been marked as low-quality image quality categories; input each of the third slit lamp image samples into the target slit lamp image quality enhancement sub-model, and obtain a fourth slit lamp image sample corresponding to each of the third slit lamp image samples output by the target slit lamp image quality enhancement sub-model, wherein the image quality category of the fourth slit lamp image sample is higher than that of the third slit lamp image sample;

[0209] The second training module 804 is used to train the initial keratitis recognition sub-model based on each fourth slit lamp image sample until convergence to obtain a target keratitis recognition sub-model; wherein the initial keratitis recognition sub-model is a dense convolutional neural network fused with a Gaussian context transformer GCT module.

[0210] Optionally, the first acquisition module 801 is further configured to:

[0211] Based on the image quality degradation algorithm, performing at least one of motion blur processing, defocus blur processing and exposure distortion processing on each first slit lamp image sample to obtain a second slit lamp image sample;

[0212] Motion blur processing is achieved through the following formula (1):

[0213]

[0214] Defocus blur processing is achieved by the following formula (2):

[0215]

[0216] Exposure distortion processing is achieved through the following formula (3):

[0217] I exposure (x, y)=α·I(x, y)+β (3)

[0218] Where, I(x, y) represents the first slit lamp image sample; I motion (x, y), I defocus (x, y) and I exposure (x, y) represent the second slit lamp image sample generated by the image quality degradation algorithm for the first slit lamp image sample; (x, y) represent the coordinates of the first slit lamp image sample; K(x, y) represents the motion blur kernel; (i, j) represents the discrete coordinates iterated to the motion blur kernel; (k, l) represents the standard deviation of the defocus blur; α and β represent the adjustment coefficients of contrast and brightness.

[0219] Optionally, the initial slit lamp image quality enhancer model also includes a discriminator;

[0220] The first training module 802 is further used to:

[0221] Step A: Based on each training data pair, the SCUNet generation model is trained so that the SCUNet generation model learns the mapping relationship between the second slit lamp image sample and the first slit lamp image sample from each training data pair, and generates a fifth slit lamp image sample with enhanced image quality corresponding to each second slit lamp image sample based on the mapping relationship;

[0222] Step B: inputting the first slit lamp image sample and the fifth slit lamp image sample into the discriminator, so that the discriminator distinguishes the difference between the fifth slit lamp image sample and the corresponding first slit lamp image sample, and adjusts the model parameters of the SCUNet generation model based on the difference;

[0223] Step C: Loop through steps AB until the SCUNet generated model converges and obtains the target slit lamp image quality enhancement sub-model.

[0224] Optionally, the SCUNet generative model is formed by fusing the Swin-Conv module and the U-Net framework;

[0225] The Swin-Conv module includes a first convolutional layer, a second convolutional layer, a Swin-Transformer module, and a residual convolutional module, and the first convolutional layer is residually connected to the second convolutional layer;

[0226] The U-Net framework includes a strided convolution module, a transposed convolution module, multiple Swin-Conv modules and convolution layers. The strided convolution module is residually connected to the transposed convolution module.

[0227] Optionally, the second training module 804 is further configured to:

[0228] Inputting each fourth slit lamp image sample into the initial keratitis recognition sub-model for iterative training until the loss value of the target loss function reaches a preset threshold, thereby obtaining a target keratitis recognition sub-model;

[0229] The objective loss function is expressed by the following formula (4):

[0230]

[0231] Among them, loss(P, Q, W) represents the loss value, P i Indicates that the output of the initial keratitis recognition sub-model is the probability of the i-th corneal state category, Q i represents the probability that the true label of the fourth slit lamp image sample is the i-th corneal state category, W i Represents the error weight set for the i-th corneal condition category.

[0232] Optionally, the device further comprises:

[0233] A third training module is used to input each first slit lamp image sample into the initial keratitis recognition sub-model for iterative training until the loss value of the target loss function reaches a preset threshold value, thereby obtaining the initial weight parameters of the target keratitis recognition sub-model;

[0234] The second training module 804 is further used to: load the initial weight parameters obtained through training based on each first slit lamp image sample into the initial keratitis recognition sub-model;

[0235] Based on each fourth slit lamp image sample, the initial keratitis recognition sub-model is trained for a second iteration until the loss value of the target loss function reaches a preset threshold, thereby obtaining the target keratitis recognition sub-model.

[0236] Fig. 9 : is a schematic diagram of the structure of the automatic keratitis identification device for low-quality slit lamp images provided by the present invention, such as Fig. 9As shown, the automatic keratitis identification device 900 for low-quality slit lamp images includes: a third acquisition module 901, a slit lamp image enhancement module 902, and a corneal state identification module 903, wherein:

[0237] The third acquisition module 901 is used to acquire the slit lamp image to be identified of the patient's eye;

[0238] The slit lamp image enhancement module 902 is used for inputting the slit lamp image to be identified into the target slit lamp image quality enhancement sub-model when the slit lamp image to be identified is of a low-quality image quality category, and obtaining an enhanced slit lamp image output by the target slit lamp image quality enhancement sub-model; the target slit lamp image quality enhancement sub-model is used to repair the noise existing in the low-quality slit lamp image;

[0239] The first corneal state recognition module 903 is used to input the enhanced slit lamp image into the target keratitis recognition sub-model to obtain the corneal state corresponding to the slit lamp image to be recognized output by the target keratitis recognition sub-model; the target keratitis recognition sub-model is used to recognize the corneal state corresponding to the slit lamp image;

[0240] Among them, the target slit lamp image quality enhancement sub-model and the target keratitis recognition sub-model are as described above. Figures 1 to 6 The model is trained by any of the training methods for automatic keratitis recognition models for low-quality slit lamp images.

[0241] Optionally, the device further comprises:

[0242] A slit lamp image quality category judgment module is used to input the slit lamp image to be identified into the image quality classification model to obtain the image quality category corresponding to the slit lamp image to be identified output by the image quality classification model;

[0243] The second corneal state recognition module is used to input the slit lamp image to be identified into the target keratitis recognition sub-model when the image quality category is high quality, so as to obtain the corneal state corresponding to the slit lamp image to be identified.

[0244] Fig.10 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.10 As shown, the electronic device may include: a processor 1010, a communication interface 1020, a memory 1030 and a communication bus 1040, wherein the processor 1010, the communication interface 1020 and the memory 1030 communicate with each other via the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute the above Figure 1The training method of the automatic keratitis recognition model for low-quality slit lamp images shown in Figure 7 The presented method for automatic keratitis recognition in low-quality slit-lamp images.

[0245] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0246] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can perform the above Figure 1 The training method of the automatic keratitis recognition model for low-quality slit lamp images shown in Figure 7 The presented method for automatic keratitis recognition in low-quality slit-lamp images.

[0247] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is executed by a processor to perform the above Figure 1 The training method of the automatic keratitis recognition model for low-quality slit lamp images shown in Figure 7 The presented method for automatic keratitis recognition in low-quality slit-lamp images.

[0248] The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0249] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0250] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for an automatic keratitis recognition model for low-quality slit lamp images, characterized in that: The keratitis automatic recognition model includes: a target slit lamp image quality enhancement sub-model for repairing noise in low-quality slit lamp images, and a target keratitis recognition sub-model for recognizing the corneal state corresponding to the slit lamp image; The training method comprises: Collecting a plurality of first slit lamp image samples that have been labeled as high-quality image quality categories; for each of the first slit lamp image samples, using an image quality degradation algorithm to generate a second slit lamp image sample, and pairing the first slit lamp image sample with the second slit lamp image sample to form a training data pair, wherein the image quality category of the second slit lamp image sample is lower than that of the first slit lamp image sample; Based on each of the training data pairs, the initial slit lamp image quality enhancer model is trained until convergence to obtain the target slit lamp image quality enhancer model; wherein the initial slit lamp image quality enhancer model includes a SCUNet generation model formed by fusing a Swin-Conv module and a U-Net framework; Collect a plurality of third slit lamp image samples that have been marked as low-quality image quality categories; input each of the third slit lamp image samples into the target slit lamp image quality enhancement sub-model, respectively, to obtain a fourth slit lamp image sample corresponding to each of the third slit lamp image samples output by the target slit lamp image quality enhancement sub-model, wherein the image quality category of the fourth slit lamp image sample is higher than that of the third slit lamp image sample; Based on each of the fourth slit lamp image samples, the initial keratitis recognition sub-model is trained until convergence to obtain the target keratitis recognition sub-model; wherein the initial keratitis recognition sub-model is a dense convolutional neural network fused with a Gaussian context transformer GCT module.

2. The training method of the automatic keratitis recognition model for low-quality slit lamp images according to claim 1, characterized in that: The step of generating a second slit lamp image sample using an image quality degradation algorithm for each of the first slit lamp image samples comprises: Based on the image quality degradation algorithm, performing at least one of motion blur processing, defocus blur processing and exposure distortion processing on each of the first slit lamp image samples to obtain the second slit lamp image sample; The motion blur processing is implemented by the following formula (1): The defocus blur processing is implemented by the following formula (2): The exposure distortion processing is achieved by the following formula (3): I exposure (x,y)=α·I(x,y)+β (3) Wherein, I(x, y) represents the first slit lamp image sample; I motion (x, y), I defocus (x, y) and I exposure (x, y) respectively represent the second slit lamp image sample generated by using the image quality degradation algorithm on the first slit lamp image sample; (x, y) represent the coordinates of the first slit lamp image sample; K(x, y) represents the motion blur kernel; (i, j) represents the discrete coordinates iterated to the motion blur kernel; (k, l) represents the standard deviation of defocus blur; α and β represent the adjustment coefficients of contrast and brightness.

3. The training method of the automatic keratitis recognition model for low-quality slit lamp images according to claim 1, characterized in that: The initial slit lamp image quality enhancer model also includes a discriminator; The initial slit lamp image quality enhancer model is trained based on each of the training data pairs until convergence to obtain the target slit lamp image quality enhancer model, including: Step A: Based on each of the training data pairs, the SCUNet generation model is trained so that the SCUNet generation model learns the mapping relationship between the second slit lamp image sample and the first slit lamp image sample from each of the training data pairs, and generates a fifth slit lamp image sample with enhanced image quality corresponding to each of the second slit lamp image samples based on the mapping relationship; Step B: inputting the first slit lamp image sample and the fifth slit lamp image sample into the discriminator, so that the discriminator distinguishes the difference between the fifth slit lamp image sample and the corresponding first slit lamp image sample, and adjusts the model parameters of the SCUNet generation model based on the difference; Step C: looping through steps AB until the SCUNet generated model converges to obtain the target slit lamp image quality enhancement sub-model.

4. The method for training a keratitis automatic recognition model for low-quality slit lamp images according to any one of claims 1 to 3, characterized in that: The SCUNet generation model is formed by fusing the Swin-Conv module and the U-Net framework; The Swin-Conv module includes a first convolutional layer, a second convolutional layer, a Swin-Transformer module and a residual convolutional module, and the first convolutional layer is residually connected to the second convolutional layer; The U-Net framework includes a strided convolution module, a transposed convolution module, a plurality of Swin-Conv modules and a convolution layer, and the strided convolution module is residually connected with the transposed convolution module.

5. The method for training a keratitis automatic recognition model for low-quality slit lamp images according to any one of claims 1 to 3, characterized in that: The initial keratitis recognition sub-model is trained based on each of the fourth slit lamp image samples until convergence to obtain the target keratitis recognition sub-model, including: Inputting each of the fourth slit lamp image samples into the initial keratitis recognition sub-model for iterative training until the loss value of the target loss function reaches a preset threshold, thereby obtaining the target keratitis recognition sub-model; The objective loss function is expressed by the following formula (4): Among them, loss(P,Q,W) represents the loss value, P i Indicates that the initial keratitis recognition sub-model outputs the probability of the i-th corneal state category, Q i represents the probability that the true label of the fourth slit lamp image sample is the i-th corneal state category, W i Represents the error weight set for the i-th corneal condition category.

6. The method for training a keratitis automatic recognition model for low-quality slit lamp images according to any one of claims 1 to 3, characterized in that: The method further comprises: Inputting each of the first slit lamp image samples into the initial keratitis identification sub-model for iterative training until the loss value of the target loss function reaches a preset threshold, thereby obtaining the initial weight parameters of the target keratitis identification sub-model; The initial keratitis recognition sub-model is trained based on each of the fourth slit lamp image samples until convergence to obtain the target keratitis recognition sub-model, including: Loading the initial weight parameters obtained through training based on each of the first slit lamp image samples into the initial keratitis recognition sub-model; Based on each of the fourth slit lamp image samples, the initial keratitis identification sub-model is iteratively trained twice until the loss value of the target loss function reaches a preset threshold, thereby obtaining the target keratitis identification sub-model.

7. A method for automatic identification of keratitis in low-quality slit lamp images, characterized in that: include: Acquire slit lamp images of the patient's eyes to be identified; In the case where the slit lamp image to be identified is of a low-quality image quality category, inputting the slit lamp image to be identified into a target slit lamp image quality enhancement sub-model to obtain an enhanced slit lamp image output by the target slit lamp image quality enhancement sub-model; The target slit lamp image quality enhancer model is used to repair the noise existing in the low-quality slit lamp image; Inputting the enhanced slit lamp image into a target keratitis recognition sub-model to obtain a corneal state corresponding to the slit lamp image to be recognized output by the target keratitis recognition sub-model; the target keratitis recognition sub-model is used to recognize the corneal state corresponding to the slit lamp image; The target slit lamp image quality enhancement sub-model and the target keratitis recognition sub-model are trained by the training method for the automatic keratitis recognition model for low-quality slit lamp images as described in any one of claims 1 to 6.

8. The method for automatic keratitis recognition based on low-quality slit lamp images according to claim 7, characterized in that: After acquiring the slit lamp image to be identified of the patient's eye, the method further includes: Inputting the slit lamp image to be identified into an image quality classification model to obtain an image quality category corresponding to the slit lamp image to be identified output by the image quality classification model; When the image quality category is high quality, the slit lamp image to be identified is input into the target keratitis identification sub-model to obtain the corneal state corresponding to the slit lamp image to be identified.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, it implements the training method of the automatic keratitis recognition model for low-quality slit lamp images as described in any one of claims 1 to 6, or the automatic keratitis recognition method for low-quality slit lamp images as described in any one of claims 7 to 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the training method of the automatic keratitis recognition model for low-quality slit lamp images as described in any one of claims 1 to 6, or the automatic keratitis recognition method for low-quality slit lamp images as described in any one of claims 7 to 8.

Images