Cataract slit lamp image grading identification method based on convolutional neural network

Abstract

The invention provides a cataract slit lamp image grading identification method based on a convolutional neural network, and relates to the technical field of medical image intelligent analysis, and the method comprises the steps: collecting and marking a plurality of groups of slit lamp images, constructing a multilayer optical imaging model composed of a cornea layer, an anterior capsule layer, a turbid layer and a background layer, and carrying out the grading identification of the slit lamp images. And an optical scattering generation mechanism is established by using each layer of scattering nucleus. The method comprises the following steps: constructing a cataract display module based on an optical model, performing layer decomposition and display processing on a training image, jointly optimizing a scattering kernel parameter through a reconstruction error and a gradient consistency error, inputting a display image into a convolutional neural network grading model, synchronously training the display module and the grading model through a joint loss function, and obtaining a classification result. And obtaining a target hierarchical network. In the inference stage, optical inversion and display processing are performed on a slit lamp image to be detected, a generated display image is input into the grading model to obtain the grading probability and confidence, and a final cataract grading result is determined.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical image analysis technology, and in particular to a method for hierarchical recognition of slit-lamp images of cataracts based on convolutional neural networks. Background Technology

[0002] Cataracts are one of the leading causes of vision impairment worldwide, and their clinical diagnosis typically relies on slit-lamp microscopy to observe and grade the degree of lens opacity. Existing cataract grading methods mainly include two categories: manual assessment and automatic identification based on digital images. Manual assessment depends on the experience of clinicians, and the grading standards vary among different physicians, resulting in problems such as strong subjectivity and poor repeatability.

[0003] With the development of medical imaging digitization, deep learning-based automatic grading methods have gradually attracted attention. However, existing deep learning methods usually take raw slit-lamp images as input directly, and their grading performance is easily affected by factors such as strong corneal surface reflection, glare streaks, pupil boundary reflection, local underexposure, and concomitant corneal lesions, resulting in unstable feature extraction and affecting recognition accuracy and the model's generalization ability under different acquisition conditions.

[0004] Secondly, existing methods generally lack modeling of the slit-lamp image formation mechanism, ignoring the physical laws of light propagation and scattering between the cornea, anterior capsule, opaque areas, and background tissues. This makes it difficult for deep networks to learn appearance features from pixel intensity, and they are unable to distinguish between reflective noise and real opaque structures, leading to erroneous classification in complex clinical scenarios. Therefore, we propose a cataract slit-lamp image classification and recognition method based on convolutional neural networks. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a cataract slit-lamp image hierarchical recognition method based on convolutional neural networks, thereby solving the technical problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A cataract slit-lamp image hierarchical recognition method based on convolutional neural networks includes the following steps:

[0008] S1. In the processor, multiple sets of cataract slit lamp images from the slit lamp image acquisition device are acquired and labeled to obtain a training sample set containing different cataract grading labels; and based on the optical path structure of the slit lamp and the ocular tissue structure, a multi-layer optical imaging model describing the corneal layer, lens layer and situ layer is established in the processor.

[0009] S2. During the training phase, the original slit-lamp images from the training sample set are input into the cataract visualization module corresponding to the multilayer optical imaging model. The multilayer optical imaging model is used to perform layer decomposition and optical scattering inversion on the original slit-lamp images to obtain cataract visualization images corresponding to the lens opacity distribution. The scattering kernel parameters in the cataract visualization module are optimized by minimizing the reconstruction error between the original slit-lamp images and the reconstructed images obtained by the multilayer optical imaging model.

[0010] S3. Input the cataract imaging image into the convolutional neural network hierarchical model, extract the hierarchical features that characterize the spatial distribution and intensity of lens opacity in the convolutional neural network hierarchical model, calculate the hierarchical loss according to the cataract hierarchical label corresponding to the cataract imaging image, and jointly minimize the reconstruction error and hierarchical loss during the training phase to obtain the target convolutional neural network hierarchical model coupled with the cataract imaging module.

[0011] S4. In the recognition stage, the slit-lamp image of the cataract to be tested is input into the cataract visualization module. According to the multi-layer optical imaging model and scattering kernel parameters determined in the training stage, the slit-lamp image of the cataract to be tested is decomposed into layers and optical scattering inversion is performed to generate the corresponding cataract visualization image.

[0012] S5. Input the image of the cataract to be tested into the target convolutional neural network grading model, obtain the grading probability vector representing multiple cataract grading levels in the target convolutional neural network grading model, and determine the cataract grading result corresponding to each cataract grading level based on the grading probability vector.

[0013] S6. The processor outputs the cataract grading result to be tested as the grading recognition result of the slit-lamp image of the cataract to be tested. By introducing a multi-layer optical imaging model and its reconstruction error constraints during the training phase, the degree of lens opacity can still be stably and robustly graded and recognized even when there is reflection, glare or combined corneal lesion interference in the slit-lamp image of the cataract to be tested.

[0014] S1 specifically involves: acquiring multi-source slit-lamp images from a slit-lamp image acquisition device, and labeling each slit-lamp image with a corresponding cataract grading label according to a preset cataract grading standard to form a training sample set; determining the optical structure composition of the corneal layer, anterior capsule layer, opacity layer, and background layer based on the slit-lamp imaging optical path, thus obtaining a multi-layer optical imaging structure description; establishing a multi-layer optical imaging model to simulate the refraction and scattering process of light after passing through various ocular tissues based on the multi-layer optical imaging structure description, and setting the initial scattering kernel parameters for each layer; constructing a cataract visualization module for optically visualizing cataract areas based on the structural definition of the multi-layer optical imaging model, which is used to extract tissue layer features related to the opacity area from the original slit-lamp images; dividing the training sample set into a training set and a validation set, and loading the multi-layer optical imaging model and the cataract visualization module into the training environment for subsequent model training.

[0015] S2 specifically involves: inputting the original slit-lamp images from the training set into the cataract visualization module; extracting corneal, opacity, and background features through layer decomposition; performing optical scattering inversion on the original slit-lamp images based on a multi-layer optical imaging model to obtain a reconstructed slit-lamp image; calculating the reconstruction error between the original and reconstructed slit-lamp images to measure whether the scattering kernel parameters of the multi-layer optical imaging model accurately describe the real imaging process; updating the scattering kernel parameters of each tissue layer in the cataract visualization module based on the reconstruction error using gradient descent to make the opacity features more closely resemble the real lens opacity distribution; and repeating layer decomposition, optical inversion, error calculation, and parameter updates until the reconstruction error converges to a set threshold, resulting in an optimized cataract visualization module.

[0016] S3 specifically involves: using the optimized cataract visualization module to visualize the cataract layer, obtaining the corresponding cataract visualization image; inputting the cataract visualization image into a convolutional neural network hierarchical model to extract hierarchical features that characterize the spatial distribution and intensity of lens opacity; calculating the hierarchical loss of the convolutional neural network hierarchical model based on the cataract hierarchical label corresponding to the cataract visualization image; weighting the reconstruction error and hierarchical loss according to a preset ratio to form a joint loss, which is used to synchronously constrain the cataract visualization module and the convolutional neural network hierarchical model; and synchronously updating the parameters of the visualization module and the convolutional neural network hierarchical model based on the joint loss during the training phase to obtain a target convolutional neural network hierarchical model suitable for cataract visualization images.

[0017] S4 specifically involves: inputting the slit-lamp image to be tested into the cataract visualization module; performing optical scattering inversion on the slit-lamp image to be tested according to the multi-layer optical imaging structure and scattering kernel parameters determined during the training phase; performing layer decomposition on the slit-lamp image to be tested based on the optical inversion results to obtain the characteristics of the opacity layer to be tested; generating a cataract visualization image to be tested based on the characteristics of the opacity layer to be tested, which is used to characterize the degree of lens opacity in the slit-lamp image to be tested; and storing the cataract visualization image to be tested as valid input for the grading recognition phase.

[0018] S5 specifically involves: inputting the cataract imaging image to be tested into the target convolutional neural network grading model; extracting the turbidity distribution features to be tested from the target convolutional neural network grading model to obtain a grading probability vector containing multiple cataract grades; determining the corresponding cataract grading grade based on the grading probability vector; performing confidence verification on the cataract grading grade, and triggering an alternative verification strategy if the confidence is lower than the threshold; proceeding to the next step if the confidence is greater than or equal to the threshold; and using the verified cataract grading grade as the target grading result for the slit-lamp image to be tested.

[0019] S6 specifically involves: receiving the target grading result of the slit-lamp image under test in the processor; associating the target grading result with the corresponding analysis information, including the revealed image, grading probability vector, and confidence parameters; outputting the target grading result to the display interface for doctors to view or for use by the automated medical system; generating an optional cataract progression suggestion set based on the target grading result to assist in diagnosis and treatment decisions; storing the target grading result and related information in the medical data recording module to support subsequent queries, reviews, or model retraining; and completing the grading recognition process for cataract slit-lamp images. By introducing the revealing constraints of a multi-layer optical imaging model, a stable and robust grading recognition effect is achieved even in the presence of reflections, glare, or interference from combined corneal lesions.

[0020] The beneficial effects of this invention are as follows:

[0021] This invention constructs a multi-layer optical imaging model consisting of the cornea, anterior capsule, opacity layer, and background layer, explicitly describing the propagation, refraction, and scattering of light in each tissue layer. This enables the visualization module to distinguish between reflective noise and true opacity structures at the pixel level. A consistency constraint is established between the original image, visualized image, and reconstructed image using optical scattering inversion formulas. Furthermore, the scattering kernel parameters are jointly optimized by reconstruction error and gradient consistency error, allowing the visualized image to more accurately reflect the scattering intensity and spatial structure of the opacity region of the lens. This inversion-reconstruction mechanism effectively suppresses strong reflections, glare streaks, and corneal surface reflection interference, improving the realism of the visualized image and the stability of subsequent grading models.

[0022] This invention enables end-to-end joint optimization of the explicitation module and the convolutional neural network hierarchical model by synchronizing the reconstruction error and classification loss. Compared to traditional hierarchical networks that extract features only from pixel textures, the joint training mechanism allows the hierarchical model to fully utilize the physically explicit turbidity structure, significantly improving its ability to recognize subtle differences in turbidity levels.

[0023] This invention generates cataract imaging images through a visualization module, which can intuitively display the brightness distribution, edge contours, and scattering intensity changes in the opaque areas of the lens. During the output stage, pseudo-color overlay or visualization enhancement makes the model's focus areas more transparent and interpretable, facilitating clinical verification or review and improving the algorithm's reliability in real-world scenarios. A confidence index is constructed using Softmax probability and information entropy, automatically triggering a review strategy when confidence is insufficient, avoiding erroneous classifications caused by imaging abnormalities, reflective interference, or special lesions. This mechanism enhances the stability and reliability of the system output, making it suitable for clinical auxiliary judgment under multiple devices and conditions. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the cataract slit-lamp image hierarchical recognition method based on convolutional neural networks according to the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Example 1: As Figure 1 As shown, this embodiment provides a cataract slit-lamp image grading and recognition method based on convolutional neural networks, including the following steps:

[0027] S1. In the processor, multiple sets of cataract slit lamp images from the slit lamp image acquisition device are acquired and labeled to obtain a training sample set containing different cataract grading labels; and based on the optical path structure of the slit lamp and the ocular tissue structure, a multi-layer optical imaging model describing the corneal layer, lens layer and situ layer is established in the processor.

[0028] S2. During the training phase, the original slit-lamp images from the training sample set are input into the cataract visualization module corresponding to the multilayer optical imaging model. The multilayer optical imaging model is used to perform layer decomposition and optical scattering inversion on the original slit-lamp images to obtain cataract visualization images corresponding to the lens opacity distribution. The scattering kernel parameters in the cataract visualization module are optimized by minimizing the reconstruction error between the original slit-lamp images and the reconstructed images obtained by the multilayer optical imaging model.

[0029] S3. Input the cataract imaging image into the convolutional neural network hierarchical model, extract the hierarchical features that characterize the spatial distribution and intensity of lens opacity in the convolutional neural network hierarchical model, calculate the hierarchical loss according to the cataract hierarchical label corresponding to the cataract imaging image, and jointly minimize the reconstruction error and hierarchical loss during the training phase to obtain the target convolutional neural network hierarchical model coupled with the cataract imaging module.

[0030] S4. In the recognition stage, the slit-lamp image of the cataract to be tested is input into the cataract visualization module. According to the multi-layer optical imaging model and scattering kernel parameters determined in the training stage, the slit-lamp image of the cataract to be tested is decomposed into layers and optical scattering inversion is performed to generate the corresponding cataract visualization image.

[0031] S5. Input the image of the cataract to be tested into the target convolutional neural network grading model, obtain the grading probability vector representing multiple cataract grading levels in the target convolutional neural network grading model, and determine the cataract grading result corresponding to each cataract grading level based on the grading probability vector.

[0032] S6. The processor outputs the cataract grading result to be tested as the grading recognition result of the slit-lamp image of the cataract to be tested. By introducing a multi-layer optical imaging model and its reconstruction error constraints during the training phase, the degree of lens opacity can still be stably and robustly graded and recognized even when there is reflection, glare or combined corneal lesion interference in the slit-lamp image of the cataract to be tested.

[0033] S1 specifically includes the following sub-steps:

[0034] S110. Slit-lamp image acquisition and annotation steps: Acquire multi-source slit-lamp images from the slit-lamp image acquisition device. The resolution of the slit-lamp images should be within the range of 512×512 to 2048×2048, and the image format should be 8-bit or 16-bit grayscale. According to the clinical cataract grading standard (e.g., LOCS III or an equivalent clinical scoring system), two or more ophthalmologists should independently grade and annotate each slit-lamp image. If the grading results of two physicians are inconsistent, a third senior physician should verify the consistency, and finally form a consensus-based cataract grading label.

[0035] Furthermore, to enhance the robustness of the training data, data augmentation processing was performed on the training sample set, including brightness enhancement, contrast stretching, random rotation (within ±10°), salt-and-pepper noise addition, random cropping (cropping ratio of 0.9 to 1.0), and gamma correction (coefficient range of 0.8 to 1.2); finally, a training sample set containing no less than 1000 images with cataract grading labels was constructed.

[0036] To ensure consistency in the data augmentation process across different imaging devices, gamma correction is calculated using the following formula:

[0037]

[0038] in, Indicates the input pixel value; This represents the corrected pixel value. The range is 0.8 to 1.2; this formula ensures that the changes in image brightness conform to the actual illumination response of slit lamp imaging.

[0039] S120. Steps for defining the multi-layered optical structure of the eye: Based on the actual imaging optical path structure of the slit-lamp image, the optical structure of the eye is divided into the corneal layer, anterior capsule layer, opacity layer, and background layer, wherein:

[0040] The corneal layer corresponds to the optical refraction and surface reflection characteristics of the corneal tissue, with a refractive index ranging from 1.35 to 1.38; the anterior capsule layer corresponds to the refraction and weak scattering characteristics of the anterior capsule tissue of the lens; the opacity layer corresponds to the enhanced scattering distribution in the opaque region of the lens, with a scattering coefficient ranging from 0.5 to 3.0; and the stigma layer corresponds to the weak scattering region of the internal aqueous humor, vitreous body, and stigma structures.

[0041] The aforementioned refractive index, scattering coefficient, and layer thickness parameters can be obtained from publicly available clinical literature, slit-lamp imaging parameter manuals, or actual measurement data, and those skilled in the art can construct corresponding optical layer models based on these parameters.

[0042] S130, Steps for constructing a multilayer optical scattering imaging model: Based on the optical structure of the eye, establish a model including the corneal scattering kernel. Turbidity layer scattering nuclei and background scattering kernel A multilayer optical scattering imaging model, wherein:

[0043] corneal scattering nucleus The size ranges from 3×3 to 7×7; turbidity scattering nuclei The size ranges from 5×5 to 11×11; background scattering kernel The size range is 3×3 to 5×5; the initial value of the scattering kernel can be initialized based on the Gaussian kernel, with a variance range of 0.5 to 2.0.

[0044] The mathematical expression for the multilayer scattering model is:

[0045]

[0046] in, Indicates the incident light field; Represents the response matrix of the turbidity layer structure; Represents the background layer response matrix; This represents a two-dimensional convolution operation. This represents the original slit lamp image that was input.

[0047] Incident light field The initial light intensity distribution when a slit lamp source illuminates the corneal surface can be obtained in the following way:

[0048] Reference images acquired using a slit lamp device on a uniform standard surface (such as a diffuse reflector); or standardized light intensity templates generated based on slit width and illumination intensity parameters; those skilled in the art can obtain the relevant information based on specific device parameters. .

[0049] The initial value of the scattering kernel is a Gaussian kernel:

[0050]

[0051] S140. Cataract visualization module construction steps: Based on the multilayer optical scattering imaging model, a cataract visualization module for generating cataract visualization images is constructed. The cataract visualization module adopts a three-branch convolutional structure, including:

[0052] Corneal layer branch: It consists of 3 to 5 convolutional layers with a kernel size of 3×3, an activation function of ReLU, and a feature channel number ranging from 16 to 64;

[0053] Turbidity layer branch: consists of 4 to 7 convolutional-deconvolutional layers, with convolutional kernel size ranging from 3×3 to 5×5, activation function being ReLU or LeakyReLU, and feature channel number ranging from 32 to 128;

[0054] Background layer branch: Consists of 2-4 convolutional layers with a kernel size of 3×3 and a channel count ranging from 8 to 32; the features from the three branches are fused using a skip-connection, achieved through a concat operation on the feature dimension.

[0055]

[0056] in This is a feature map of the corneal layer. This is a characteristic image of the lens opacity layer. It is the feature map of the background layer; This indicates a stitching operation along the channel direction; after stitching, a 3×3 convolutional layer is used for feature compression to enhance the structural consistency of the visualized image; and after fusion, a convolutional layer is added to generate a cataract visualized image, which is an enhanced grayscale image or a pseudo-color image with 1 or 3 channels.

[0057] If the output is a pseudo-color image, it should be generated using a Look-Up Table (LUT) or a three-channel convolution method.

[0058] LUT method: Increase the intensity of a single channel Map to a preset color lookup table;

[0059] Three-channel convolution method: A pseudo-color display image is generated through a convolutional layer with a kernel size of 1×1 and 3 output channels.

[0060] Both methods can be implemented by those skilled in the art without altering the turbidity distribution structure represented by the revealed image.

[0061] S150, Training data preparation steps: Divide the training sample set into training set, validation set and test set in a ratio of 8:1:1; Load the cataract visualization module and multilayer optical scattering imaging model on the GPU computing platform. The GPU model can be NVIDIA RTX series or equivalent graphics processor.

[0062] The training process uses the Adam or SGD optimizer, with an initial learning rate ranging from 1e-4 to 1e-3, a batch size ranging from 16 to 64, and a training epoch number ranging from 50 to 300. An early-stop mechanism is used during training; if the validation set loss does not decrease within 10 to 20 consecutive epochs, training is terminated early.

[0063] The training environment and parameter settings described above are all conventional deep learning training methods that can be implemented by those skilled in the art.

[0064] S2 specifically includes the following sub-steps:

[0065] S210, Layer Decomposition Feature Extraction Step: Input the original slit-lamp images from the training set into the cataract visualization module; the cataract visualization module, based on a three-branch convolutional structure, outputs corneal layer feature maps respectively. Turbidity layer characteristic diagram and background layer feature maps ,in:

[0066] The number of channels ranges from 16 to 64; The number of channels ranges from 32 to 128; The number of channels ranges from 8 to 32; the spatial dimensions of the feature map are consistent with those of the input image.

[0067] Corneal layer features are used to characterize the refractive and reflective structures of the anterior corneal surface, opacity layer features are used to characterize the high scattering texture of the opaque areas of the lens, and background layer features are used to characterize the weak scattering background response of the aqueous humor and vitreous humor.

[0068] S220, Optical Scattering Inversion and Reconstruction Steps: Based on the multilayer optical scattering imaging model, construct the optical scattering inversion operator for the original slit lamp image; the mathematical expression for optical scattering inversion is:

[0069]

[0070] in, This represents the reconstructed slit lamp image; , , The scattering kernels represent the corneal layer, the opacity layer, and the background layer, respectively; the size of the scattering kernels ranges from 3×3 to 11×11; the initial values ​​of the scattering kernels are initialized based on Gaussian kernels, and the initial variance ranges from 0.5 to 2.0; the inversion calculation is implemented using standard two-dimensional convolution.

[0071] Through inversion operations, the features of each layer output by the cataract visualization module are mapped back to the complete imaging domain to simulate the scattering process after the slit lamp light penetrates each layer of tissue.

[0072] S230, Reconstruction Error Calculation Steps: Record the original slit-lamp image as... The reconstructed slit lamp image obtained by optical scattering inversion is denoted as Construct a reconstruction error to constrain the explicit module and scattering kernel parameters. Its mathematical expression is:

[0073]

[0074] The first term represents the pixel-level L2 reconstruction error; the second term represents the gradient consistency error, used to constrain and reveal the image edge structure. Represents the image gradient operator; This represents the gradient error weighting coefficient, with a value range of 0.01 to 0.2.

[0075] gradient operator Implemented using Sobel operators, the Sobel horizontal and vertical operators are as follows:

[0076]

[0077] This operator is used to obtain a gradient magnitude map, thereby improving the consistency constraints of the structure edges.

[0078] Reconstruction error is used to measure the optical consistency between the inverted image and the true image.

[0079] S240, Scattering kernel parameter optimization steps: based on reconstruction error To optimize the target, the scattering kernel parameters are updated using the backpropagation algorithm. , , And the convolution parameters of the corneal layer branch, the opacity layer branch, and the background layer branch in the cataract visualization module.

[0080] Specifically, the Adam or SGD optimizer is used to update the gradient of the scattering kernel, where the learning rate of the optimizer ranges from 10. -4 -10 -3 The momentum parameter ranges from 0.8 to 0.99.

[0081] The scattering kernel parameters participate in backpropagation in each iteration, gradually fitting the scattering pattern of real slit-lamp imaging; among them, the update priority of the scattering kernel of the turbidity layer is higher than that of the scattering kernels of the corneal layer and the background layer, so as to converge the scattering structure of the lens turbidity more quickly.

[0082] S250, Convergence Judgment and Manifestation Module Solidification Steps: Repeat steps S210-S240 until the reconstruction error is reached. The reduction stops; the convergence condition is defined as follows: within 10-20 consecutive epochs, the reduction in reconstruction error is less than a threshold. , where the threshold The value range is 10 -4 -10 -3 .

[0083] When the convergence condition is met, the scattering kernel parameters are solidified. , , The network parameters of the cataract visualization module are used as a fixed module for subsequent cataract visualization image generation and hierarchical recognition.

[0084] S3 specifically includes the following sub-steps:

[0085] S310, Cataract visualization image generation steps: Apply the solidified cataract visualization module to the original slit-lamp images in the training set, perform corneal layer feature extraction, opacity layer feature extraction and background layer feature extraction on the original slit-lamp images, and perform feature fusion based on the features of the three layers.

[0086] Cataract visualization images are generated by a fused convolutional layer. The cataract visualization images are 1-channel or 3-channel enhanced grayscale images or pseudo-color images, and their spatial dimensions are consistent with the input images.

[0087] The visualized image is used to enhance the high scattering response of the opaque region of the lens and suppress corneal reflection, glare streaks and background noise, so that the subsequent grading model can learn the opaque structure more accurately.

[0088] S320, Feature extraction steps of the convolutional neural network hierarchical model: Input the cataract visualization image into the convolutional neural network hierarchical model; the convolutional neural network hierarchical model includes:

[0089] Feature extraction section: Consists of 4 to 10 convolutional layers connected in series. The kernel size of each convolutional layer is 3×3 or 5×5, the number of channels ranges from 32 to 256, and the activation function is ReLU or LeakyReLU. Batch normalization is used between convolutional layers to stabilize gradients.

[0090] Spatial compression section: includes a global average pooling layer to compress spatial feature maps into global statistical feature vectors;

[0091] The classification output section includes a fully connected layer and a softmax layer, used to generate probability vectors corresponding to each cataract severity category.

[0092]

[0093] Where P is the classification probability vector, which consists of the predicted probabilities of each category; : The predicted probability of the i-th category; This indicates the number of cataract classification categories.

[0094] A convolutional neural network hierarchical model is used to extract texture gradient features, scattering intensity features, and spatial distribution features that characterize lens opacity in cataract imaging images.

[0095] S330, Steps for calculating the classification loss: Based on the real cataract classification labels corresponding to the cataract imaging images, construct the classification loss for training the convolutional neural network classification model. :

[0096]

[0097] in, Represents the one-hot encoding of the i-th class of real labels; Represents the first in the predicted hierarchical probability vector The grading loss is used to measure the consistency between the prediction results of the convolutional neural network grading model and the actual cataract grading labels.

[0098] Furthermore, to improve the model's ability to detect errors at adjacent levels, Optional addition of hierarchical distance weighting terms:

[0099]

[0100] in, Indicates the actual grade number; The value range is 0.01 to 0.2.

[0101] S340, Joint Loss Construction Steps: Reconstructing the error... With classification loss Weighted fusion is performed to form a joint loss function:

[0102]

[0103] in, The value range is 0.2-0.7; The value range is 0.3-0.8.

[0104] By using joint loss, the cataract visualization module and the convolutional neural network hierarchical model are constrained simultaneously during the training phase, so that the visualization image maintains the authenticity of the optical scattering structure, while the hierarchical model obtains a more accurate representation of the turbidity features.

[0105] The joint optimization steps of S350, the explicit module, and the hierarchical model: using joint loss To optimize the target, the backpropagation algorithm was used to optimize the convolution parameters of the cataract visualization module, all network parameters in the three-branch convolutional structure, and the scattering kernel parameters. , , And the parameters of all convolutional layers, batch normalized layers, and fully connected layers in the hierarchical convolutional neural network model are updated synchronously.

[0106] Joint training is performed using either the Adam or SGD optimizer, with the optimizer's learning rate ranging from 10. -4 ~10 -3 The momentum parameter ranges from 0.8 to 0.99, and the number of training epochs ranges from 50 to 300. During joint training, the decreasing trend of the joint loss is monitored using the validation set. If the decrease in loss is less than a threshold of 10 within 10 to 20 consecutive epochs, the training is considered successful. -4 If the training is terminated, the early-stop mechanism is triggered; finally, the target convolutional neural network hierarchical model is obtained, which is used for cataract hierarchical recognition of subsequent slit-lamp images to be tested.

[0107] S4 specifically includes the following sub-steps:

[0108] S410, Inputting the Slit Lamp Image to be Tested: Input the slit lamp image to be tested into the solidified cataract visualization module; the slit lamp image to be tested has the same input format and resolution range (512×512~2048×2048) as the original slit lamp image used in the training phase, and adopts the same data normalization method (e.g., normalizing the pixel intensity to the range of 0~1 or -1~1) to ensure that the image to be tested is consistent with the optical scattering distribution in the training phase.

[0109] S420, Perform the optical scattering inversion step: based on the multilayer optical scattering imaging model and the solidified scattering kernel parameters. , , Optical scattering inversion is performed on the slit lamp image to be tested; the inversion calculation expression is:

[0110]

[0111] in, This represents the slit lamp image to be reconstructed. , , These represent the features of the corneal layer, the opacity layer, and the background layer extracted by the cataract visualization module, respectively.

[0112] When performing optical scattering inversion, the convolution operation uses the SAME padding mode to make the output image size consistent with the input image; for boundary pixels, reflection padding is used to avoid edge artifacts caused by zero padding.

[0113] The scattering kernel parameters are fixed parameters after optimization and convergence during the training phase and will not be updated. Optical scattering inversion is used to simulate the real scattering effect of light passing through various layers of tissue in the eye, thereby enhancing the accuracy of the visualization results in response to the structure of the turbidity layer.

[0114] S430. Perform the layer decomposition step of the image to be tested: Based on the three-branch convolutional structure of the cataract visualization module, extract corneal layer features from the slit-lamp image to be tested. Turbidity layer characteristics Features of the background layer .

[0115] Among them, corneal layer features are used to remove highly reflective areas on the corneal surface; opacity layer features are used to enhance the opacity texture of the lens; and background layer features are used to suppress intraocular background noise.

[0116] The feature maps at the three levels all maintain the same spatial dimensions as the input image, with the number of channels ranging from 16 to 64, 32 to 128, and 8 to 32, respectively.

[0117] S440. Steps for generating the cataract visualization image to be tested: Input the three-level feature maps into the feature fusion structure of the cataract visualization module, and perform channel-by-channel fusion of corneal layer features, opacity layer features, and background layer features through skip-connection; After fusion, perform 1-2 layers of convolution operations to generate the cataract visualization image to be tested. The format of the displayed image is: a 1-channel grayscale displayed image or a 3-channel pseudo-color displayed image.

[0118] Among them, the imaging can highlight the scattering intensity of the cloudy area of ​​the lens and reduce the influence of corneal surface reflection, glare streaks, pupil boundary interference, corneal scar stray light and other factors on the grading judgment.

[0119] The formula for generating the explicit image is:

[0120]

[0121] in, This indicates a feature-level join operation. This indicates a convolution operation.

[0122] S450, Step 1: Input the solidified image as the grading recognition model: Use the image of the cataract to be tested as input. The image is saved as a valid input for the slit lamp image to be tested; the pixel value range of the image is renormalized to the range of 0 to 1 or -1 to 1 to meet the input requirements of the convolutional neural network hierarchical model; the solidified image of the cataract to be tested is used in the cataract hierarchical identification steps of S510 to S550 to ensure the consistency of optical scattering, the consistency of feature representation and the stability of model inference.

[0123] S5 specifically includes the following sub-steps:

[0124] S510, Step 1: Input the image of the cataract to be tested: Enter the solidified image of the cataract to be tested... Input the target convolutional neural network hierarchical model; before input, normalize the explicit image in the same way as in the training stage (e.g., normalize to the range of 0 to 1 or -1 to 1), and maintain the same spatial size as in the training stage (within the range of 512×512 to 2048×2048) to ensure the consistency between the input distribution and the training distribution.

[0125] The target convolutional neural network hierarchical model includes a feature extraction part, a global average pooling part, and a softmax classification part, and its structure is completely consistent with the training stage.

[0126] S520. Extracting opacity structural features and calculating grading probability: The cataract imaging image to be tested is sequentially processed in the target convolutional neural network grading model as follows:

[0127] Convolutional feature extraction: The gradient features of turbid texture, scattering structure features and spatial distribution features are extracted through 4 to 10 convolutional layers; the kernel size of each convolutional layer is 3×3 or 5×5, the number of channels ranges from 32 to 256, and the activation function is ReLU or LeakyReLU;

[0128] Feature normalization: Batch normalization is applied after each convolutional layer to maintain a stable feature distribution;

[0129] Global feature compression: The spatial feature map is compressed into a global statistical feature vector of length D through a global average pooling layer, where D ranges from 32 to 256;

[0130] Classification probability generation: Input the global statistical feature vector into the fully connected layer and the Softmax layer, and output the cataract grading probability vector. ;in, This represents the total number of cataract classification categories (e.g., 3-5). To avoid numerical overflow during Softmax calculation, the input vector... The log-sum-exp numerical stabilization method is used:

[0131]

[0132] in The i-th logit value (unnormalized score) output by the classification network; Represents the logit vector for all categories; This represents the maximum value in the logit vector, used to improve the numerical stability of Softmax calculation.

[0133] S530, Preliminary classification determination step: Based on the classification probability vector The category number corresponding to the element with the highest probability is used to determine the preliminary cataract classification of the slit-lamp image being tested, denoted as:

[0134]

[0135] in This indicates the preliminary classification result (i.e., the category number corresponding to the highest probability). Indicates the first The predicted probability of each grade level, This indicates the number of grading categories. The preliminary grading results serve as the first-stage grading determination for the slit lamp image under test.

[0136] S540, Confidence Verification and Review Triggering Steps: Calculate the confidence level of the grading probability vector to determine the reliability of the model for the preliminary grading results; typical methods for calculating the confidence level include:

[0137]

[0138] Alternatively, you can use the information entropy method:

[0139]

[0140] Among them, with "Closer" indicates higher confidence; Conf is the confidence level of the image grading result.

[0141] Confidence threshold During the training phase, the ROC curve analysis of the validation set was used to determine the selection of the Youden index. The maximum threshold is used as the default value; in actual deployment... The value range is 0.6 to 0.9, which can be slightly adjusted according to the actual situation of the hospital; (Sensitivity): Indicates the ability to detect positive samples; (Specificity): Indicates the ability to exclude negative samples.

[0142] If the confidence level is lower than the preset threshold If the verification strategy is triggered, the displayed image and the preliminary grading result will be marked as "low confidence" for clinicians to conduct manual review, or for the system to execute a secondary inference process based on a cross-model.

[0143] S550. Steps to determine the final classification and identification result: When the confidence level is not lower than the threshold At that time, the initial cataract classification level will be determined. The final cataract grading identification result is determined from the slit-lamp image to be tested; the final identification result will be used as the input content for the subsequent output and storage steps S610 to S660.

[0144] By performing deep convolution feature extraction, grading probability inference, and confidence verification on cataract imaging images, the final grading results are guaranteed to remain stable and robust under interference from reflections, glare, pupil boundary noise, and corneal lesions.

[0145] S6 specifically includes the following sub-steps:

[0146] S610, Grading Result Receiving Step: Receive the final cataract grading identification result output by S550. And simultaneously receive the corresponding hierarchical probability vector. and confidence parameters The final cataract classification and identification results are inferred based on the deep convolutional features of the cataract imaging image, which serves as the core input for the system's subsequent display, analysis, and data recording.

[0147] S620, Step 6: Assembling the grading results with auxiliary information: Assembling the final cataract grading identification results... Corresponding cataract imaging images Hierarchical probability vector and confidence parameters Perform associated storage;

[0148] During the assembly process, according to the preset data structure, the above four types of information are organized into data record units composed of the fields "grade_result, enhanced_image, grade_probabilities, confidence_score";

[0149] Wherein, “grade_result” represents the final cataract grade; “enhanced_image” represents the image of the cataract to be visualized; “grade_probabilities” represents the probability vector output by Softmax; and “confidence_score” represents the confidence score calculated in S540.

[0150] Data recording units are used to improve the interpretability and traceability of output data.

[0151] S630, Grading Result Display Steps: Display the data recording unit on a display terminal, which can be a medical display screen, a computer screen, or a mobile terminal interface; the displayed content includes: the final grading result. Cataract imaging images and probability vectors are presented in the form of bar charts or line charts, along with confidence values ​​and grading information. The display interface overlays pseudo-color areas or highlighted areas on the imaging images to highlight the cloudy areas of interest to the model, creating an interpretable visualization effect. The display steps allow clinicians to intuitively view the degree of cataract cloudiness and make auxiliary judgments through the imaging images.

[0152] S640. Steps for generating cataract progression trends and follow-up recommendations: Based on the final cataract grading identification results... Based on the cataract grade and historical ipsilateral optical imaging changes, this data is used to generate cataract progression trends and clinical follow-up recommendations, including:

[0153] If the severity level is mild (e.g., level one or level two), the output will be "Recommended follow-up period is 6 to 12 months";

[0154] If the severity level is moderate (e.g., level 2 or 3), the output should be "The recommended follow-up period is 3 to 6 months, and the impact on visual function should be assessed".

[0155] If the severity level is severe (e.g., level three or four), the output will be "Further examination and evaluation of surgical indications are recommended".

[0156] The above suggestions are auxiliary prompts automatically generated by the system based on the grading results. They are not diagnostic conclusions and are intended for clinicians' reference.

[0157] S650, Data storage steps: Store data record units in a database or local persistent storage module; the storage format can be JSON, binary serialization format, or database structured record format (e.g., MySQL / PostgreSQL table records); the stored content includes the final grading level, visualized image, probability vector, confidence level, and follow-up recommendations; data storage uses a timestamp indexing method so that the system can quickly index the corresponding records when performing trend analysis, model retraining, batch review, or medical record retrieval in the future.

[0158] S660, Grading Recognition Process Completion Steps: Complete the grading recognition process for cataract slit-lamp images; wherein, by constructing a joint training mechanism for the cataract visualization module, optical scattering inversion model and convolutional neural network grading model, and combining confidence verification and interpretability display process, the system can still output stable, reliable and traceable cataract grading recognition results under conditions of strong reflection, glare stripes, pupil boundary noise and corneal lesion interference.

[0159] Steps S610 to S660 enable the reception, organization, display, analysis, and storage of cataract classification and identification results, thus forming a closed loop in the entire identification process technically.

[0160] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0161] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0162] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0163] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0164] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0165] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0166] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0167] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0168] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0169] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A cataract slit lamp image grading recognition method based on a convolutional neural network, characterized in that, The method comprises the following steps: S1, collect and label multiple sets of cataract slit lamp images to obtain a training sample set containing different cataract grading labels; based on the optical path structure of the slit lamp and the eye tissue structure, a multi-layer optical imaging model is established to describe the corneal layer, the lens layer and the background layer; S2, input the original slit lamp image in the training sample set into the cataract visualization module corresponding to the multi-layer optical imaging model, use the multi-layer optical imaging model to perform layer decomposition and optical scattering inversion on the original slit lamp image, obtain the cataract visualization image corresponding to the lens turbidity distribution, and optimize the scattering kernel parameters in the cataract visualization module by minimizing the reconstruction error between the original slit lamp image and the reconstructed image obtained by the multi-layer optical imaging model; S3, input the cataract visualization image into the convolutional neural network grading model, extract the grading features representing the spatial distribution and turbidity intensity of the lens turbidity in the convolutional neural network grading model, calculate the grading loss according to the cataract grading label corresponding to the cataract visualization image, and minimize the reconstruction error and the grading loss in the training stage to obtain the target convolutional neural network grading model coupled with the cataract visualization module; S4, input the to-be-tested cataract slit lamp image into the cataract visualization module, perform layer decomposition and optical scattering inversion on the to-be-tested cataract slit lamp image according to the multi-layer optical imaging model and the scattering kernel parameters determined in the training stage, and generate the corresponding to-be-tested cataract visualization image; S5, input the to-be-tested cataract visualization image into the target convolutional neural network grading model, obtain the grading probability vector representing multiple cataract grading levels in the target convolutional neural network grading model, and determine the to-be-tested cataract grading result corresponding to each cataract grading level based on the grading probability vector. 2.The cataract slit-lamp image grading recognition method based on a convolutional neural network according to claim 1, characterized in that, Further comprising: S6, output the to-be-tested cataract grading result as the grading recognition result of the to-be-tested cataract slit lamp image by the processor, wherein by introducing the multi-layer optical imaging model and its reconstruction error constraint in the training stage, the lens turbidity degree can still be stably and robustly graded and recognized when the to-be-tested cataract slit lamp image is disturbed by reflection, glare or combined corneal lesions. 3.The cataract slit-lamp image grading recognition method based on convolutional neural network according to claim 1, characterized in that, S1 specifically comprises: Collecting multiple source slit lamp images from a slit lamp image collection device, and labeling a corresponding cataract grading label for each slit lamp image according to a preset cataract grading standard to form a training sample set; According to the slit lamp imaging light path, the optical structure composition of the corneal layer, the anterior capsule layer, the turbidity layer and the background layer is determined to obtain a multi-layer optical imaging structure description; based on the multi-layer optical imaging structure description, a multi-layer optical imaging model is established to simulate the refraction and scattering process of light passing through each eye tissue, and the initial scattering kernel parameters of each layer are set; According to the structure definition of the multi-layer optical imaging model, a cataract visualization module for optical visualization of the cataract region is constructed, which is used to extract the tissue layer features related to the turbidity region from the original slit lamp image; the training sample set is divided into a training set and a validation set, and the multi-layer optical imaging model and the cataract visualization module are loaded into a training environment for subsequent model training. 4.The cataract slit-lamp image grading recognition method based on convolutional neural network according to claim 1, characterized in that, S2 specifically comprises: The original slit lamp image in the training set is input into the cataract visualization module, and the corneal layer feature, the turbidity layer feature and the background layer feature are extracted through layer decomposition operation respectively; based on the multi-layer optical imaging model, the original slit lamp image is optically scattered and inverted to obtain a reconstructed slit lamp image; The reconstruction error between the original slit lamp image and the reconstructed slit lamp image is calculated, which is used to measure whether the scattering kernel parameters of the multi-layer optical imaging model accurately describe the real imaging process; Based on the reconstruction error, the scattering kernel parameters corresponding to each tissue layer in the cataract visualization module are updated by gradient descent, so that the visualized turbidity layer feature is closer to the real lens turbidity distribution; the layer decomposition, optical inversion, error calculation and parameter updating are repeatedly performed until the reconstruction error converges to a set threshold, and the optimized cataract visualization module is obtained.

5. The cataract slit lamp image grading recognition method based on a convolutional neural network according to claim 4, characterized in that, S3 specifically is: The cataract layer is visualized by the optimized cataract visualization module to obtain a corresponding cataract visualization image; the cataract visualization image is input into the convolutional neural network hierarchical model to extract hierarchical features that can represent the spatial distribution and turbidity intensity of the lens turbidity; According to the cataract classification label corresponding to the cataract visualization image, the hierarchical loss of the convolutional neural network hierarchical model is calculated; the reconstruction error and the hierarchical loss are weighted according to a preset proportion to form a joint loss, which is used to simultaneously constrain the cataract visualization module and the convolutional neural network hierarchical model; the parameters of the visualization module and the convolutional neural network hierarchical model are simultaneously updated based on the joint loss during the training phase to obtain a target convolutional neural network hierarchical model suitable for cataract visualization images.

6. The cataract slit lamp image grading recognition method based on a convolutional neural network according to claim 3, characterized in that, S4 specifically is: The to-be-tested slit lamp image is input into the cataract visualization module; optical scattering inversion is performed on the to-be-tested slit lamp image according to the multi-layer optical imaging structure and scattering kernel parameters determined in the training phase; Based on the optical inversion result, the to-be-tested slit lamp image is layer-decomposed to obtain to-be-tested turbidity layer features; The to-be-tested cataract visualization image is generated according to the to-be-tested turbidity layer features, which is used to represent the degree of lens turbidity in the to-be-tested slit lamp image; The to-be-tested cataract visualization image is stored as an effective input for the hierarchical identification phase.

7. The Convolutional Neural Network based grading recognition method for cataract slit lamp images according to claim 1, characterized in that, S5 specifically is: The to-be-tested cataract visualization image is input into the target convolutional neural network hierarchical model; the to-be-tested turbidity distribution features are extracted in the target convolutional neural network hierarchical model to obtain a hierarchical probability vector containing multiple cataract grades; The corresponding cataract classification grade is determined based on the hierarchical probability vector; the confidence of the cataract classification grade is checked, if the confidence is lower than a threshold, an alternative review strategy is triggered; if the confidence is greater than or equal to the threshold, the next step is entered; The checked cataract classification grade is taken as the target classification result of the to-be-tested slit lamp image. 8.The cataract slit-lamp image grading recognition method based on convolutional neural network according to claim 2, characterized in that, S6 specifically is: The target classification result of the to-be-tested slit lamp image is received in the processor; the target classification result is associated with corresponding analysis information, including the visualization image, the hierarchical probability vector and the confidence parameter; the target classification result is output to a display interface for doctors to view or for an automatic medical system to call; based on the target classification result, an optional cataract progression suggestion set is generated to assist in diagnosis and treatment decision-making.

9. The cataract slit lamp image grading recognition method based on a convolutional neural network according to claim 8, characterized in that, S6 further comprises: storing the target grading result and related information into the medical data record module to support subsequent query, review or model retraining; completing the grading identification process of the cataract slit lamp image, wherein by introducing the visualization constraint of the multi-layer optical imaging model, stable and robust grading identification effect is achieved even in the presence of interference such as reflection, glare or combined corneal lesions.

Images