AI-based learning system for anterior segment disease recognition using smartphone macro imaging

Abstract

This invention relates to the field of artificial intelligence-assisted diagnosis, and discloses an AI-based learning system for anterior segment disease recognition based on smartphone macro imaging. The system includes: a perception acquisition module for real-time acquisition of inertial measurement data from the smartphone; a multi-scale fusion enhancement module for registering and wavelet decomposition of the multi-frame images to generate enhanced images; a quality verification module for non-reference image quality assessment and diagnostic relevance verification of the enhanced images; a disease recognition module for inputting qualified images into a pre-trained deep learning model; and a self-learning optimization module for optimization and updating. By performing optical flow registration on the acquired multi-frame images, optimizing the fusion weights using information theory and optical flow confidence, and employing iterative guided filtering and modulation transfer function compensation based on pathological priors, high-quality enhanced images are generated. This achieves high-fidelity imaging and accurate recognition of minute pathological features such as conjunctival hyperemia and corneal opacity under handheld shooting conditions.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence-assisted diagnostic technology, specifically to an AI-based learning system for anterior segment disease recognition based on smartphone macro imaging. Background Technology

[0002] With the accelerating aging of the population and the increasing frequency of electronic device use, the incidence of anterior segment diseases is showing a year-on-year upward trend. If anterior segment diseases are not detected and intervened in a timely manner, they may lead to vision loss or even blindness, placing a heavy burden on patients' quality of life and the public health system. Therefore, achieving early screening and accurate diagnosis of anterior segment diseases has significant clinical and social value.

[0003] Currently, existing anterior segment imaging solutions based on smartphones typically use an external macro lens in conjunction with the phone's camera to acquire images, and then use a mobile application to preview, store, and transmit the images. While leveraging the portability and widespread availability of smartphones has provided a new technical approach for anterior segment examinations, the acquired images often suffer from problems such as motion blur, local overexposure or underexposure, and poor focus due to physiological micro-tremors of the hand, slight movements of the subject, and changes in ambient lighting. This results in the loss of pathological details such as conjunctival congestion and corneal opacity. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an AI-based learning system for anterior segment disease recognition based on smartphone macro imaging, which solves the problem of lost pathological feature details such as conjunctival congestion and corneal opacity caused by existing smartphone-based anterior segment imaging.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an AI-based learning system for anterior segment disease recognition based on smartphone macro imaging, comprising:

[0006] The sensing and acquisition module is used to acquire inertial measurement data of the smartphone in real time, calculate the intensity of handshake, and trigger multi-frame burst shooting when the intensity of the shake exceeds a preset threshold to obtain multi-frame images;

[0007] The multi-scale fusion enhancement module is used to register and decompose the multi-frame images to obtain multi-scale sub-band coefficients, fuse each sub-band coefficient based on information theory and optical flow confidence, perform iterative guided filtering enhancement on the fused reconstructed image, and perform modulation transfer function compensation based on pathological priors to generate an enhanced image.

[0008] The quality verification module is used to perform no-reference image quality assessment and diagnostic correlation verification on the enhanced image. When the quality indicators meet the preset conditions, the enhanced image is output to form a qualified image.

[0009] The disease identification module is used to input qualified images into a pre-trained deep learning model, classify anterior segment diseases, and output the classification results.

[0010] The self-learning optimization module is used to collect historical diagnostic data and optimize and update it with the goal of maximizing diagnostic accuracy.

[0011] By employing the above technical solution, real-time sensing of handheld shaking and adaptive triggering of multi-frame continuous shooting are used to perform optical flow registration, wavelet decomposition, and frequency domain fusion on the acquired multi-frame images. The fusion weights are optimized by combining information theory and optical flow confidence, and iterative guided filtering and modulation transfer function compensation based on pathological priors are used to generate high-quality enhanced images. The enhanced images undergo no-reference quality assessment and diagnostic relevance verification to ensure that the input images are clear and the pathological features are significant. A deep learning model is used for accurate classification. Parameters are continuously iterated based on historical data, thereby improving the preservation of anterior segment pathological details and the image signal-to-noise ratio. This achieves high-fidelity imaging and accurate identification of minute pathological features such as conjunctival congestion and corneal opacity under handheld shooting conditions, solving the problem of loss of pathological feature details due to factors such as shaking and lighting in existing smartphone-based anterior segment imaging.

[0012] Preferably, the step of calculating the handheld shake intensity and triggering multi-frame burst shooting when the shake intensity exceeds a preset threshold to obtain multi-frame images includes the following steps:

[0013] It reads gyroscope and accelerometer data from the smartphone at a preset frequency;

[0014] Based on the gyroscope and accelerometer data, the current jitter intensity is calculated using an evaluation function that includes angular velocity intensity, linear acceleration, and jitter accumulation effect.

[0015] When the jitter intensity exceeds a preset threshold, multiple frames are captured in a burst, and the number of frames in the burst is adaptively determined based on the ratio of jitter intensity to threshold.

[0016] During continuous shooting, the storage compression ratio is adaptively allocated based on the information entropy of each frame in the eye region and the jitter intensity at the time of frame acquisition, resulting in multiple frames of images.

[0017] Preferably, the step of registering and decomposing the multi-frame images to obtain multi-scale sub-band coefficients includes the following steps:

[0018] Using the first frame in a multi-frame image as the reference frame, the optical flow method is used to estimate the translational motion parameters and rotation angles of each frame relative to the reference frame, and the optical flow confidence of each pixel is calculated.

[0019] Multi-level two-dimensional discrete wavelet transform is performed on each frame of the image to obtain the low-frequency subband coefficients and high-frequency subband coefficients at different scales;

[0020] Based on the motion parameters, frequency domain registration is performed on the coefficients of each sub-band to align each frame at the pixel level, resulting in registered multi-scale sub-band coefficients.

[0021] Preferably, the step of fusing the coefficients of each sub-band based on information theory and optical flow confidence, and performing iterative guided filtering enhancement on the fused reconstructed image, includes the following steps:

[0022] For the low-frequency subband in the multi-scale subband coefficients, calculate the local information entropy and local gradient energy of each frame coefficient, construct the fusion weights by combining the optical flow confidence, and perform a weighted average to obtain the low-frequency fusion coefficients.

[0023] For the high-frequency subband in the multi-scale subband coefficients, the local energy of each frame coefficient is calculated, and the weighted energy is obtained by combining the optical flow confidence. The frame with the largest weighted energy is selected as the fusion source, and neighborhood consistency verification is performed to obtain the high-frequency fusion coefficients.

[0024] After optimizing the cross-scale correlation of the low-frequency and high-frequency fusion coefficients, the fused image is reconstructed by inverse wavelet transform.

[0025] The fused image is subjected to iterative guided filtering, and the regularization intensity is adaptively adjusted according to the local texture complexity in each iteration to suppress noise while preserving the edges, thus obtaining the filtered image.

[0026] Preferably, the step of performing modulation transfer function compensation based on pathological priors to generate enhanced images includes the following steps:

[0027] Anterior segment structure segmentation is performed on the filtered image to obtain masks for the conjunctival region, corneal region, and iris region;

[0028] Based on the distribution patterns of pathological features in different frequency bands, frequency band weighting functions are constructed for different regions;

[0029] Based on the frequency band weighting function and the pre-estimated modulation transfer function, a frequency domain compensation function is constructed. The filtered image is then subjected to a Fourier transform, multiplied by the compensation function, and then subjected to an inverse Fourier transform to obtain the enhanced image.

[0030] Preferably, the process of forming the qualified image includes the following steps:

[0031] Calculate the no-reference image quality index of the enhanced image and obtain a weighted overall quality score;

[0032] The expected diagnostic accuracy is estimated based on the comprehensive quality score using a pre-trained quality-diagnosis regression model.

[0033] Calculate the detectability index of pathological features within the ocular region to reflect the significance of pathological features in the image;

[0034] If the overall quality score is lower than the preset threshold, or the pathological feature detectability index is lower than the preset threshold, or the expected diagnostic accuracy is lower than the preset threshold, then the enhanced image quality is unqualified and multiple frames of images are re-acquired; otherwise, the enhanced image is output as the qualified image.

[0035] Preferably, the step of classifying anterior segment diseases and outputting the classification results includes the following steps:

[0036] The qualified images are input into a pre-trained deep learning classification model, which outputs the probability distribution of the preset disease category.

[0037] The category corresponding to the highest probability value in the probability distribution is used as the diagnostic result, and this probability value is used as the confidence level output.

[0038] Preferably, the optimization and update includes the following steps:

[0039] Successfully acquired and verified images and corresponding diagnostic data are collected periodically to form an optimized dataset.

[0040] With the optimization objective of maximizing the average diagnostic accuracy on the dataset, preset parameters are searched, optimized, and updated.

[0041] This invention provides an AI-based learning system for anterior segment disease recognition based on smartphone macro imaging. It offers the following advantages:

[0042] 1. This invention uses real-time handheld shaking detection and adaptive triggering of multi-frame continuous shooting to perform optical flow registration, wavelet decomposition, and frequency domain fusion on the acquired multi-frame images. It optimizes the fusion weights by combining information theory and optical flow confidence, and uses iterative guided filtering and modulation transfer function compensation based on pathological priors to generate high-quality enhanced images. The enhanced images are then subjected to no-reference quality assessment and diagnostic relevance verification to ensure that the input images are clear and the pathological features are significant. This improves the preservation of anterior segment pathological details and the image signal-to-noise ratio, and achieves high-fidelity imaging and accurate identification of minute pathological features such as conjunctival hyperemia and corneal opacity under handheld shooting conditions.

[0043] 2. This invention performs optical flow registration and wavelet decomposition on multiple frames of images, and evaluates the registration reliability by combining optical flow confidence. In the low-frequency sub-band fusion, local information entropy and gradient energy are introduced to construct weights. In the high-frequency sub-band fusion, a local energy maximization strategy is adopted and neighborhood consistency verification is performed. This can maximize the preservation of clear details of each frame of image, eliminate registration errors and isolated noise, and reconstruct a fused image with complete information and sharp edges.

[0044] 3. This invention enhances the fused image through iterative guided filtering, adaptively adjusts the regularization intensity according to the local texture complexity, strengthens noise suppression in flat areas, maintains structural sharpness in edge areas, and combines modulation transfer function compensation based on pathological priors to perform targeted enhancement of different anterior segment regions in frequency bands. This can significantly improve the significance of pathological features such as conjunctival congestion and corneal opacity, providing high-quality input images for subsequent disease identification. Attached Figure Description

[0045] Figure 1 This is an architecture diagram of the AI ​​recognition and learning system for anterior segment diseases based on smartphone macro imaging proposed in this invention.

[0046] Figure 2 This is a flowchart of the AI ​​recognition and learning method for anterior segment diseases based on smartphone macro imaging proposed in an embodiment of the present invention. Detailed Implementation

[0047] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Example 1:

[0049] In a first embodiment of the present invention, the present invention provides an AI recognition and learning system for anterior segment diseases based on smartphone macro imaging, such as... Figure 1 As shown, it includes:

[0050] The sensing and acquisition module is used to acquire inertial measurement data from the smartphone in real time, calculate the intensity of handshake, and trigger multi-frame burst shooting when the intensity of shake exceeds a preset threshold to obtain multiple frames of images.

[0051] Furthermore, the handheld shake intensity is calculated. When the shake intensity exceeds a preset threshold, multi-frame burst shooting is triggered to obtain multiple frames of images, including the following steps:

[0052] It reads gyroscope and accelerometer data from the smartphone at a preset frequency;

[0053] Based on gyroscope and accelerometer data, the current jitter intensity is calculated using an evaluation function that includes angular velocity intensity, linear acceleration, and jitter accumulation effect.

[0054] When the shake intensity exceeds a preset threshold, multiple frames are captured in a burst, and the number of frames in the burst is adaptively determined based on the ratio of the shake intensity to the threshold.

[0055] During continuous shooting, the storage compression ratio is adaptively allocated based on the information entropy of each frame in the eye region and the jitter intensity at the time of frame acquisition, resulting in multiple frames of images.

[0056] Specifically, the sensing and acquisition module first reads the smartphone's gyroscope and accelerometer data at a preset frequency. This preset frequency can be set to 200 Hz to ensure that the main frequency components of hand-held shaking can be captured. The gyroscope data includes angular velocity components about three orthogonal axes, which are the instantaneous angular velocities of the phone's rotation about the roll, pitch, and yaw axes at a given moment. The accelerometer data includes linear acceleration components along three orthogonal axes, which are the instantaneous linear accelerations of the phone along each axis of the phone's coordinate system at a given moment.

[0057] Based on gyroscope and accelerometer data, the current jitter intensity is calculated using an evaluation function that includes angular velocity intensity, linear acceleration, and jitter accumulation effect. Jitter Intensity Evaluation Function Defined as: in, For a moment The magnitude of the angular velocity vector is used to characterize the severity of instantaneous rotational jitter; To deduct gravitational acceleration The magnitude of the linear acceleration vector is used to characterize the severity of the instantaneous translational jitter. The gravitational acceleration constant is 9.8 m / s². From time At the time The integral of the angular velocity modulus over a time period is used to characterize the cumulative effect of jitter, that is, the total energy of jitter over a time period, where The preset integration time window can be set to 0.1 seconds. , , These are weighting coefficients, used to adjust the contribution ratio of instantaneous rotational jitter, instantaneous translational jitter, and jitter accumulation effect to the total jitter intensity, respectively. The sum of the three is usually 1.

[0058] When the calculated jitter intensity Exceeding the preset threshold If significant hand-held shaking is detected, requiring continuous shooting to obtain sufficient image information for subsequent fusion enhancement, then a continuous shooting mode is triggered, and the number of frames in the burst is adaptively determined based on the ratio of shaking intensity to a threshold. (Burst frame count) It can be determined according to the following formula: in, To round down, when the jitter intensity is slightly higher than the threshold, a lower number of consecutive frames, such as 3 to 4, is used; when the jitter intensity is significantly higher than the threshold, a higher number of consecutive frames is used, but the upper limit is 9 frames, in order to avoid the storage and computing pressure caused by an excessively high number of consecutive frames.

[0059] During continuous shooting, to further optimize storage resource allocation, the storage compression ratio is adaptively allocated based on the information entropy of the eye region in each frame and the jitter intensity at the time of frame acquisition. First, eye region detection is performed on the preview image to obtain the eye region bounding box B. For the k-th frame in the continuous shooting sequence, the local information entropy within the eye region B is calculated. Local information entropy reflects the richness of information in an image region; the higher the information entropy, the richer the detailed information contained in that region. Simultaneously, the jitter intensity corresponding to the frame acquisition time is recorded. Adaptive compression ratio It can be determined according to the following formula: in, The base compression ratio can be set to 0.8; This represents the maximum information entropy of all frames in the current burst sequence within the eye region. This is a very small positive number used to avoid division by zero. Frames with higher information entropy, i.e., richer details, receive a higher compression ratio (lower compression level) to retain more detail; frames with lower information entropy, i.e., fewer details, can accept a higher compression level to save storage space. Simultaneously, frames with lower jitter intensity receive a higher compression ratio because these frames are inherently of better quality and deserve to retain more information; frames with higher jitter intensity, even if they retain more information, may be unusable due to motion blur, so the compression level can be appropriately increased. Through the above adaptive compression ratio allocation, a set of differentially compressed multi-frame images is finally obtained for subsequent multi-scale fusion enhancement processing.

[0060] The multi-scale fusion enhancement module is used to register and decompose multiple frames of images to obtain multi-scale sub-band coefficients. Based on information theory and optical flow confidence, the sub-band coefficients are fused, the fused and reconstructed image is enhanced by iterative guided filtering, and the modulation transfer function is compensated based on pathological priors to generate an enhanced image.

[0061] Furthermore, registration and wavelet decomposition are performed on multiple frames of images to obtain multi-scale sub-band coefficients, including the following steps:

[0062] Using the first frame in a multi-frame image as the reference frame, the optical flow method is used to estimate the translational motion parameters and rotation angles of each frame relative to the reference frame, and the optical flow confidence of each pixel is calculated.

[0063] Multi-level two-dimensional discrete wavelet transform is performed on each frame of the image to obtain the low-frequency subband coefficients and high-frequency subband coefficients at different scales;

[0064] Based on the motion parameters, frequency domain registration is performed on the coefficients of each sub-band to align each frame at the pixel level, resulting in the registered multi-scale sub-band coefficients.

[0065] Furthermore, based on information theory and optical flow confidence, the coefficients of each sub-band are fused, and the fused reconstructed image is enhanced by iterative guided filtering, including the following steps:

[0066] For the low-frequency subband in the multi-scale subband coefficients, calculate the local information entropy and local gradient energy of each frame coefficient, construct the fusion weights by combining the optical flow confidence, and perform a weighted average to obtain the low-frequency fusion coefficients.

[0067] For the high-frequency subband in the multi-scale subband coefficients, the local energy of each frame coefficient is calculated, and the weighted energy is obtained by combining the optical flow confidence. The frame with the largest weighted energy is selected as the fusion source, and neighborhood consistency verification is performed to obtain the high-frequency fusion coefficients.

[0068] After optimizing the cross-scale correlation of the low-frequency and high-frequency fusion coefficients, the fused image is reconstructed by inverse wavelet transform.

[0069] The fused image is subjected to iterative guided filtering, and the regularization intensity is adaptively adjusted according to the local texture complexity in each iteration to suppress noise while preserving the edges, thus obtaining the filtered image.

[0070] Furthermore, modulation transfer function compensation is performed based on pathological priors to generate enhanced images, including the following steps:

[0071] Anterior segment structure segmentation is performed on the filtered image to obtain masks for the conjunctival region, corneal region, and iris region;

[0072] Based on the distribution patterns of pathological features in different frequency bands, frequency band weighting functions are constructed for different regions;

[0073] Based on the frequency band weighting function and the pre-estimated modulation transfer function, a frequency domain compensation function is constructed. The filtered image is then subjected to a Fourier transform, multiplied by the compensation function, and then subjected to an inverse Fourier transform to obtain the enhanced image.

[0074] Specifically, the multi-scale fusion enhancement module first registers and decomposes the multi-frame images output by the sensing and acquisition module. First, using the first frame of the multi-frame image as the reference frame, the motion parameters of each frame relative to the reference frame are estimated using optical flow. For the k-th frame image... The Lucas-Kanade optical flow algorithm is used to calculate the optical flow vector of each pixel, and then the translational motion parameters of the entire frame are obtained by least-squares fitting. and rotation angle .in, This represents the displacement along the horizontal direction of the image. This represents the displacement along the vertical direction of the image. The rotation angle of the image around the center.

[0075] Simultaneously, during the optical flow calculation process, the optical flow confidence level of each pixel is also obtained. This confidence level reflects the reliability of the motion estimation at that pixel and is defined as a function of the local texture richness and the matching residual. Calculated by the following formula: in, In pixels local window centered The variance of the grayscale of the internal image is used to measure the richness of the texture in that region. The larger the variance, the richer the texture and the more reliable the optical flow estimation. For the first in this local window The mean square error between the frame image and the reference frame image is used to measure the matching residual; the smaller the residual, the more accurate the registration. This is a balancing coefficient used to adjust the weight of the matching residual in the confidence score; it can be set to 0.1. Optical flow confidence score. The value ranges from 0 to 1, and the larger the value, the more reliable the motion estimation at that pixel.

[0076] Secondly, multi-level two-dimensional discrete wavelet transform is performed on each frame of the image to obtain low-frequency subband coefficients and high-frequency subband coefficients at different scales. The Daubechies-4 wavelet basis is used for J-level decomposition, where J can be set to 3. After the j-th level decomposition, each frame of the image is decomposed into a low-frequency subband. and three high-frequency subbands , , These correspond to high-frequency detail information in the horizontal, vertical, and diagonal directions, respectively. Low-frequency sub-band coefficients reflect the overall image structure, while high-frequency sub-band coefficients contain detailed information such as edges and textures. All sub-band coefficients are denoted as […]. ,in .

[0077] Finally, based on the estimated motion parameters Frequency domain registration is performed on each sub-band coefficient to align frames at the pixel level. For low-frequency sub-band coefficients, bicubic interpolation is used to perform an affine transformation in the spatial domain to align them with the low-frequency sub-bands of the reference frame. For high-frequency sub-band coefficients, phase compensation is performed using the translation property of the Fourier transform; that is, a Fourier transform is performed on each high-frequency sub-band, multiplied by the corresponding phase shift factor in the frequency domain, and then an inverse Fourier transform is performed to obtain the registered high-frequency sub-band coefficients. After the above processing, the registered multi-scale sub-band coefficients are obtained. This provides multi-frame, multi-scale information that is strictly aligned in spatial location for subsequent fusion steps.

[0078] After registration and wavelet decomposition, the multi-scale fusion enhancement module fuses the coefficients of each sub-band based on information theory and optical flow confidence, and performs iterative guided filtering enhancement on the fused reconstructed image. First, for the low-frequency sub-bands in the multi-scale sub-band coefficients, the local information entropy and local gradient energy of each frame coefficient are calculated, and fusion weights are constructed in combination with optical flow confidence. A weighted average is then performed to obtain the low-frequency fusion coefficients. Specifically, for the j-th layer low-frequency sub-band, at pixel... Calculate local information entropy and local gradient energy Local information entropy is based on... The coefficient distribution within a 3×3 window centered on the region is calculated to measure the information richness of that region. The local gradient energy is calculated based on the sum of the squares of the horizontal and vertical gradients of the coefficients within the window, characterizing the edge strength of that region. Fusion weights. Determined by the following formula: in, Total number of frames This represents the optical flow confidence score for the corresponding pixel. This weighting considers information entropy, gradient energy, and registration reliability simultaneously, giving higher weights to information-rich, edge-prominent, and reliably registered regions.

[0079] Low-frequency fusion coefficient The weighted average is obtained as follows: Secondly, for the high-frequency subband in the multi-scale subband coefficients, the local energy of each frame coefficient is calculated, and the weighted energy is obtained by combining it with the optical flow confidence. The frame with the largest weighted energy value is selected as the fusion source, and neighborhood consistency verification is performed to obtain the high-frequency fusion coefficients. For the high-frequency subband... In pixels Calculate local energy This refers to the sum of squared coefficients within a 3×3 window. Weighted energy. for: Select the frame index with the highest weighted energy. The coefficients of this frame are used as the initial fusion result. To further eliminate misselection caused by isolated noise points, neighborhood consistency verification is performed: statistical analysis is conducted using... The mode of the source frame indices selected for each pixel within a 3×3 neighborhood centered on the center is used. If the index of the current pixel is not equal to the mode, it is corrected to the mode index. The high-frequency fusion coefficients are then obtained. .

[0080] Next, cross-scale correlation optimization is performed on the low-frequency and high-frequency fusion coefficients. A feature vector is constructed from the fusion coefficients of the same pixel location at different scales and sub-bands, and principal component analysis is performed on this feature vector for decorrelation. Alternatively, the image is divided into 32×32 image blocks, and the multi-scale coefficients of all pixels within each block form a feature matrix V. Its covariance matrix is ​​calculated, and the eigenvector is obtained. Principal components with a cumulative contribution rate exceeding 95% are selected for transformation, and then inversely transformed back to the original space to obtain the optimized multi-scale coefficients. This step eliminates redundant information between coefficients across scales, making the fusion result more compact.

[0081] Then, the optimized multi-scale coefficients are subjected to inverse wavelet transform to reconstruct the fused image.

[0082] Finally, the fused image is enhanced by iterative guided filtering to further suppress noise while preserving edges. T iterations are performed, where T can be set to 3. The input image for the t-th iteration is... The guiding image is Initially In a local window Within this framework, assume that the output image q(t) and the guiding image satisfy a linear relationship: Solving for the linear coefficients by minimizing the cost function: in, The adaptive regularization parameter is determined by the following formula: here, The basic regularization strength can be set to 0.01; This is the attenuation coefficient, which can be set to 0.5; To guide the image in the window The local variance within the region is used to characterize texture complexity. In flat regions, the local variance is smaller. The variance is relatively large, which enhances the smoothing and noise reduction effect; however, in the edge regions, the local variance is relatively large. Smaller, maintaining edge sharpness. Solving yields... Then, calculate the output value for each pixel. After completing one iteration, update. Then proceed to the next iteration. After T iterations, the filtered image is obtained.

[0083] To further restore high-frequency details lost in the optical system and enhance disease-related features, a multi-scale fusion enhancement module compensates for the modulation transfer function of the filtered image based on pathological priors, generating the final enhanced image. First, the filtered image undergoes anterior segmentation to obtain masks for the conjunctival, corneal, and iris regions. A lightweight semantic segmentation network, such as MobileNetV3-Seg, is then used to classify the filtered image pixel-by-pixel, outputting three binary masks: a conjunctival region mask. Corneal area mask and iris area mask The value 1 indicates that the pixel belongs to the corresponding region, and 0 indicates that it does not belong to the region.

[0084] Secondly, based on the distribution patterns of pathological features across different frequency bands, frequency band weighting functions are constructed for different regions. The conjunctival region's hyperemia is mainly manifested as a vascular network, with its energy concentrated in the low- to mid-frequency bands; the corneal region's opacity features include changes in boundary and internal density, with the boundary corresponding to high frequencies and the internal density corresponding to mid-frequency frequencies; the iris region has a wider distribution of texture features. Therefore, frequency band weighting functions can be pre-established for each region through statistical analysis of clinical images. As an option, for the conjunctival region, the weighting function... Using low-pass filter: in, For spatial frequency, The cutoff frequency can be set to 0.2 cycles / pixel. For the corneal region, the weighting function... It can be designed as a bimodal form, such as the superposition of two Gaussian functions, to simultaneously enhance mid-frequency and high-frequency components. For the iris region, the weighting function... The value can be set to a constant of 1, resulting in full-band equalization enhancement. In practical applications, the corresponding weighting function can be selected based on the preliminary disease classification results. If there is no classification information available, the default general weighting function can be used.

[0085] Then, combining the pre-estimated modulation transfer function, a frequency domain compensation function is constructed. This describes the imaging system's ability to transmit various spatial frequency components, which can be obtained through prior calibration. It is typically assumed to be rotationally symmetric. Frequency domain compensation function Defined as: in, It is a very small positive number, used to prevent division by zero; The appropriate frequency band weighting function is selected based on the region to which the pixel belongs. For pixels that do not belong to any region... Set it to 1.

[0086] Finally, a two-dimensional fast Fourier transform is performed on the filtered image to obtain its frequency domain representation. Multiplying by the compensation function in the frequency domain yields the compensated frequency domain: Again Perform an inverse Fourier transform to obtain the final enhanced image: This enhanced image preserves the original structural information while providing targeted enhancement to the frequency bands where different pathological features are located, and compensates for the attenuation of the optical system, providing higher quality input for subsequent quality verification and disease identification.

[0087] The quality verification module is used to perform no-reference image quality assessment and diagnostic relevance verification on the enhanced image. When the quality indicators meet the preset conditions, the enhanced image is output to form a qualified image.

[0088] Furthermore, the process of creating a qualified image includes the following steps:

[0089] Calculate the no-reference image quality index of the enhanced image and obtain a weighted overall quality score;

[0090] The expected diagnostic accuracy is estimated based on the comprehensive quality score using a pre-trained quality-diagnosis regression model.

[0091] Calculate the detectability index of pathological features within the ocular region to reflect the significance of pathological features in the image;

[0092] If the overall quality score is lower than the preset threshold, or the detectability index of pathological features is lower than the preset threshold, or the expected diagnostic accuracy is lower than the preset threshold, then the enhanced image quality is unqualified and multiple frames of images are re-acquired; otherwise, the enhanced image is output as the qualified image.

[0093] Specifically, the quality verification module is used to perform no-reference image quality assessment and diagnostic relevance verification on the enhanced images generated by the multi-scale fusion enhancement module. When the quality indicators meet the preset conditions, the enhanced image is output as a qualified image for subsequent disease identification.

[0094] The quality verification module first calculates the no-reference image quality index for the enhanced image. It employs three classic no-reference image quality assessment algorithms—BRISQUE, PIQE, and NIQE—to obtain three quality scores, denoted as follows: These three scores have all been linearly transformed to a range of 0 to 100; a higher score indicates better image quality. Overall Quality Score The weighted average is obtained as follows: ,in , , These are weighting coefficients, and the sum of the three is 1. Generally, we can set them as follows: =0.4、 =0.3、 =0.3, making the BRISQUE metric dominant, while also taking into account the evaluation results of the other two metrics.

[0095] The quality verification module estimates the expected diagnostic accuracy based on a pre-trained quality-diagnosis regression model and a comprehensive quality score. This regression model, trained on a large amount of historical image data, establishes a mapping relationship between the comprehensive quality score and the subsequent AI disease identification accuracy. The model uses logistic regression. ,in, The expected diagnostic accuracy is represented by a value between 0 and 1. and The regression model parameters are obtained from the training data through maximum likelihood estimation. This model can predict the probability of obtaining a correct diagnosis if the current image is input into a disease recognition module, based on the image's overall quality score.

[0096] In addition, the quality verification module also calculates the detectability index of pathological features within the eye region to quantify the salience of pathological features in the image. First, it uses the bounding boxes output by the eye region detection module... Identify the region of interest. Then, calculate the enhanced image. The gradient amplitude within this region, combined with the pathological significance map. Perform a weighted average: in, For the image in pixels The gradient magnitude at a given point reflects the edge strength at that point. The pathological saliency map can be obtained through a pre-trained saliency detection network or a simple thresholding method. The larger the value, the higher the probability that the pixel belongs to a pathological feature. This represents the total number of pixels within the eye area. This metric... The larger the value, the more significant the pathological features in the image, and the easier it is for the subsequent disease identification module to classify them correctly.

[0097] Finally, the quality verification module makes a comprehensive judgment based on the above three indicators. Three preset thresholds are set: a comprehensive quality score threshold... Threshold for detectability indicators of pathological features and expected diagnostic accuracy threshold .when or or If the enhanced image is deemed unqualified, the sensing and acquisition module is triggered to re-acquire multiple frames of images; otherwise, the current enhanced image is output as a qualified image to the disease recognition module. This closed-loop quality feedback mechanism ensures that images entering subsequent diagnostic stages all have a high quality level, thereby improving the overall diagnostic reliability of the system.

[0098] The disease identification module is used to input qualified images into a pre-trained deep learning model, classify anterior segment diseases, and output the classification results.

[0099] Further, anterior segment disease classification is performed and the classification results are output, including the following steps:

[0100] The qualified images are input into a pre-trained deep learning classification model, which outputs the probability distribution of the preset disease category.

[0101] The category corresponding to the highest probability value in the probability distribution is used as the diagnostic result, and this probability value is used as the confidence level output.

[0102] Specifically, the disease identification module first adjusts the qualifying images to the input size required by the pre-trained model and performs normalization processing. EfficientNet-B0 is used as the backbone architecture of the classification network, which is trained on a dedicated dataset containing ten types of anterior segment diseases, including conjunctivitis, keratitis, and cataracts.

[0103] After the preprocessed images that meet the criteria are input into the deep learning classification model, the model calculates the output value for each category through forward propagation. The last layer of the model uses the softmax function to convert the output value into a probability distribution. ,in, The last layer of the model corresponds to the first Output values ​​for each category, The total number of categories, For belonging to the first The probability of each category.

[0104] The disease identification module selects the category corresponding to the highest probability value from the probability distribution as the diagnostic result. At the same time, this maximum probability value As a confidence score output, this value reflects the model's level of certainty regarding its predictions. Through the above processing, automated identification from qualified images to disease classification results is achieved.

[0105] The self-learning optimization module is used to collect historical diagnostic data and optimize and update it with the goal of maximizing diagnostic accuracy.

[0106] Further optimization and updates will be performed, including the following steps:

[0107] Successfully acquired and verified images and corresponding diagnostic data are collected periodically to form an optimized dataset.

[0108] With the optimization objective of maximizing the average diagnostic accuracy on the dataset, preset parameters are searched, optimized, and updated.

[0109] Specifically, the self-learning optimization module periodically collects image data successfully acquired by users and verified by doctors, along with corresponding diagnostic results, to form an optimization dataset. This optimization cycle can be set to once a week to ensure sufficient new data while maintaining timely updates.

[0110] The optimization objective is to maximize the average diagnostic accuracy across the dataset, and the key parameters of the system are searched and optimized. The average diagnostic accuracy is defined by the following formula: ,in, To optimize the dataset, This represents the total number of samples in the dataset. For the current parameters, the system's response to the first... The predicted category for each sample; The actual category confirmed by the doctor; This is an indicator function; it takes a value of 1 when the predicted class matches the true class, and 0 otherwise.

[0111] A Bayesian optimization algorithm is employed to search and optimize a predefined parameter set. The parameters to be optimized include the jitter threshold, the weighting coefficients of the jitter evaluation function, the relative importance index of information entropy and gradient energy in low-frequency fusion, the guided filter parameters, the compensation cutoff frequency of the modulation transfer function, and the quality judgment threshold. Bayesian optimization constructs a Gaussian process surrogate model to efficiently search for the optimal parameter combination that maximizes the average diagnostic accuracy within the parameter space.

[0112] After optimization, the self-learning optimization module updates the optimized parameter set to the system, replacing the original parameters, and realizes the adaptive evolution of the system. This allows it to continuously adapt to different user groups and usage environments, and continuously improve diagnostic performance.

[0113] Example 2:

[0114] In a second embodiment of the present invention, the present invention provides an AI recognition and learning method for anterior segment diseases based on smartphone macro imaging, such as... Figure 2 As shown, it includes the following steps:

[0115] The system acquires inertial measurement data from the smartphone in real time, calculates the intensity of hand-held shaking, and triggers multi-frame burst shooting when the shaking intensity exceeds a preset threshold to obtain multiple frames of images.

[0116] Multi-frame images are registered and decomposed using wavelet decomposition to obtain multi-scale sub-band coefficients. The sub-band coefficients are then fused based on information theory and optical flow confidence. The fused and reconstructed image is enhanced by iterative guided filtering, and modulation transfer function compensation is performed based on pathological priors to generate an enhanced image.

[0117] The enhanced image is subjected to no-reference image quality assessment and diagnostic relevance verification. When the quality indicators meet the preset conditions, the enhanced image is output to form a qualified image.

[0118] The qualified images are input into a pre-trained deep learning model to classify anterior segment diseases and output the classification results;

[0119] Collect historical diagnostic data and optimize and update it with the goal of maximizing diagnostic accuracy.

[0120] In the daily medical practice of primary healthcare centers in remote areas, due to the lack of professional ophthalmologists and large-scale examination equipment such as slit lamps, many patients with anterior segment diseases struggle to receive timely and accurate diagnoses. Elderly patients often present with eye discomfort, complaining of blurred vision and increased eye discharge. The general practitioners who see them only have smartphones and an external macro lens, unable to conduct detailed examinations using specialized equipment like those in larger hospitals. To address these issues, this invention employs an AI-based learning method for anterior segment disease recognition based on smartphone macro imaging, the process of which is as follows: Figure 2 As shown. The specific implementation process of this method is as follows:

[0121] First, launch the mobile application, point the macro lens at the patient's eye, and read the data from the phone's gyroscope and accelerometer in real time to calculate the shaking intensity. When the doctor's hand stability is insufficient, it automatically triggers multi-frame burst shooting and adaptively acquires 3 to 9 frames of images according to the degree of shaking. At the same time, it allocates storage compression ratio according to the information entropy and shaking intensity differences of each frame of image.

[0122] Subsequently, optical flow registration was performed using the first frame as a reference. After wavelet decomposition, each frame was aligned in the frequency domain. Low-frequency subbands were fused based on information entropy, gradient energy, and optical flow confidence, while high-frequency subbands were fused based on local energy and confidence. After principal component analysis to eliminate correlation, the fused image was reconstructed. The fused image underwent three iterative guided filtering, with the regularization intensity adaptively adjusted according to the local texture complexity to suppress noise while maintaining edge sharpness. Next, the filtered image underwent anterior segment structure segmentation. A weight function was constructed based on the frequency band distribution of pathological features in the conjunctiva and cornea regions. Compensation was performed in the frequency domain in conjunction with a pre-calibrated modulation transfer function, and the enhanced image was obtained after inverse transformation.

[0123] The BRISQUE, PIQE, and NIQE indices of the enhanced image are calculated and weighted to obtain a comprehensive quality score. The expected accuracy is estimated through a quality-diagnosis regression model. At the same time, the detectability index of pathological features in the ocular region is calculated. When all indices meet the preset thresholds, the image is judged to be up to standard.

[0124] The qualified images are input into the EfficientNet-B0 deep learning model, which outputs the probability distribution of ten types of diseases such as conjunctivitis and keratitis. The highest probability category is used as the diagnosis result and the confidence level is output.

[0125] Finally, the diagnostic data was collected anonymously and used for weekly model parameter optimization to continuously improve diagnostic performance. Through the above process, intelligent screening and diagnosis of anterior segment diseases can be completed using only a mobile phone.

[0126] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An AI-based learning system for anterior segment disease recognition based on smartphone macro imaging, characterized in that, include: The sensing and acquisition module is used to acquire inertial measurement data of the smartphone in real time, calculate the intensity of handshake, and trigger multi-frame burst shooting when the intensity of the shake exceeds a preset threshold to obtain multi-frame images; The multi-scale fusion enhancement module is used to register and decompose the multi-frame images to obtain multi-scale sub-band coefficients, fuse each sub-band coefficient based on information theory and optical flow confidence, perform iterative guided filtering enhancement on the fused reconstructed image, and perform modulation transfer function compensation based on pathological priors to generate an enhanced image. The quality verification module is used to perform no-reference image quality assessment and diagnostic correlation verification on the enhanced image. When the quality indicators meet the preset conditions, the enhanced image is output to form a qualified image. The disease identification module is used to input qualified images into a pre-trained deep learning model, classify anterior segment diseases, and output the classification results. The self-learning optimization module is used to collect historical diagnostic data and optimize and update it with the goal of maximizing diagnostic accuracy.

2. The AI ​​recognition and learning system for anterior segment diseases based on smartphone macro imaging according to claim 1, characterized in that: The calculation of handheld shake intensity, and the triggering of multi-frame burst shooting when the shake intensity exceeds a preset threshold to obtain multi-frame images, includes the following steps: It reads gyroscope and accelerometer data from the smartphone at a preset frequency; Based on the gyroscope and accelerometer data, the current jitter intensity is calculated using an evaluation function that includes angular velocity intensity, linear acceleration, and jitter accumulation effect. When the jitter intensity exceeds a preset threshold, multiple frames are captured in a burst, and the number of frames in the burst is adaptively determined based on the ratio of jitter intensity to threshold. During continuous shooting, the storage compression ratio is adaptively allocated based on the information entropy of each frame in the eye region and the jitter intensity at the time of frame acquisition, resulting in multiple frames of images.

3. The AI ​​recognition and learning system for anterior segment diseases based on smartphone macro imaging according to claim 1, characterized in that: The process of registering and decomposing the multi-frame images to obtain multi-scale sub-band coefficients includes the following steps: Using the first frame in a multi-frame image as the reference frame, the optical flow method is used to estimate the translational motion parameters and rotation angles of each frame relative to the reference frame, and the optical flow confidence of each pixel is calculated. Multi-level two-dimensional discrete wavelet transform is performed on each frame of the image to obtain the low-frequency subband coefficients and high-frequency subband coefficients at different scales; Based on the motion parameters, frequency domain registration is performed on the coefficients of each sub-band to align each frame at the pixel level, resulting in registered multi-scale sub-band coefficients.

4. The AI ​​recognition and learning system for anterior segment diseases based on smartphone macro imaging according to claim 1, characterized in that: The method of fusing the coefficients of each sub-band based on information theory and optical flow confidence, and then performing iterative guided filtering enhancement on the fused and reconstructed image, includes the following steps: For the low-frequency subband in the multi-scale subband coefficients, calculate the local information entropy and local gradient energy of each frame coefficient, construct the fusion weights by combining the optical flow confidence, and perform a weighted average to obtain the low-frequency fusion coefficients. For the high-frequency subband in the multi-scale subband coefficients, the local energy of each frame coefficient is calculated, and the weighted energy is obtained by combining the optical flow confidence. The frame with the largest weighted energy is selected as the fusion source, and neighborhood consistency verification is performed to obtain the high-frequency fusion coefficients. After optimizing the cross-scale correlation of the low-frequency and high-frequency fusion coefficients, the fused image is reconstructed by inverse wavelet transform. The fused image is subjected to iterative guided filtering, and the regularization intensity is adaptively adjusted according to the local texture complexity in each iteration to suppress noise while preserving the edges, thus obtaining the filtered image.

5. The AI ​​recognition and learning system for anterior segment diseases based on smartphone macro imaging according to claim 1, characterized in that: The process of compensating for the modulation transfer function based on pathological priors to generate enhanced images includes the following steps: Anterior segment structure segmentation is performed on the filtered image to obtain masks for the conjunctival region, corneal region, and iris region; Based on the distribution patterns of pathological features in different frequency bands, frequency band weighting functions are constructed for different regions; Based on the frequency band weighting function and the pre-estimated modulation transfer function, a frequency domain compensation function is constructed. The filtered image is then subjected to a Fourier transform, multiplied by the compensation function, and then subjected to an inverse Fourier transform to obtain the enhanced image.

6. The AI ​​recognition and learning system for anterior segment diseases based on smartphone macro imaging according to claim 1, characterized in that: The process of creating a qualified image includes the following steps: Calculate the no-reference image quality index of the enhanced image and obtain a weighted overall quality score; The expected diagnostic accuracy is estimated based on the comprehensive quality score using a pre-trained quality-diagnosis regression model. Calculate the detectability index of pathological features within the ocular region to reflect the significance of pathological features in the image; If the overall quality score is lower than the preset threshold, or the pathological feature detectability index is lower than the preset threshold, or the expected diagnostic accuracy is lower than the preset threshold, then the enhanced image quality is unqualified and multiple frames of images are re-acquired; otherwise, the enhanced image is output as the qualified image.

7. The AI ​​recognition and learning system for anterior segment diseases based on smartphone macro imaging according to claim 1, characterized in that: The process of classifying anterior segment diseases and outputting the classification results includes the following steps: The qualified images are input into a pre-trained deep learning classification model, which outputs the probability distribution of the preset disease category. The category corresponding to the highest probability value in the probability distribution is used as the diagnostic result, and this probability value is used as the confidence level output.

8. The AI ​​recognition and learning system for anterior segment diseases based on smartphone macro imaging according to claim 1, characterized in that: The optimization and update process includes the following steps: Successfully acquired and verified images and corresponding diagnostic data are collected periodically to form an optimized dataset. With the optimization objective of maximizing the average diagnostic accuracy on the dataset, preset parameters are searched, optimized, and updated.

Images