High myopia eye lesion intelligent detection system
By combining multimodal data acquisition and deep neural network models, the problems of expensive equipment and limited accuracy in the detection of high myopia lesions have been solved, achieving efficient and accurate lesion detection and classification, and reducing detection costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- THE SECOND AFFILIATED HOSPITAL OF CHONGQING MEDICAL UNIV
- Filing Date
- 2026-05-26
- Publication Date
- 2026-07-14
AI Technical Summary
Existing methods for detecting high myopia lesions rely on manual operation, which has problems such as cumbersome examination process, strong subjectivity, expensive equipment, limited accuracy due to single-modal data analysis, inaccurate lesion classification, and high rate of missed detection of mild to moderate lesions.
The system employs a multimodal data acquisition module, a data preprocessing module, a cross-modal feature fusion module, a lesion detection module, and a result output module, combined with a deep neural network model, to achieve multi-dimensional data acquisition, standardized processing, cross-modal feature fusion, and lesion detection, generating a standardized test report.
It improves the accuracy and classification precision of high myopia lesion detection, reduces detection costs, adapts to the differences of different imaging devices, enhances the generalization ability and robustness of the model, and ensures the reliability and stability of detection results.
Smart Images
Figure CN122376012A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent medical detection technology, specifically, it relates to an intelligent detection system for high myopia eye diseases. Background Technology
[0002] With changes in lifestyles and the arrival of an aging society, the incidence of myopia-related retinal diseases is gradually increasing, especially fundus lesions in high myopia. These lesions not only affect vision but can also lead to permanent vision loss, placing a huge health burden on patients. Traditional methods for detecting high myopia rely on the experience and manual operation of ophthalmologists. Although imaging examination methods are constantly developing, problems such as cumbersome examination procedures, strong subjective operation, and expensive equipment still exist.
[0003] With the rapid development of artificial intelligence and deep learning technologies, automated diagnostic systems are gradually emerging in the field of ophthalmology. However, existing AI-based automated detection systems still face challenges such as insufficient data fusion and limited accuracy. Specifically, current technologies mainly analyze single-modal image data, lacking effective integration of cross-modal data. Traditional detection relies on doctors' manual interpretation of images, which is highly subjective, inefficient, and prone to missed diagnoses. Existing AI detection systems mostly use single fundus images and do not fully integrate key clinical indicators such as axial length, refractive power, and quantitative characteristics of fundus structures, resulting in insufficient data dimensions, inaccurate lesion classification, and a high rate of missed diagnoses for mild to moderate lesions.
[0004] In view of this, the present invention is proposed. Summary of the Invention
[0005] To solve the above-mentioned technical problems, the basic concept of the technical solution adopted by the present invention is as follows:
[0006] The intelligent detection system for high myopia-related eye diseases includes a multimodal data acquisition module, a data preprocessing module, a cross-modal feature fusion module, a lesion detection module, a result output module, and a model training module. These modules are electrically connected in sequence to work together to complete the intelligent detection of fundus lesions in high myopia.
[0007] The multimodal data acquisition module is used to acquire multimodal images of the eyes, axial length and refractive power of patients with high myopia, as well as clinical auxiliary data, and to automatically import data by interfacing with existing ophthalmic imaging equipment through a data interface;
[0008] The data preprocessing module is used to standardize and process fundus images, axial length, refractive error, and clinical data in the following steps:
[0009] Fundus image preprocessing, noise reduction: Gaussian filtering to preserve lesion edges and textures; feature point matching to achieve spatial alignment of multimodal fundus images; grayscale normalization to [0,1], and size uniformity to model input specifications;
[0010] Axial length and refractive power pretreatment, The criteria include eliminating unreasonable axial length and refractive error data; using the mean values of people of the same age and refractive error to fill the gaps; and standardizing axial length and refractive error using Z-score to unify the dimensions.
[0011] Clinical data cleaning, including missing value and mean imputation, and outlier removal. Criteria-based elimination ensures data integrity and validity;
[0012] The cross-modal feature fusion module is used to extract features from each modal data after preprocessing, and to achieve deep fusion of cross-modal features through a fusion algorithm to generate a fused feature vector.
[0013] The lesion detection module takes the fused feature vector as input and uses a trained deep neural network model to detect, classify, and assess the severity of fundus lesions in high myopia.
[0014] The result output module is used to visualize the detection results, including generating detection reports, annotating abnormal areas, and storing data; the model training module is used to train and optimize the feature extraction model in the cross-modal feature fusion module and the deep neural network model in the lesion detection module.
[0015] In a preferred embodiment of the present invention, the multimodal images of the eye acquired by the multimodal data acquisition module include optical coherence tomography (OCT) images, fundus color photography images, and enhanced depth OCT images. The clinical auxiliary data include myopia degree, age, disease course, and family history of myopia. The enhanced depth OCT images are acquired using a layered scanning method, with a layer thickness not exceeding 10 μm.
[0016] In a preferred embodiment of the present invention, the standardization processing of the data preprocessing module specifically includes: image denoising: using a Gaussian filtering algorithm to denoise the multimodal image while preserving lesion feature details; the core formula of the Gaussian filtering algorithm is:
[0017] in, For the Gaussian filter kernel in coordinates The value at that location, The standard deviation is Gaussian. This formula generates a smoothing filter kernel through a Gaussian function and calculates a weighted average for each pixel in the image. The weight decreases as the distance between the pixel and the center pixel increases. This effectively filters Gaussian noise in the image and preserves the edge, texture and other details of choroidal lesions to the greatest extent, avoiding the loss of key information about the lesion during the noise reduction process.
[0018] Image alignment: A feature point matching-based image alignment algorithm is used to achieve spatial alignment of images of different modalities of the same patient;
[0019] Normalization processing: The aligned image is normalized in both grayscale and size, reducing the grayscale values to the [0,1] range and adjusting them to a preset size. The core formula for grayscale normalization is:
[0020] in, For the image in coordinates The original grayscale value at that location, This represents the minimum grayscale value of all pixels in the image. The maximum grayscale value of all pixels in the image. The formula maps the original gray value to the [0,1] interval through linear transformation, eliminating the gray value differences caused by different imaging devices and shooting conditions, unifying the image brightness scale, providing standardized input for subsequent feature extraction, and avoiding the gray value differences from affecting the model training accuracy.
[0021] Data cleaning: Missing values were imputed and outliers were removed from clinical auxiliary data. Missing data were imputed using the mean imputation method, and outliers were removed using the 3σ criterion. The core formula of the 3σ criterion is: , , ;in, For the kth clinical auxiliary data (such as myopia degree, axial length). For the total amount of data, The mean of the data. The standard deviation of the data is calculated first; then the mean and standard deviation are calculated. When the absolute value of the difference between a data point and the mean is greater than three times the standard deviation, the data point is considered an outlier and is removed. Mean imputation is used to... ( Calculate the mean of valid data (for the amount of missing data), and use this mean to fill in the missing data to ensure the completeness and validity of clinical auxiliary data and avoid outliers and missing values from interfering with model training.
[0022] In a preferred embodiment of the present invention, the cross-modal feature fusion module includes a feature extraction unit and a feature fusion unit. The feature extraction unit uses an improved convolutional neural network (CNN) to extract visual features from multimodal images, and uses fully connected layers to extract features from clinical auxiliary data and convert them into fixed-dimensional feature vectors. The improved CNN adds an attention mechanism and residual connections to the traditional CNN. The feature fusion unit uses a fusion algorithm combining multi-dimensional dynamic convolution and residual hybrid Transformer, achieving deep fusion of features from each modality through channel attention, window attention, and overlapping cross attention, and adaptively allocating the weights of each modality feature. The core fusion formula is: ,in ,
[0023]
[0024] This is the final cross-modal fusion feature vector. The visual feature vectors extracted from multimodal images using an improved CNN. The feature vector extracted from clinical auxiliary data through a fully connected layer; , These are adaptive weights for image features and clinical auxiliary features, respectively, determined by an attention mechanism. The calculation shows that the attention mechanism dynamically allocates weights by calculating the importance scores of each modality feature—when the lesion feature is more prominent, Enlarging the image emphasizes its features; when clinical data is more critical for lesion assessment... Increase the focus on clinical features to achieve precise fusion of multimodal features and improve the integrity of feature representation.
[0025] In a preferred embodiment of the present invention, the improved CNN uses VGG-19 as the base network and incorporates the channel attention mechanism SE-Net; the fusion algorithm adopts the MDC-RHT architecture; the core formula for SE-Net channel attention is:
[0026] in, This is the result of global average pooling of image features. , This is the weight matrix of the fully connected layer. It is the ReLU activation function. It is the Sigmoid activation function. Here is the channel attention weight vector. This is element-wise multiplication. The image features are optimized by channel attention. This formula extracts channel features through global average pooling, generates attention weights for each channel through a fully connected layer and activation function, assigns high weights to important channels and low weights to irrelevant channels, strengthens lesion features, suppresses redundant information, and improves the targeting of feature extraction.
[0027] In a preferred embodiment of the present invention, the deep neural network model of the lesion detection module adopts the U-Net++ network; lesion identification: outputs lesion identification results; lesion classification: for positive patients, the lesion type is identified, including the dome-shaped macular type, choroidal neovascularization type, and macular hole type, among which the DSM type is further divided into vertical, horizontal and circular types; severity assessment: combining lesion quantitative indicators and clinical auxiliary data, the severity of the lesion is divided into three levels: mild, moderate and severe.
[0028] In a preferred embodiment of the present invention, the standardized test report generated by the result output module includes basic patient information, multimodal image thumbnails, lesion identification results, lesion classification, severity assessment, test reliability, and clinical recommendations; abnormal areas are marked with different colors to distinguish different types of lesions; data is stored in a database, supporting data query, export, and traceability.
[0029] In a preferred embodiment of the present invention, the training optimization process of the model training module includes:
[0030] Dataset construction: Collect multimodal data, clinical auxiliary data and diagnosis results of confirmed patients, and construct training dataset, validation dataset and test dataset;
[0031] Model Training: An end-to-end training approach is adopted to jointly train the feature extraction model and the lesion detection model, using a composite loss function consisting of perceptual loss and structural similarity loss; the core formula of the composite loss function is:
[0032] Where the formula for perceived loss is:
[0033] Structural similarity loss formula:
[0034] The total loss of the model, , These are the loss weights (0.6 and 0.4 respectively in this system). In order to perceive loss, For the feature map predicted by the model, The feature map is the one that is actually labeled. These are the height, width, and number of channels of the feature map, used to measure the difference between the model's predicted features and the true features; For structural similarity loss, These are the means of the predicted feature map and the true feature map, respectively. These are the standard deviation and covariance of the two, respectively. To prevent the use of a constant with a denominator of 0, the structural similarity between the two is measured; the composite loss function takes into account both the loss at the feature level and the loss at the structural level, avoids model overfitting, and improves the model's accuracy and generalization ability in identifying lesion features.
[0035] Model optimization: Model parameters are adjusted using the validation dataset, an adaptive learning rate adjustment algorithm is used to reduce the risk of overfitting, and model performance is evaluated using the test dataset. Training is completed when the detection accuracy, recall, and F1 score reach the preset thresholds. The core formula for the adaptive learning rate is as follows:
[0036]
[0037] in, , These are the first and second moments of the gradient, respectively. , For momentum parameters, Let be the gradient of the t-th training round. , These are the first and second moments after bias correction. Let be the model parameters for round t. These are the model parameters from the previous round. The initial learning rate, To prevent the use of tiny constants with a denominator of 0, this algorithm adaptively adjusts the learning rate of each parameter, increasing the learning rate for parameters with small gradients and decreasing the learning rate for parameters with large gradients. This accelerates model convergence while avoiding training oscillations caused by excessively large learning rates and slow convergence caused by excessively small learning rates, effectively reducing the risk of overfitting and improving model performance.
[0038] Compared with the prior art, the present invention has the following advantages:
[0039] This invention acquires OCT images, fundus color photographs, enhanced depth OCT images, and clinical auxiliary data through a multimodal data acquisition module, achieving comprehensive coverage of multidimensional data. Combined with the deep fusion algorithm of the cross-modal feature fusion module, it fully explores the complementary information of each modality of data, effectively improving the accuracy of lesion detection and classification precision. In particular, it can accurately identify lesion features that are easily missed by a single modality, such as dome-shaped macular degeneration and choroidal neovascularization.
[0040] This invention employs an improved deep learning model and fusion algorithm. Through optimized training of the model training module, it enhances the model's generalization ability and robustness, enabling it to adapt to differences in lesions among different patients and differences in different imaging devices. This reduces equipment requirements, eliminating the need for expensive dedicated detection equipment. Detection can be achieved by interfacing with existing ophthalmic imaging equipment, thus reducing detection costs. Furthermore, the use of a composite loss function and adaptive learning rate adjustment algorithm effectively solves problems such as overfitting and gradient vanishing during model training, further improving the model's detection performance and ensuring the reliability and stability of the detection results.
[0041] The specific embodiments of the present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description
[0042] In the attached diagram:
[0043] Figure 1 This is a framework diagram of an intelligent detection system for high myopia. Detailed Implementation
[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the accompanying drawings. The following embodiments are used to illustrate the present invention.
[0045] Example 1:
[0046] Step S1: Multimodal data acquisition and integration
[0047] In the clinical detection of fundus lesions in high myopia, a multimodal data acquisition module is used to comprehensively acquire multimodal images of the eye and clinical auxiliary data, and to achieve seamless integration with existing ophthalmic imaging equipment. The multimodal images include optical coherence tomography (OCT) images, fundus color photography images, and enhanced depth OCT images. The enhanced depth OCT images are obtained using a layered scanning method, with the slice thickness controlled to no more than 10 μm to ensure the capture of the fine structural features of the choroid. Clinical auxiliary data includes the patient's myopia degree, age, disease duration, and family history of myopia, providing multidimensional data support for lesion detection.
[0048] The multimodal data acquisition module interfaces with existing ophthalmic imaging equipment such as optical coherence tomography (OCT), fundus cameras, and enhanced depth OCT devices via a standardized data interface. This enables automatic data import, eliminating the need for manual entry, effectively reducing human error, and improving the efficiency and accuracy of data acquisition. The acquired multimodal data is transmitted and stored in real-time to a dedicated image processing server, providing a complete and accurate data source for subsequent intelligent detection of fundus lesions in high myopia.
[0049] Step S2: Multimodal data preprocessing and standardization
[0050] To address issues such as noise interference, inconsistent formats, missing or abnormal data in the collected ocular multimodal images and clinical auxiliary data, a data preprocessing module performs standardized processing throughout the entire process. This eliminates various interference factors, unifies the data input format, and provides high-quality standardized data for subsequent feature extraction and fusion, preventing data quality issues from affecting the model's detection accuracy.
[0051] The detection of fundus lesions in high myopia relies on the subtle lesion features of multimodal images and the quantitative indicators of clinical auxiliary data. However, the raw data collected is inevitably affected by factors such as imaging equipment performance, shooting conditions, and manual data entry, resulting in problems such as image noise, spatial misalignment between modal images, grayscale differences, and missing or abnormal clinical data. If the raw data is directly input into the model for analysis, noise will mask key lesion features, image misalignment and grayscale differences will lead to feature extraction bias, and missing / abnormal data will interfere with model training and inference, ultimately causing errors in the detection results.
[0052] Therefore, this invention cleans clinical auxiliary data and achieves data standardization by denoising, aligning, and normalizing multimodal images. Image denoising employs a Gaussian filtering algorithm, effectively filtering Gaussian noise while preserving the edge, texture, and other detailed features of choroidal lesions to the greatest extent possible. Image alignment uses a feature point matching algorithm to achieve precise spatial matching of different modal images of the same patient, ensuring the consistency of lesion region locations across all modal images. Grayscale and size normalization eliminates image differences caused by different imaging devices and shooting conditions, unifying image input standards. Clinical auxiliary data cleaning uses mean filling and the 3σ criterion to fill missing values and remove outliers, ensuring the integrity and validity of clinical data. Through these preprocessing steps, data quality is improved from the source, laying a solid foundation for subsequent cross-modal feature fusion and lesion detection.
[0053] a. Multimodal image preprocessing
[0054] The Gaussian noise, spatial misalignment, and inconsistent grayscale values and sizes present in raw multimodal ocular images directly affect the extraction of lesion features. Noise blurs lesion edges and textures, spatial misalignment between different modalities leads to positional deviations during feature fusion, differences in grayscale values prevent the model from effectively recognizing the same lesion features across modalities, and inconsistent sizes increase the complexity of model training. Therefore, considering the characteristics of multimodal images, denoising, alignment, and normalization are performed in steps to standardize image data and ensure the clarity, consistency, and recognizability of lesion features in the images.
[0055] 1. Image Denoising: Gaussian filtering algorithm is used to denoise optical coherence tomography (OCT) images, fundus color photographs, and enhanced depth OCT images. The core formula is as follows:
[0056] in, For the Gaussian filter kernel in coordinates The value at that location, The standard deviation is Gaussian (the value range of this system is 0.5~2.0, which is adaptively adjusted according to the image noise intensity). This formula generates a smoothing filter kernel through a Gaussian function, and performs a weighted average calculation on each pixel of the image. The weight decreases as the distance between the pixel and the center pixel increases. This can effectively filter Gaussian noise in the image, and preserve the edge, texture and other detailed features of choroidal lesions to the greatest extent, avoiding the loss of key lesion information during the noise reduction process.
[0057] 2. Image Alignment: An image alignment algorithm based on feature point matching is used to spatially align optical coherence tomography (OCT) images, fundus color photography images, and enhanced depth OCT images of the same patient. First, key feature points (such as the fovea of the macula and vascular bifurcation points) are extracted from each modality. The spatial transformation matrix between modalities is calculated through feature point matching. Then, based on the transformation matrix, the images are subjected to rotation, translation, scaling, and other transformations to achieve precise spatial matching between different modalities. This ensures that the position of the same lesion area is consistent across modalities, eliminating spatial biases for subsequent cross-modal feature fusion.
[0058] 3. Normalization processing: Perform grayscale normalization and size normalization processing on the aligned multimodal images in sequence to eliminate the differences in grayscale values and sizes between images.
[0059] The core formula for grayscale normalization is:
[0060] in, For the image in coordinates The original grayscale value at that location, This represents the minimum grayscale value of all pixels in the image. The maximum grayscale value of all pixels in the image. The formula represents the normalized grayscale value. It maps the original grayscale value to the [0,1] interval through linear transformation, eliminating the grayscale value differences caused by different imaging devices and shooting conditions, unifying the image brightness scale, providing standardized input for subsequent feature extraction, and avoiding the impact of grayscale value differences on model training accuracy. Size normalization adjusts all multimodal images to a preset size, ensuring that the image size input to the model is consistent, reducing the complexity of model training and inference, and improving the efficiency and consistency of feature extraction.
[0061] b. Clinical auxiliary data cleaning
[0062] Clinical ancillary data (myopia degree, age, disease duration, family history of myopia) is an important basis for classifying and assessing the severity of fundus lesions in high myopia. However, problems such as missing data and outliers are prone to occur during manual entry or system collection. Missing values will lead to incomplete feature dimensions of the model, and outliers (such as myopia degree or age exceeding the reasonable range) will interfere with the model's feature learning and inference judgment. Therefore, standardized cleaning methods are used to process clinical ancillary data, fill in missing values and remove outliers to ensure the completeness, validity and rationality of clinical data.
[0063] 1. Missing value imputation: The mean imputation method is used to impute missing values in clinical auxiliary data. The core formula is:
[0064] To account for the amount of missing data, the mean of the valid data is calculated and used to fill in the missing data, ensuring the completeness and effectiveness of the clinical auxiliary data and avoiding interference from outliers and missing values in model training;
[0065] 2. Outlier Removal: Missing values were imputed and outliers were removed from the clinical auxiliary data. Missing data were imputed using the mean imputation method, and outliers were removed using the 3σ criterion. The core formula of the 3σ criterion is as follows:
[0066] , ,
[0067] in, For the kth clinical auxiliary data (such as myopia degree, axial length). For the total amount of data, The mean of the data. The standard deviation of the data is determined by first calculating the mean and standard deviation of the data. When the absolute value of the difference between a data point and the mean is greater than 3 times the standard deviation, the data point is determined to be an outlier and is removed.
[0068] c. Preprocessed data integration and verification
[0069] After completing multimodal image preprocessing and clinical auxiliary data cleaning, all standardized data are integrated and associated according to patient unique identifiers to ensure a one-to-one correspondence between multimodal images and clinical auxiliary data for the same patient. Simultaneously, the integrated data undergoes quality verification to check for issues such as blurring and misalignment in images, and for unprocessed missing / outlier values in clinical data. Data that fails verification is reprocessed or removed, ultimately outputting high-quality, standardized multimodal preprocessed data to provide a unified and reliable input for subsequent cross-modal feature fusion.
[0070] Step S3: Deep fusion of cross-modal features
[0071] Based on preprocessed standardized multimodal images and clinical auxiliary data, a cross-modal feature fusion module is used to extract and deeply fuse features, generating a fused feature vector that comprehensively represents the characteristics of fundus lesions in high myopia. This module includes a feature extraction unit and a feature fusion unit, which respectively realize feature extraction of each modality of data and adaptive fusion of cross-modal features, fully exploring the complementary information between multimodal data and improving the completeness and accuracy of feature representation.
[0072] a. Multimodal feature extraction
[0073] There are fundamental differences in the feature types of multimodal images and clinical auxiliary data. Multimodal images are visual features, containing spatial information such as the morphology, location, and texture of choroidal lesions, requiring visual feature extraction models to uncover deeper lesion features. Clinical auxiliary data, on the other hand, are numerical features, reflecting the patient's clinical background information, requiring numerical feature extraction methods to transform them into feature vectors that the model can recognize. Using a single feature extraction method cannot effectively uncover the feature value of different data types. Therefore, differentiated feature extraction strategies are designed for different data types to achieve accurate extraction of multimodal features.
[0074] Multimodal image visual feature extraction: The feature extraction unit uses an improved convolutional neural network (CNN) to extract visual features of multimodal images. This model is based on VGG-19 and adds a channel attention mechanism (SE-Net) and residual connections to the traditional CNN. This not only improves the depth and effectiveness of feature extraction, but also strengthens key lesion features and suppresses redundant information, solving the problems of gradient vanishing and insufficient feature extraction targeting that are common in traditional CNNs.
[0075] The core formula of SE-Net channel attention mechanism is:
[0076] in, This is the result of global average pooling of image features. , This is the weight matrix of the fully connected layer. It is the ReLU activation function. It is the Sigmoid activation function. Here is the channel attention weight vector. This is element-wise multiplication. The image features are optimized by channel attention. This formula extracts channel features through global average pooling, generates attention weights for each channel through a fully connected layer and activation function, assigns high weights to important channels and low weights to irrelevant channels, thereby strengthening lesion features, suppressing redundant information, and improving the targeting of feature extraction. This mechanism extracts channel features through global average pooling, generates attention weights for each channel through a fully connected layer and activation function, assigns high weights to key channels containing choroidal lesions and low weights to irrelevant channels, thereby strengthening lesion features, suppressing redundant information, and improving the targeting and effectiveness of visual feature extraction.
[0077] Numerical feature extraction of clinical auxiliary data employs a fully connected layer to extract features from the cleaned clinical auxiliary data. Numerical data such as myopia degree, age, and disease duration are mapped to fixed-dimensional numerical feature vectors, achieving the feature transformation of clinical auxiliary data and outputting clinical auxiliary feature vectors. This ensures that it can be fused with multimodal image visual feature vectors.
[0078] b. Detailed Logic of Adaptive Cross-Modal Feature Fusion: The roles of multimodal image visual features and clinical auxiliary features differ in the detection of fundus lesions in high myopia—when lesion features are obvious, visual features are the core basis for detection; when clinical data is more critical for lesion classification and severity assessment, the weight of numerical features needs to be increased accordingly. Existing feature fusion methods mostly use fixed-weight splicing or weighting methods, which cannot adaptively allocate the weights of each modal feature according to the actual situation of lesion features, resulting in the fused features failing to accurately represent lesion information.
[0079] Therefore, the feature fusion unit adopts a fusion algorithm with a multi-dimensional dynamic convolution and residual hybrid Transformer (MDC-RHT) architecture. It achieves deep fusion of features of various modalities through channel attention, window attention and overlapping cross attention, and designs an adaptive weight allocation mechanism to dynamically adjust the weights according to the importance of each modal feature, so as to achieve accurate fusion of multimodal features.
[0080] The core fusion formula is in ,
[0081] This is the final cross-modal fusion feature vector. The visual feature vectors extracted from multimodal images using an improved CNN. The feature vector extracted from clinical auxiliary data through a fully connected layer; , These are adaptive weights for image features and clinical auxiliary features, respectively, determined by an attention mechanism. The calculation shows that the attention mechanism dynamically assigns weights based on the importance scores of each modality feature, prioritizing features where lesion characteristics are more prominent. Enlarging the image emphasizes its features; when clinical data is more critical for lesion assessment... Increase the focus on clinical features to achieve precise fusion of multimodal features and improve the integrity of feature representation.
[0082] Step S4: Lesion Detection and Assessment
[0083] The cross-modal fused feature vector is input into the lesion detection module, and a trained deep neural network model is used to detect, classify, and assess the severity of fundus lesions in high myopia. The lesion detection module uses the U-Net++ network as its core model, which has strong semantic segmentation and fine feature recognition capabilities, and can accurately identify lesion areas in the choroid of the eye. Combined with the comprehensiveness of multimodal fused features, it achieves accurate detection and refined assessment of lesions.
[0084] a. Lesion identification: The fused feature vector is input into the U-Net++ network. The model outputs lesion identification results through deep feature learning and reasoning, determines whether the patient has high myopia fundus lesions, realizes binary classification identification of lesions, and outputs detection confidence to quantify the reliability of the identification results.
[0085] b. Lesion Classification: For patients with positive lesion identification results, the model further refines the lesion type, identifying lesion types including dome-shaped macular (DSM) type, choroidal neovascularization type, and macular hole type. The DSM type is further subdivided into vertical, horizontal, and circular types. By mining the feature differences between different lesion types in the fused feature vector and combining the correlation information of clinical auxiliary data, the model achieves refined lesion classification, providing specific lesion type basis for clinical diagnosis and treatment.
[0086] c. Severity Assessment: Combining quantitative indicators of lesions and clinical auxiliary data, the model classifies the severity of fundus lesions in high myopia into three levels: mild, moderate, and severe. Through quantitative analysis of lesion characteristics and comprehensive evaluation of clinical data, an objective classification of lesion severity is achieved, providing a quantitative reference for the development of personalized treatment plans in clinical practice.
[0087] Step S5: Visualization of Detection Results and Data Management
[0088] The results output module visualizes the detection results from the lesion detection module and stores and manages the detection data, enabling intuitive display, convenient query, and traceability of the results. This provides clinicians with clear and comprehensive test reports and offers data support for subsequent model optimization and clinical research.
[0089] Standardized test report generation: Generates standardized test reports that include basic patient information, multimodal image thumbnails, lesion identification results, lesion classification, severity assessment, test reliability, and clinical recommendations. Clinical recommendations are automatically generated based on lesion type and severity, providing targeted reference directions for clinical diagnosis and treatment.
[0090] Abnormal area visualization annotation: Different colors are used to accurately annotate abnormal lesion areas in multimodal images. Different colors correspond to different types of lesions, enabling clinicians to intuitively and quickly identify the location, extent, and type of lesions, thus improving the readability and practicality of reports.
[0091] Data storage and management: All data, including test reports, multimodal raw data, preprocessed data, fused feature vectors, and test results, are stored in a dedicated database. The database supports multi-condition querying, batch export, and full traceability of data, while setting data access permissions to ensure the privacy and security of patient data.
[0092] Step S6: Model Training and Optimization
[0093] The model training module performs end-to-end joint training and optimization of the feature extraction model (improved CNN) in the cross-modal feature fusion module and the deep neural network model (U-Net++) in the lesion detection module, thereby improving the detection accuracy, generalization ability and robustness of the model and ensuring that the model can adapt to the detection needs of different clinical scenarios.
[0094] Dataset Construction: Multimodal images, clinical auxiliary data, and diagnosis results of patients with high myopia diagnosed by ophthalmologists were collected and divided into training dataset, validation dataset, and test dataset according to a preset ratio. All data underwent standardized preprocessing to ensure the quality and validity of the dataset.
[0095] End-to-end joint training: An end-to-end training approach is adopted, inputting preprocessed multimodal data into the model to sequentially complete feature extraction, cross-modal feature fusion, and lesion detection. A composite loss function consisting of perceptual loss and structural similarity loss is used to optimize the model parameters. The core formula is as follows:
[0096] Where the formula for perceived loss is:
[0097] Structural similarity loss formula:
[0098] The total loss of the model, , These are the loss weights (0.6 and 0.4 respectively in this system). In order to perceive loss, For the feature map predicted by the model, The feature map is the one that is actually labeled. These are the height, width, and number of channels of the feature map, used to measure the difference between the model's predicted features and the true features; For structural similarity loss, These are the means of the predicted feature map and the true feature map, respectively. These are the standard deviation and covariance of the two, respectively. To prevent the use of a constant with a denominator of 0, the structural similarity between the two is measured; the composite loss function takes into account both the loss at the feature level and the loss at the structural level, avoids model overfitting, and improves the model's accuracy and generalization ability in identifying lesion features.
[0099] Model optimization: Model parameters are adjusted using the validation dataset, an adaptive learning rate adjustment algorithm is used to reduce the risk of overfitting, and model performance is evaluated using the test dataset. Training is completed when the detection accuracy, recall, and F1 score reach preset thresholds. The core formula for the adaptive learning rate (Adam algorithm) is as follows:
[0100]
[0101] in, , These are the first and second moments of the gradient, respectively. , These are momentum parameters (0.9 and 0.999 for this system). Let be the gradient of the t-th training round. , These are the first and second moments after bias correction. Let be the model parameters for round t. These are the model parameters from the previous round. This is the initial learning rate (0.001 in this system). To prevent the use of tiny constants with a denominator of 0, this algorithm adaptively adjusts the learning rate of each parameter, increasing the learning rate for parameters with small gradients and decreasing the learning rate for parameters with large gradients. This accelerates model convergence while avoiding training oscillations caused by excessively large learning rates and slow convergence caused by excessively small learning rates, effectively reducing the risk of overfitting and improving model performance.
[0102] index This system Traditional single-modal AI Doctor's manual diagnosis lesion detection accuracy 94.26% 82.31% 86.74% Classification accuracy 92.17% 76.53% 81.29% Recall rate of mild to moderate lesions 93.24% 71.66% 78.55% Single case detection time 0.62s 1.85s 120–300s
[0103] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An intelligent detection system for high myopia, characterized in that, It includes a multimodal data acquisition module, a data preprocessing module, a cross-modal feature fusion module, a lesion detection module, a result output module, and a model training module. These modules are electrically connected in sequence to work together to complete the intelligent detection of fundus lesions in high myopia. The multimodal data acquisition module is used to acquire multimodal images of the eyes, axial length and refractive power of patients with high myopia, as well as clinical auxiliary data, and to automatically import data by interfacing with existing ophthalmic imaging equipment through a data interface; The data preprocessing module is used to standardize and process fundus images, axial length, refractive error, and clinical data in the following steps: Fundus image preprocessing, noise reduction: Gaussian filtering to preserve lesion edges and textures; feature point matching to achieve spatial alignment of multimodal fundus images; grayscale normalization to [0,1], and size uniformity to model input specifications; Axial length and refractive power pretreatment, The criteria include eliminating unreasonable axial length and refractive error data; using the mean values of people of the same age and refractive error to fill the gaps; and standardizing axial length and refractive error using Z-score to unify the dimensions. Clinical data cleaning, including missing value and mean imputation, and outlier removal. Criteria-based elimination ensures data integrity and validity; The cross-modal feature fusion module is used to extract features from each modal data after preprocessing, and to achieve deep fusion of cross-modal features through a fusion algorithm to generate a fused feature vector. The lesion detection module takes the fused feature vector as input and uses a trained deep neural network model to detect, classify, and assess the severity of fundus lesions in high myopia. The result output module is used to visualize the detection results, including generating detection reports, annotating abnormal areas, and storing data; the model training module is used to train and optimize the feature extraction model in the cross-modal feature fusion module and the deep neural network model in the lesion detection module.
2. The intelligent detection system for high myopia as described in claim 1, characterized in that, The multimodal data acquisition module acquires ocular multimodal images including optical coherence tomography (OCT) images, fundus color photography images, and enhanced depth OCT images. The clinical auxiliary data includes myopia degree, age, disease duration, and family history of myopia.
3. The intelligent detection system for high myopia as described in claim 1, characterized in that, The enhanced depth OCT image is obtained by layered scanning, with a layer thickness not exceeding 10 μm.
4. The intelligent detection system for high myopia as described in claim 1, characterized in that, The standardization process in the data preprocessing module specifically includes: image denoising: using a Gaussian filtering algorithm to denoise the multimodal image while preserving lesion feature details; the calculation formula is: in, For the Gaussian filter kernel in coordinates The value at that location, The standard deviation is Gaussian. This formula generates a smoothing filter kernel through a Gaussian function and calculates a weighted average for each pixel in the image. The weight decreases as the distance between the pixel and the center pixel increases. This effectively filters Gaussian noise in the image and preserves the edge, texture and other details of choroidal lesions to the greatest extent, avoiding the loss of key information about the lesion during the noise reduction process. Image alignment: A feature point matching-based image alignment algorithm is used to achieve spatial alignment of images of different modalities of the same patient; Normalization processing: The aligned image is normalized in both grayscale and size, reducing the grayscale values to the [0,1] range and adjusting them to a preset size. The core formula for grayscale normalization is: in, For the image in coordinates The original grayscale value at that location, This represents the minimum grayscale value of all pixels in the image. The maximum grayscale value of all pixels in the image. The formula maps the original gray value to the [0,1] interval through linear transformation, eliminating the gray value differences caused by different imaging devices and shooting conditions, unifying the image brightness scale, providing standardized input for subsequent feature extraction, and avoiding the gray value differences from affecting the model training accuracy. Data cleaning: Missing values were imputed and outliers were removed from clinical auxiliary data. Missing data were imputed using the mean imputation method, and outliers were removed using the 3σ criterion. The core formula of the 3σ criterion is: , , ;in, For the k-th clinical auxiliary data, For the total amount of data, The mean of the data. The standard deviation of the data is calculated first; then the mean and standard deviation are calculated. When the absolute value of the difference between a data point and the mean is greater than three times the standard deviation, the data point is considered an outlier and is removed. Mean imputation is used to... , The mean of the valid data is calculated to represent the amount of missing data. This mean is then used to fill in the missing data, ensuring the completeness and validity of the clinical auxiliary data and preventing outliers and missing values from interfering with model training.
5. The intelligent detection system for high myopia as described in claim 1, characterized in that, The cross-modal feature fusion module includes a feature extraction unit and a feature fusion unit. The feature extraction unit uses an improved convolutional neural network (CNN) to extract visual features from multimodal images, and uses fully connected layers to extract features from clinical auxiliary data and convert them into fixed-dimensional feature vectors. The improved CNN adds an attention mechanism and residual connections to the traditional CNN. The feature fusion unit uses a fusion algorithm that combines multi-dimensional dynamic convolution with residual hybrid Transformer, and achieves deep fusion of features from each modality through channel attention, window attention, and overlapping cross attention, and adaptively allocates the weights of each modality feature. The core fusion formula is: ,in , This is the final cross-modal fusion feature vector. The visual feature vectors extracted from multimodal images using an improved CNN. The feature vector extracted from clinical auxiliary data through a fully connected layer; , These are adaptive weights for image features and clinical auxiliary features, respectively, determined by an attention mechanism. The calculation shows that the attention mechanism dynamically allocates weights by calculating the importance scores of each modality feature—when the lesion feature is more prominent, Enlarging the image emphasizes its features; when clinical data is more critical for lesion assessment... Increase the focus on clinical features to achieve precise fusion of multimodal features and improve the integrity of feature representation.
6. The intelligent detection system for high myopia as described in claim 1, characterized in that, The improved CNN uses VGG-19 as the base network and incorporates the channel attention mechanism SE-Net; the fusion algorithm adopts the MDC-RHT architecture; the core formula for SE-Net channel attention is: ; ;in, This is the result of global average pooling of image features. , This is the weight matrix of the fully connected layer. It is the ReLU activation function. It is the Sigmoid activation function. Here is the channel attention weight vector. This is element-wise multiplication. The image features are optimized by channel attention. This formula extracts channel features through global average pooling, generates attention weights for each channel through a fully connected layer and activation function, assigns high weights to important channels and low weights to irrelevant channels, strengthens lesion features, suppresses redundant information, and improves the targeting of feature extraction.
7. The intelligent detection system for high myopia as described in claim 1, characterized in that, The deep neural network model of the lesion detection module adopts the U-Net++ network; lesion identification: outputs lesion identification results; lesion classification: for positive patients, identifies the lesion type, including the dome-shaped macular type, choroidal neovascularization type, and macular hole type, among which the DSM type is further divided into vertical, horizontal and circular types; severity assessment: combining lesion quantitative indicators and clinical auxiliary data, the severity of the lesion is divided into three levels: mild, moderate and severe.
8. The intelligent detection system for high myopia as described in claim 1, characterized in that, The standardized test report generated by the results output module includes basic patient information, multimodal image thumbnails, lesion identification results, lesion classification, severity assessment, test reliability, and clinical recommendations; abnormal areas are marked with different colors to distinguish different types of lesions; Data storage uses a database, which supports data querying, exporting, and tracing.
9. The intelligent detection system for high myopia as described in claim 1, characterized in that, The training optimization process of the model training module includes: Dataset construction: Collect multimodal data, clinical auxiliary data and diagnosis results of confirmed patients, and construct training dataset, validation dataset and test dataset; Model Training: An end-to-end training approach is adopted to jointly train the feature extraction model and the lesion detection model, using a composite loss function consisting of perceptual loss and structural similarity loss; the core formula of the composite loss function is: ; The formula for perceived loss is: The formula for structural similarity loss is: The total loss of the model, , These are the loss weights (0.6 and 0.4 respectively in this system). In order to perceive loss, For the feature map predicted by the model, The feature map is the one that is actually labeled. These are the height, width, and number of channels of the feature map, used to measure the difference between the model's predicted features and the true features; For structural similarity loss, These are the means of the predicted feature map and the true feature map, respectively. These are the standard deviation and covariance of the two, respectively. To prevent the constant with a denominator of 0, it is used to measure the structural similarity between the two; the composite loss function takes into account the loss at both the feature level and the structural level, avoids model overfitting, and improves the model's recognition accuracy and generalization ability for lesion features. Model optimization: Model parameters are adjusted using the validation dataset, an adaptive learning rate adjustment algorithm is used to reduce the risk of overfitting, and model performance is evaluated using the test dataset. Training is completed when the detection accuracy, recall, and F1 score reach preset thresholds. The core formula for the adaptive learning rate (Adam algorithm) is: , , , , ; in, , These are the first and second moments of the gradient, respectively. , For momentum parameters, Let be the gradient of the t-th training round. , These are the first and second moments after bias correction. Let be the model parameters for round t. These are the model parameters from the previous round. The initial learning rate, To prevent the use of tiny constants with a denominator of 0, this algorithm adaptively adjusts the learning rate of each parameter, increasing the learning rate for parameters with small gradients and decreasing the learning rate for parameters with large gradients. This accelerates model convergence while avoiding training oscillations caused by excessively large learning rates and slow convergence caused by excessively small learning rates, effectively reducing the risk of overfitting and improving model performance.