Method and system for constructing eye age difference index based on fundus image and electronic device
By training a dual-channel fusion attention network model and a paired matrix loss function, and combining left and right retinal images, an eye age difference index is constructed, which solves the problem of inaccurate fundus age prediction in existing technologies and achieves more accurate fundus age assessment and healthy aging trajectory judgment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING TIANTAN HOSPITAL AFFILIATED TO CAPITAL MEDICAL UNIV
- Filing Date
- 2023-05-11
- Publication Date
- 2026-06-26
AI Technical Summary
Existing technologies have failed to effectively utilize fundus image data to fuse retinal features of the left and right eyes, and lack the concept of biomarkers for fundus age differences, resulting in inaccurate fundus age prediction and affecting disease risk assessment.
A dual-channel fusion attention network model is adopted. The model is trained by pairing matrix loss function, and the eye age difference index is constructed by combining the left and right eye retinal images. The dual attention module is used to improve the feature extraction efficiency, and the backpropagation and gradient descent algorithms are used to optimize the network.
It improves the accuracy of fundus age prediction, enabling better assessment of the degree to which an individual deviates from the healthy aging trajectory, and provides a reliable basis for judging the healthy aging trajectory.
Smart Images
Figure CN116503373B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a method, system, and electronic device for constructing an eye age difference index based on fundus images. Background Technology
[0002] With the increasing severity of global population aging, age-related diseases are placing a growing burden on society. As humans age, various organs undergo subtle structural changes, leading to a decline in their normal functions. The retina is a crucial window into systemic diseases. Ophthalmologists frequently participate in consultations for systemic conditions such as hypertension and sarcoidosis. This implies that image analysis based on fundus images holds significant potential value in ophthalmology and related fields, providing valuable insights into the pathophysiology of diseases and contributing to higher-quality diagnosis and treatment.
[0003] Artificial intelligence methods can use color fundus photography to build a predictive model of retinal aging, thereby predicting the fundus age of a population. Fundus age can indicate the current stage of retinal aging and even predict the risk of related future diseases. Predictive models built from imaging data of healthy individuals actually describe a normal retinal aging trajectory in older adults. The difference between fundus age and chronological age can reflect the degree to which an individual deviates from the normal aging trajectory, indicating whether retinal aging is premature or delayed. It has been shown that the greater the difference between fundus age and chronological age, the higher the risk of mental or physical problems, the higher the cardiovascular risk, and the greater the likelihood of premature death. Clinically, doctors can use this indicator as a reference to assess the degree of aging in older adults and take appropriate intervention measures.
[0004] In 2018, Google researchers used the Inception-V3 network, a deep learning technology, to extract and quantify a variety of cardiovascular risk factors from retinal images, including age, gender, and systolic blood pressure. These risk factors are core components in many cardiovascular risk scoring systems.
[0005] Previous studies have not modeled the concept of retinal aging, nor have they proposed the concept of fundus age difference as a novel biomarker. Furthermore, existing artificial intelligence analysis methods based on fundus photography data rarely perform simultaneous analysis of left and right eye images of samples, or incorporate algorithm designs that integrate features from both. Summary of the Invention
[0006] To address the aforementioned problems in the existing technology, this invention provides a method, system, and electronic device for constructing an eye age difference index based on fundus images.
[0007] To achieve the above objectives, the present invention provides the following solution:
[0008] A method for constructing an ocular age difference index based on fundus images includes:
[0009] Acquire historical image data; the historical image data includes historical bilateral color retinal fundus images and sample age labels;
[0010] The historical image data is preprocessed, and a training set is constructed based on the preprocessed historical image data;
[0011] Construct a dual-channel fusion attention network model;
[0012] Based on the mean squared error, the dual-channel fusion attention network model is trained using the paired matrix loss function and the training set to obtain a trained dual-channel fusion attention network model, and the trained dual-channel fusion attention network model is used as the fundus age prediction model.
[0013] The fundus images of both eyes of the subject are input into the fundus age prediction model to obtain the fundus age prediction value;
[0014] An eye age difference index is constructed based on the predicted fundus age and the actual age of the subject.
[0015] Optionally, the dual-channel fusion attention network model includes: a branch network and a fully connected layer; the branch network includes: an n×n convolutional kernel, an m-channel pre-convolutional layer, and N convolutional blocks; each convolutional block includes multiple convolutional units, and the number of convolutional units in each convolutional block is different; each convolutional unit includes n convolution-batch normalization-activation groups; a dual attention module is added at the end of each convolutional unit.
[0016] Optionally, n = 3, N = 4, m = 32.
[0017] Optionally, the pairing matrix loss function includes: an age difference loss based on the actual age and an age difference loss based on the predicted fundus age.
[0018] Among them, the age difference loss based on the actual age is L. AD2 :
[0019]
[0020] The age difference loss based on the fundus age prediction value is L. AD3 :
[0021]
[0022] In the formula, NP y represents the number of paired samples (i,j). j Let j be the true age value of sample j. Let y be the predicted fundus age of sample j. i Let i be the true age value of sample i. Let be the predicted fundus age of sample i.
[0023] Optionally, the dual-channel fusion attention network model is trained using backpropagation and gradient descent algorithms, based on the mean squared error, with a pairing matrix loss function and the training set to obtain a trained dual-channel fusion attention network model.
[0024] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:
[0025] The present invention provides a method for constructing an ocular age difference index based on fundus images. By training a dual-channel fusion attention network model using a training set and a paired matrix loss function, the method can reduce prediction errors between samples beyond the mean squared error, thereby further improving the prediction accuracy of the trained dual-channel fusion attention network model. Using this trained network model as a fundus age prediction model to analyze the fundus images of the left and right retina of the subject and fusing the features, a more accurate fundus age can be obtained, facilitating the precise construction of the ocular age difference index and providing a valuable basis for judging the degree of human health aging trajectory.
[0026] Corresponding to the above-described method for constructing an ocular age difference index based on fundus images, this invention also provides the following implementation structure:
[0027] A system for constructing an age difference index based on fundus images, applied to the aforementioned method for constructing an age difference index based on fundus images; the system includes:
[0028] The image data acquisition module is used to acquire historical image data, which includes historical bilateral color retinal fundus images and sample age labels.
[0029] The data preprocessing module is used to preprocess the historical image data and construct a training set based on the preprocessed historical image data.
[0030] The network model building module is used to build a dual-channel fusion attention network model;
[0031] The network model training module is used to train the dual-channel fusion attention network model using the pairing matrix loss function and the training set based on the mean squared error to obtain a trained dual-channel fusion attention network model, and to use the trained dual-channel fusion attention network model as a fundus age prediction model.
[0032] The age prediction module is used to input the bilateral color retinal fundus images of the subject into the fundus age prediction model to obtain the fundus age prediction value;
[0033] The eye age difference index construction module is used to construct an eye age difference index based on the predicted fundus age value and the actual age value of the subject to be tested.
[0034] An electronic device, comprising:
[0035] Memory, which stores computer instructions;
[0036] A processor, connected to the memory, is used to retrieve and execute the computer instructions to implement the above-described method for constructing an ocular age difference index based on fundus images.
[0037] Optionally, the memory is a computer-readable storage medium.
[0038] Since the technical effects achieved by the two implementation structures provided by this invention are the same as those achieved by the above-mentioned method for constructing an eye age difference index based on fundus images, they will not be described again here. Attached Figure Description
[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0040] Figure 1 A flowchart illustrating the method for constructing an ocular age difference index based on fundus images provided by this invention;
[0041] Figure 2 This is a schematic diagram of the structure of the fundus age prediction model provided in an embodiment of the present invention;
[0042] Figure 3 This is a schematic diagram of the branch network structure provided in an embodiment of the present invention;
[0043] Figure 4 This is a schematic diagram of the structure of the dual attention module provided in an embodiment of the present invention;
[0044] Figure 5 This is a flowchart illustrating the implementation of the method for constructing an ocular age difference index based on fundus images, as provided in an embodiment of the present invention. Detailed Implementation
[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0046] The purpose of this invention is to provide a method, system, and electronic device for constructing an eye age difference index based on fundus images, which can accurately construct the eye age difference index and thus provide a favorable basis for judging the degree of human health aging trajectory.
[0047] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0048] Example 1
[0049] This embodiment provides a method for constructing an ocular age difference index based on fundus images, such as... Figure 1 As shown, the method includes:
[0050] Step 100: Obtain historical image data. Historical image data includes historical bilateral color retinal fundus images and sample age labels.
[0051] Step 101: Preprocess the historical image data and construct a training set based on the preprocessed historical image data.
[0052] Step 102: Construct a dual-channel fusion attention network model. For example, Figure 2 As shown, the dual-channel fusion attention network model includes: branch networks and fully connected layers. Figure 3 As shown, the branch network includes: n×n convolutional kernels, m-channel pre-convolutional layers, and N convolutional blocks. Each convolutional block contains multiple convolutional units, and the number of convolutional units in each block is different. Each convolutional unit includes n convolution-batch normalization-activation groups. A dual attention module is added at the end of each convolutional unit, and the specific structure of the dual attention module is shown below. Figure 4 As shown. In practical applications, the values of n, N, and m can be: n = 3, N = 4, m = 32.
[0053] Step 103: Based on the mean squared error, the dual-channel fusion attention network model is trained using the paired matrix loss function and the training set to obtain the trained dual-channel fusion attention network model, which is then used as the fundus age prediction model. In this embodiment, backpropagation and gradient descent algorithms can be used to train the dual-channel fusion attention network model using the paired matrix loss function and the training set based on the mean squared error.
[0054] In this embodiment, the age difference loss is defined as the true age difference y between two samples. i -y j Difference from predicted fundus age The mean square error is expressed mathematically as follows:
[0055]
[0056] Based on this, the pairing matrix loss function used in this embodiment includes: age difference loss based on the real age value and age difference loss based on the fundus age prediction value.
[0057] Among them, the age difference loss based on the actual age is L. AD2 :
[0058]
[0059] The age difference loss based on the fundus age prediction value is L. AD3 :
[0060]
[0061] In the formula, N P y represents the number of paired samples (i,j). j Let j be the true age value of sample j. Let y be the predicted fundus age of sample j. i Let i be the true age value of sample i. Let be the predicted fundus age of sample i.
[0062] Step 104: Input the bilateral color retinal fundus images of the subject into the fundus age prediction model to obtain the fundus age prediction value.
[0063] Step 105: Construct an ocular age difference index based on the predicted fundus age and the actual age of the subject. The ocular age difference index mainly refers to the difference between fundus age and actual age.
[0064] Example 2
[0065] This embodiment illustrates the implementation process of the eye age difference index construction method based on fundus images provided in Embodiment 1 above by way of example. Figure 5 As shown, the details are as follows:
[0066] S1: Construct the original input image
[0067] The original input images consist of a dataset composed of bilateral color retinal fundus images and sample age labels.
[0068] S2: Perform preprocessing operations on the original input image and construct a training set using the preprocessed original input image.
[0069] Preprocessing operations include image centering and image normalization. Centering involves cropping the original input image to a target size, centered on the center, while preserving the entire retinal image. The cropped image is then subjected to Gaussian normalization.
[0070] S3: Constructing a fundus age prediction model: Two-Branch Attention Network (TBAN)
[0071] The branch network structure includes a pre-convolutional layer with a 3×3 convolutional kernel and 32 channels, and four repeated convolutional blocks, each containing a different number of convolutional units. Each convolutional unit contains three convolution-batch normalization-activation groups and residual connections are set. In addition, to optimize feature extraction and improve feature utilization efficiency, this embodiment adds a dual attention module at the end of each convolutional unit in the above network. The number of convolutional units in the four convolutional blocks are 3, 4, 6, and 3, respectively, and the number of feature channels within the convolutional units are 32, 64, 128, and 512, respectively, to infer a sufficiently rich representation of fundus image information.
[0072] S4: Using the image data and real age of healthy individuals, based on the mean squared error, the second-stage refined network of TBAN is trained and learned using the pairing matrix loss function proposed in Example 1 through backpropagation and gradient descent algorithms, and the model with high prediction accuracy and strong generalization performance is selected and saved.
[0073] S5: Use cross-validation to validate the TBAN model and adjust the hyperparameters of the TBAN convolutional neural network model during training.
[0074] S6: Obtain the trained TBAN model and use it as a fundus age prediction model.
[0075] The branch network in the fundus age prediction model uses fixed-size color retinal fundus photography data of the left and right eyes as input. After the data of the left and right eyes are processed by the branch network, the features are cascaded and fed into the fully connected layer. Finally, the fundus age prediction value is calculated. The difference between fundus age and true age can be calculated to obtain the fundus age difference index, which is used to assess the degree to which the current sample deviates from the healthy aging trajectory.
[0076] Based on the above description, the Two-Branch Attention Network (TBAN) proposed in Embodiments 1 and 2 is characterized by two channels, which means that the retinal images of the left and right eyes are each input into a branch convolutional network for data analysis. In the two-channel branch convolutional network, an attention module is introduced to increase the feature extraction efficiency and the model prediction accuracy. Finally, the features extracted by the two branch networks are fused to obtain the assessment of fundus age.
[0077] In Examples 1 and 2, in addition to using mean squared error (MSE) as the loss function, a paired matrix loss function is also employed to train the TBAN network. The basic form of this loss is also known as the "age difference loss," defined as the mean squared error between the true age difference and the predicted age difference between two samples. Various variations can be implemented using different inputs as the baseline to optimize the TBAN network.
[0078] Based on this, the methods provided in Embodiment 1 and Embodiment 2 have the following advantages:
[0079] 1. Using a dual-channel fusion attention network, two branch convolutional neural networks are used to analyze the fundus images of the left and right retinas respectively, and the features are fused to make fundus age prediction more accurate, comprehensive and scientific.
[0080] 2. The concept of the fundus age difference index is proposed. This index uses deep learning algorithms to analyze fundus retinal image information and serves as a novel biomarker to describe an individual's aging status, providing an accurate reference for assessing the degree to which an individual deviates from a healthy aging trajectory.
[0081] 3. A set of loss functions for fundus age prediction is proposed: the paired matrix loss function, which can further improve the prediction accuracy of the model by reducing the prediction error between samples, in addition to the mean squared error.
[0082] 4. The proposed branch convolutional network introduces a dual attention module, which can improve the efficiency of feature utilization.
[0083] Example 3
[0084] This embodiment provides a system for constructing an ocular age difference index based on fundus images, applicable to the ocular age difference index construction method based on fundus images provided in Embodiment 1 or Embodiment 2 above. The system includes:
[0085] The image data acquisition module is used to acquire historical image data. Historical image data includes historical bilateral color retinal fundus images and sample age labels.
[0086] The data preprocessing module is used to preprocess historical image data and construct a training set based on the preprocessed historical image data.
[0087] The network model building module is used to build a dual-channel fusion attention network model.
[0088] The network model training module is used to train the dual-channel fusion attention network model using the paired matrix loss function and training set based on the mean squared error, and then use the trained dual-channel fusion attention network model as the fundus age prediction model.
[0089] The age prediction module is used to input the bilateral color retinal fundus images of the subject into the fundus age prediction model to obtain the fundus age prediction value.
[0090] The eye age difference index construction module is used to construct an eye age difference index based on the predicted fundus age value and the actual age value of the subject.
[0091] Example 4
[0092] This embodiment provides an electronic device, which includes:
[0093] Memory, which stores computer instructions.
[0094] The processor, connected to the memory, is used to retrieve and execute computer instructions to implement the above-described method for constructing an eye age difference index based on fundus images.
[0095] Furthermore, when the computer program in the memory of this embodiment is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.
[0096] Based on the above description, the advantages of the present invention over the prior art are as follows:
[0097] 1. This invention uses deep learning to predict and assess the fundus age of a sample using bilateral retinal fundus image data, and calculates a novel and innovative index called the age difference, thereby assessing the degree to which an individual deviates from the healthy aging trajectory.
[0098] 2. The TBAN method proposed in this invention uses dual-channel fusion to make fundus age prediction more accurate, comprehensive and scientific.
[0099] 3. The branch convolutional network proposed in this invention introduces a dual attention module to improve the efficiency of feature utilization.
[0100] 4. The set of loss functions proposed in this invention can further improve the prediction accuracy of the model by reducing the prediction error between samples, in addition to the mean squared error.
[0101] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.
[0102] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for constructing an ocular age difference index based on fundus images, characterized in that, include: Acquire historical image data; the historical image data includes historical bilateral color retinal fundus images and sample age labels; The historical image data is preprocessed, and a training set is constructed based on the preprocessed historical image data; Construct a dual-channel fusion attention network model; Based on the mean squared error, the dual-channel fusion attention network model is trained using the paired matrix loss function and the training set to obtain a trained dual-channel fusion attention network model, and the trained dual-channel fusion attention network model is used as the fundus age prediction model. The pairing matrix loss function includes: age difference loss based on the actual age and age difference loss based on the predicted age of the fundus. Among them, the age difference loss based on the actual age value is L AD2 : ； The age difference loss based on the fundus age prediction value is L AD3 : ； In the formula, For paired samples The number of For the sample j Real age value For the sample j Predictive value of fundus age, For the sample i Real age value For the sample i Predicted fundus age; The fundus images of both eyes of the subject are input into the fundus age prediction model to obtain the fundus age prediction value; An eye age difference index is constructed based on the predicted fundus age and the actual age of the subject.
2. The method for constructing an ocular age difference index based on fundus images according to claim 1, characterized in that, The dual-channel fusion attention network model includes: a branch network and a fully connected layer; the branch network includes: an n×n convolutional kernel, an m-channel pre-convolutional layer, and N convolutional blocks; each convolutional block includes multiple convolutional units, and the number of convolutional units in each convolutional block is different; each convolutional unit includes n convolution-batch normalization-activation groups; a dual attention module is added at the end of each convolutional unit.
3. The method for constructing an ocular age difference index based on fundus images according to claim 2, characterized in that, n=3, N=4, m=32.
4. The method for constructing an ocular age difference index based on fundus images according to claim 1, characterized in that, The dual-channel fusion attention network model is trained using backpropagation and gradient descent algorithms, based on the mean squared error, with a pairwise matrix loss function and the training set.
5. A system for constructing an ocular age difference index based on fundus images, characterized in that, The system is applied to the method for constructing an ocular age difference index based on fundus images as described in any one of claims 1-4; the system comprises: The image data acquisition module is used to acquire historical image data, which includes historical bilateral color retinal fundus images and sample age labels. The data preprocessing module is used to preprocess the historical image data and construct a training set based on the preprocessed historical image data. The network model building module is used to build a dual-channel fusion attention network model; The network model training module is used to train the dual-channel fusion attention network model using the pairing matrix loss function and the training set based on the mean squared error to obtain a trained dual-channel fusion attention network model, and to use the trained dual-channel fusion attention network model as a fundus age prediction model. The age prediction module is used to input the bilateral color retinal fundus images of the subject into the fundus age prediction model to obtain the fundus age prediction value; The eye age difference index construction module is used to construct an eye age difference index based on the predicted fundus age value and the actual age value of the subject to be tested.
6. An electronic device, characterized in that, include: Memory, which stores computer instructions; A processor, connected to the memory, is used to retrieve and execute the computer instructions to implement the method for constructing an ocular age difference index based on fundus images as described in any one of claims 1-4.
7. The electronic device according to claim 6, characterized in that, The memory is a computer-readable storage medium.