Eye fundus disease diagnosis system and method based on deep learning

Abstract

The invention discloses a fundus disease diagnosis system and method based on deep learning, and relates to the technical field of medical diagnosis, and the fundus disease diagnosis system based on deep learning comprises a light field imaging module which is used for collecting fundus image data; the biological heuristic neural network construction module simulates functions and tissue structures of biological retina cells, constructs a neural network model and is used for preliminary classification of fundus diseases; the micro-fluidic chip transfer learning module is used for simulating propagation and change of disease characteristics through fluid flow by utilizing a micro-fluidic chip technology and a micro-fluidic channel network for simulating fundus disease characteristics so as to realize evaluation of characteristic importance and transfer learning; and a clinical information fusion module. The adaptability of the model to eye fundus image features is enhanced; the generalization ability of the model is improved; various kinds of information can be integrated, and the diagnosis accuracy is comprehensively improved.

Description

Technical Field

[0001] The present invention relates to the field of medical diagnosis technology, and in particular to a fundus disease diagnosis system and method based on deep learning. Background Art

[0002] Fundus diseases are serious diseases that affect human vision and health, including diabetic retinopathy, macular degeneration, retinal detachment and other types. These diseases will gradually damage important fundus structures such as the retina, choroid, and optic nerve, leading to decreased vision, visual field defects, and even blindness in severe cases. Therefore, early and accurate diagnosis is crucial for treatment and prognosis. Timely intervention can effectively prevent the disease from worsening, increase the possibility of vision recovery, and improve the patient's quality of life.

[0003] Traditionally, the diagnosis of fundus diseases relies mainly on the doctor's experience and manual examination, by directly observing the fundus image or using simple measurement tools to judge the health status. However, this method is inefficient when dealing with a large number of patients and may lead to delayed diagnosis.

[0004] With the development of deep learning technology, it has become possible to automatically analyze fundus images using computer vision technology. Deep learning algorithms can mine valuable information from a large amount of fundus image data and provide an objective basis for diagnosis. For example, convolutional neural networks (CNNs) can automatically learn fundus image feature patterns for classification and diagnosis.

[0005] However, existing deep learning-based methods for diagnosing fundus diseases have shortcomings: (1) Insufficient image preprocessing: Fundus image noise removal is not accurate and is easily affected by factors such as electronic noise of the imaging device, slight eye movement, and uneven light. Existing denoising methods may blur important fundus structure information and affect diagnostic accuracy. At the same time, contrast enhancement strategies lack specificity and fail to fully consider the particularity of fundus images, making it difficult to effectively highlight lesion features.

[0006] (2) Limitations of model construction: Deep learning models have limited learning capabilities for fundus disease characteristics and are unable to fully learn complex pathological features, leading to misdiagnosis. For example, the model may fail to capture subtle changes in vascular characteristics of early lesions, leading to misdiagnosis.

[0007] (3) Insufficient integration of clinical information: The diagnosis of fundus diseases requires the combination of clinical information of patients, but existing methods often do not make full use of this information. For example, age factors have an important impact on the characteristics and risk of fundus lesions, but existing models may not take into account such age-related differences, affecting the accuracy of diagnosis. Summary of the invention

[0008] In view of this, the present invention proposes a fundus disease diagnosis system and method based on deep learning to solve the problems of the prior art.

[0009] The present invention proposes a fundus disease diagnosis system based on deep learning, comprising: A light field imaging module, used to collect fundus image data; Bio-inspired neural network building module, simulating the function and organizational structure of biological retinal cells, building a neural network model for the preliminary classification of fundus diseases; The microfluidic chip transfer learning module uses microfluidic chip technology to simulate the microfluidic channel network of fundus disease characteristics, simulates the propagation and change of disease characteristics through fluid flow, and realizes the evaluation and transfer learning of feature importance; The clinical information fusion module integrates the clinical information of fundus images and generates gene coding sequences through gene coding conversion technology based on the mapping relationship between clinical information and gene expression patterns to improve the comprehensiveness of diagnosis.

[0010] This application also proposes a method for diagnosing fundus diseases based on deep learning, including: Step 1, using light field imaging technology to accurately collect and preprocess fundus image data; Step 2: construct a biologically inspired multi-view dynamic neural network model for preliminary classification of fundus diseases; Step 3: Transfer learning and dynamic learning rate adjustment strategy based on microfluidic chip; Step 4: Refine the diagnosis based on the clinical information of gene encoding and GRN-LSTM.

[0011] Specifically, step 1 includes: using a light field camera equipped with a microlens array to take fundus images, and the images collected by the light field camera Represented as a vector about the light direction and the position vector Function , and during the acquisition process, the focus and aperture of the camera are adjusted to ensure clear capture of the fundus structure.

[0012] Specifically, step 1 also includes: using a hierarchical denoising method based on light field depth estimation to estimate the depth of different structures in the fundus image relative to the camera by analyzing the light propagation path and angle differences in the fundus image. Through the function Calculate, where is the light propagation parameter; the fundus image is divided into different depth layers , for each depth layer, the denoising process is expressed as ,in For the The noise standard deviation of the layer, For the The denoising function of the layer, Representative Depth Layer The original fundus image, Representative Depth Layer Fundus image after denoising.

[0013] Specifically, step 1 also includes: constructing an angle-contrast mapping function by analyzing the brightness and contrast relationship formed by light at different angles on the fundus image ,in is the angle, is a nonlinear function, the original contrast is , the enhanced contrast satisfy .

[0014] Specifically, step 1 also includes: performing vector quantization encoding of light field data on the processed fundus image, assuming that the fundus image data is , whose distribution mean is , the standard deviation is , the quantization step size is , the codebook size is ,in , , the encoded quantized data By formula calculate.

[0015] Specifically, step 2 includes: Step 2.1, study the function and organization of biological retinal cells, simulate their information processing mechanism, construct the hierarchical structure and neuron connection relationship of the neural network, and ensure that the neural network can simulate the hierarchical processing characteristics and intercellular connection patterns of the biological retina; Step 2.2, referring to the functions and structural characteristics of cones, rods, bipolar cells and ganglion cells, according to the response characteristics of the above cells to light signals, set the input characteristics of neurons; according to the connection mode between the above cells, set the connection mode of neurons; according to the integration and processing capabilities of the above cells for information, set the output mode of neurons; Step 2.3, according to the organizational hierarchy and cell distribution of the biological retina, a multi-layer neural network of the neural network model is constructed, where each layer of the network corresponds to a functional level in the retina and contains neurons of the corresponding type; Step 2.4, introduce a dynamic synaptic adjustment mechanism driven by multi-view information of fundus images, detect feature changes by comparing the morphology, direction, thickness of fundus blood vessels at different view angles, as well as the texture and color feature changes of different retinal regions; according to the detected feature changes, adaptively adjust the connection weights between neurons to optimize the neural network's ability to extract and process fundus image features; Step 2.5, optimizing the neural network model using a biofeedback mechanism; Step 2.6, use the neural network model to perform preliminary classification of fundus diseases.

[0016] Specifically, in steps 2.1 to 2.3, the input of the neuron is , the output is , the activation function of the neuron is , the output of the neuron is expressed as , Representative The neuron and The connection weights between the inputs, Representative The bias of each neuron is calculated and the hierarchical structure is constructed according to the multi-layer neural network. The number of neurons in the layer is , No. Layer to The connection weight matrix of the layer is , No. The neuron output of the layer is , Representative The output of the layer neurons, Representative The bias of a neuron.

[0017] Specifically, step 3 includes: Step 3.1, using the microchannel structure of the microfluidic chip to simulate the characteristics of different regions such as fundus blood vessels and retina, and construct a microfluidic channel network that simulates the characteristics of fundus diseases; Step 3.2, based on the fluid flow simulation results in the microfluidic chip, evaluate the importance of fundus disease features; use transfer learning strategy to freeze and fine-tune the pre-trained model; Step 3.3, use the change of fluid pressure in the microfluidic chip to dynamically adjust the learning rate.

[0018] Specifically, step 4 includes: Step 4.1, processing the clinical information of the fundus image to convert it into a gene coding sequence using gene coding conversion technology; Step 4.2, input the converted gene coding sequence into GRN-LSTM; Step 4.3, focusing on the clinical information that has an important impact on the diagnosis results based on the synergistic or inhibitory interaction between genes; In step 4.4, the output of GRN-LSTM is used to refine the preliminary classification results.

[0019] The beneficial effects of the present invention are: When light field imaging technology collects fundus images, it ensures image quality in many ways. First, it achieves the acquisition of rich information and clear imaging; second, it uses a layered denoising method to protect structural information; third, it adaptively enhances contrast to highlight key areas; and finally, it improves the efficiency and stability of model training through vector quantization coding.

[0020] When building a biologically inspired neural network, the functions and organizational forms of biological cells are simulated, which not only helps to understand and process image information, but also automatically learns features to reduce errors. In addition, the dynamic synaptic adjustment mechanism improves the adaptability of the model to different fundus conditions, while the optimization of the pooling method and the Dropout layer improves the effectiveness of feature extraction and the generalization ability of the model.

[0021] In the microfluidic chip transfer learning strategy, a simulation channel is first constructed to help understand the development of the disease and the treatment effect. Then, the model's ability to recognize specific disease characteristics is improved through layer freezing and fine-tuning operations. In addition, the learning rate is dynamically adjusted using the changes in fluid pressure in the microfluidic chip, which not only speeds up the convergence of the model, but also avoids missing the optimal solution and reduces dependence on a large amount of labeled data.

[0022] Combining clinical information with gene expression patterns, multiple factors are integrated to improve the comprehensiveness and accuracy of diagnosis.

[0023] The light field imaging module, bio-inspired neural network building module, microfluidic chip transfer learning module and clinical information fusion module work together to form a complete process from image acquisition to result output, thereby improving the accuracy, efficiency and reliability of the entire system in diagnosing fundus diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 It is a structural schematic diagram of the system of the present invention; Figure 2 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0026] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0027] The present invention proposes a method for diagnosing fundus diseases based on deep learning, comprising: step 1, using light field imaging technology to accurately collect and preprocess fundus image data; step 2, constructing a bio-inspired multi-perspective dynamic neural network model for preliminary classification of fundus diseases; step 3, transfer learning and dynamic learning rate adjustment strategy based on microfluidic chips; step 4, clinical information refinement diagnosis based on gene encoding and GRN-LSTM, wherein GRN-LSTM is a long short-term memory network inspired by gene regulatory networks.

[0028] Among them, step 1, using light field imaging technology to accurately collect and pre-process fundus image data, includes: Step 1.1, collect fundus image data. Use a light field camera equipped with a microlens array to shoot the fundus and record the direction and position information of the light. Shoot the fundus from different angles to obtain fundus images containing rich light information. During the acquisition process, adjust the camera focal length and aperture parameters to reduce the patient's eye movement and ensure clear capture of all parts of the fundus structure. Specifically, the image collected by the light field camera Represented as a vector about the light direction and the position vector Function , and during the acquisition process, the focus and aperture of the camera are adjusted to ensure clear capture of the fundus structure.

[0029] Step 1.2: Hierarchical denoising of fundus images. A hierarchical denoising method based on light field depth estimation is used to estimate the depth of different structures in the fundus image relative to the camera by analyzing the light propagation path and angle differences in the fundus image. The fundus image is divided into different depth layers based on the depth information, and different denoising algorithms or parameters are used for the different noise characteristics of each depth layer to avoid destroying the fundus image structure information. Specifically, the depth Through the function Calculate, where is the light propagation parameter; the fundus image is divided into different depth layers , for each depth layer, the denoising process is expressed as ,in For the The noise standard deviation of the layer, For the The denoising function of the layer, Representative Depth Layer The original fundus image, Representative Depth Layer Fundus image after denoising.

[0030] Step 1.3, adaptive contrast enhancement. Adaptively enhance contrast based on light field angle information. By analyzing the relationship between brightness and contrast formed by light at different angles on the fundus image, an angle-contrast mapping function is constructed. Adaptively enhance the contrast of the fundus image based on the mapping function to highlight key structures and lesion features. Specifically, the angle-contrast mapping function ,in is the angle, is a nonlinear function, the original contrast is , the enhanced contrast satisfy .

[0031] Step 1.4, vector quantization encoding of light field data. Perform vector quantization encoding of light field data on the processed fundus image. Determine vector quantization encoding parameters such as quantization step size and codebook size based on the fundus disease diagnosis requirements and the distribution characteristics of fundus image data such as the numerical range and distribution concentration. Ensure that the encoded data can retain enough information to accurately reflect the fundus image characteristics. Specifically, assume that the fundus image data is , whose distribution mean is , the standard deviation is , the quantization step size is , the codebook size is ,in , , the encoded quantized data By formula calculate.

[0032] Step 2: Build a biologically inspired multi-view dynamic neural network model for preliminary classification of fundus diseases, including: Step 2.1, simulate biological retinal cells to build a neural network framework. First, we deeply study the functions and organizational forms of biological retinal cells and simulate their complex information processing mechanisms. Based on these research results, we construct the hierarchical structure and neuronal connection relationship of the neural network to ensure that the neural network can simulate the hierarchical processing characteristics of the biological retina and the connection pattern between cells.

[0033] Step 2.2, design the neuron structure and set the input and output characteristics. In the process of building a neural network, refer to the functions and structural characteristics of cones, rods, bipolar cells and ganglion cells to design the structure of neurons. Specifically, according to the response characteristics of these cells to light signals, set the input characteristics of neurons; according to the connection pattern between cells, set the connection mode of neurons; according to the ability of cells to integrate and process information, set the output mode of neurons.

[0034] Step 2.3, construct a multi-layer neural network to simulate the multi-layer structure of the retina. According to the organizational hierarchy and cell distribution of the biological retina, a multi-layer neural network of the neural network model is constructed. Each layer of the network corresponds to a functional level in the retina and contains neurons of the corresponding type. Through carefully designed connection relationships, the connection between neurons is similar to the connection between biological retinal cells, thereby simulating the multi-layer structure and information processing process of the retina. Specifically, the input of the neuron is , the output is , the activation function of the neuron is , the output of the neuron is expressed as , Representative The neuron and The connection weights between the inputs, Representative The bias of each neuron is calculated and the hierarchical structure is constructed according to the multi-layer neural network. The number of neurons in the layer is , No. Layer to The connection weight matrix of the layer is , No. The neuron output of the layer is , Representative The output of the layer neurons, Representative The bias of a neuron.

[0035] Step 2.4, introduce a dynamic synaptic adjustment mechanism. In order to enhance the adaptive ability of the neural network, a dynamic synaptic adjustment mechanism driven by multi-view information of fundus images is introduced. This mechanism detects feature changes by comparing the morphology, direction, thickness of fundus blood vessels at different viewing angles, as well as the changes in texture and color features of different retinal regions. According to the detected feature changes, the connection weights between neurons are adaptively adjusted to optimize the neural network's ability to extract and process fundus image features. When the features observed at a certain viewing angle change, and this change indicates that there may be lesions in the fundus structure, the connection weights of the relevant neurons are increased. On the contrary, if the features remain stable and normal, the connection weights of these neurons are reduced. Specifically, for fundus images at different viewing angles, , features in fundus images The amount of change at different viewing angles , the original connection weight is , the adjusted connection weight satisfy , where η is the learning rate.

[0036] Step 2.5, use the biofeedback mechanism to optimize the neural network model. During the model training process, the pooling method based on the biofeedback mechanism and the Dropout layer setting are used for optimization. According to the feedback information during the model training process, the appropriate pooling method is selected to improve the effectiveness of feature extraction. When selecting the pooling method, the fundus image is first divided into different regions, and the vascular density and cell distribution characteristics of each region are considered. For areas with high vascular density, the average pooling method tends to be used; for areas with more uniform cell distribution, the maximum pooling method is used. In addition, it is necessary to analyze the changes in the loss function during the model training process and the feedback information of the feature extraction effect to ensure that the selected pooling method can optimize the model performance.

[0037] When setting up the neural network, the optimization of the Dropout layer is also considered to reduce the risk of overfitting of the model and to strike a balance between reducing overfitting and maintaining model performance. During the optimization process of the Dropout layer setting, the Dropout probability is adjusted according to the speed of decrease of the loss function in the early stage of model training and when it tends to stabilize. In addition, the feature complexity of the fundus image is also considered: when the features are complex, the Dropout probability is increased; when the features are simple, the Dropout probability is reduced. These optimization measures together improve the generalization ability and robustness of the neural network.

[0038] Specifically, during the model training process, the loss function is assumed to be , the number of training rounds is , the average pooling method is used in the area with high blood vessel density, where the blood vessel density is set to , the pooling window size is , the output after average pooling ,in For the area, the maximum pooling method is used in the area where the cells are evenly distributed. , and in the Dropout layer setting, when hour, is the threshold, Dropout probability , It is a general symbol for the Dropout probability of the Dropout layer; When the conditions are met , the specific Dropout probability value assigned.

[0039] Step 2.6, use the neural network model to perform preliminary classification of fundus diseases. Use the constructed bio-inspired neural network model to perform preliminary classification of fundus images. The model can automatically learn the complex features in fundus images and classify fundus diseases based on these features. The classification performance and accuracy of the model can be evaluated by comparing and verifying with the actual diagnosis results. Based on the evaluation results, the model can be further adjusted and optimized to improve its performance in the fundus disease classification task.

[0040] Step 3, transfer learning and dynamic learning rate adjustment strategy based on microfluidic chip, including: Step 3.1, microfluidic chip construction and fundus disease characteristics simulation. First, microfluidic chip technology is used to construct a microfluidic channel network that can simulate the characteristics of fundus diseases. This network can simulate the propagation and change process of fundus disease characteristics through the fine design within the microfluidic chip. Specifically, the microchannel structure of the microfluidic chip is used to simulate the characteristics of different areas such as fundus blood vessels and retina, and the dynamic changes of these characteristics are simulated through the flow of fluid in the microchannel.

[0041] Step 3.2, feature importance evaluation and model layer freezing and fine-tuning. Based on the results of fluid flow simulation in the microfluidic chip, the importance of fundus disease features is evaluated. This step is the key to subsequent model optimization because it can help us determine which features are the most critical in the diagnosis of fundus diseases. Next, the pre-trained model was layer frozen and fine-tuned using a transfer learning strategy. Layer freezing means keeping the parameters of the layers in the pre-trained model corresponding to the general feature simulation channels in the microfluidic chip unchanged. These layers usually contain a large amount of general feature information and have a certain reference value for the diagnosis of fundus diseases. Fine-tuning is to adjust the parameters of the layers related to the simulation channels of specific features of fundus diseases to meet the specific needs of fundus disease diagnosis tasks.

[0042] Step 3.3, dynamic learning rate adjustment strategy. During the model training process, the changes in fluid pressure in the microfluidic chip are also used to dynamically adjust the learning rate. Specifically, in the early stages of training, the model's convergence speed is accelerated according to the larger rate of change corresponding to the higher fluid pressure. This is because in the early stages of training, the model needs to quickly learn a large amount of feature information, and a higher learning rate can help the model update parameters faster, thereby accelerating convergence. However, when approaching convergence, the smaller rate of change corresponding to the lower fluid pressure is used to avoid the model missing the optimal solution. This is because when approaching convergence, the model has learned most of the feature information, and at this time, the parameters need to be adjusted more finely to avoid overfitting or underfitting. By dynamically adjusting the learning rate, the present application successfully reduces the reliance on a large amount of labeled fundus disease data and improves the generalization ability of the model. Specifically, assuming that the fluid pressure in the microfluidic chip is , the learning rate is , establish the pressure-learning rate mapping relationship, according to The value is adjusted according to the mapping relationship , in the early stages of training When is a specific pressure value, the learning rate is adjusted according to a larger change rate, and when it is close to convergence, When For another specific pressure value, the learning rate is adjusted according to a smaller change rate.

[0043] Step 4: Refine the diagnosis based on the clinical information of gene coding and GRN-LSTM, including: Step 4.1, gene coding conversion of clinical information. First, the clinical information of the fundus image is processed to convert it into a gene coding sequence using gene coding conversion technology. This conversion process is based on the mapping relationship between clinical information and gene expression patterns, that is, a mapping database is pre-established, which stores the association information between clinical information and corresponding gene expression patterns. By querying this database, clinical information can be accurately converted into corresponding gene coding sequences.

[0044] The clinical information includes the patient's age, gender, medical history, symptoms and family genetic history factors. The medical history includes whether the patient suffers from chronic diseases such as diabetes and hypertension. The symptoms include the degree of vision loss and visual field loss. Based on this mapping relationship, the clinical information is converted into a gene coding sequence using gene coding conversion technology.

[0045] Specifically, the clinical information includes the patient's age A, gender G (male is 1, female is 0), whether he has diabetes D (yes is 1, no is 0), whether he has hypertension E (yes is 1, no is 0), the degree of vision loss V (quantified as a numerical value according to a certain standard), the visual field defect F (quantified as a numerical value according to a certain standard), and the family genetic history Fh (if there is a relevant genetic history, it is 1, otherwise it is 0); the clinical information is represented as a vector , gene expression patterns are represented as vectors , is the gene expression pattern vector The elements, representing a quantified gene expression value in the gene expression pattern. represents the dimension of the gene expression pattern vector, Specifically reflects the The expression levels of related genes are quantified. By calculating the similarity index between the two , is the clinical information vector The elements, Corresponding to the patient's age ,gender , whether you have diabetes , whether you have high blood pressure , Degree of visual impairment , visual field loss and family history A specific value of one of these clinical information. is the gene expression pattern vector The elements, used to calculate the similarity index When the clinical information vector and gene expression pattern vector are weighed, The difference in dimensions is used to comprehensively evaluate the similarity between the two vectors. When the relationship is considered to exist, the gene coding conversion technology is used to convert clinical information into gene coding sequences. .

[0046] Step 4.2, the gene coding sequence is input into the GRN-LSTM network. The converted gene coding sequence is input into the long short-term memory network (GRN-LSTM) inspired by the gene regulatory network. The GRN-LSTM network is a special neural network structure that combines the idea of ​​the gene regulatory network with the characteristics of the long short-term memory network. It can process time series data and capture the long-term dependencies therein. By inputting the gene coding sequence into the network, the learning ability of the network can be used to further analyze and process the clinical information.

[0047] Step 4.3, attention mechanism based on gene interaction. In the GRN-LSTM network, a sequence modeling method based on the attention mechanism of gene interaction is adopted. The core idea of ​​this method is to focus on the part of clinical information that has an important impact on the diagnosis results based on the synergistic or inhibitory interaction relationship between genes. Specifically, by analyzing the interaction relationship between genes, different attention weights are assigned to different genes, so that the network can pay more attention to those clinical information that have an important impact on the diagnosis results. In this way, the accuracy of fundus disease diagnosis can be further improved.

[0048] Step 4.4, refinement of preliminary classification results. The output of the GRN-LSTM network is used to refine the preliminary classification results. In the preliminary classification stage, some possible types of fundus diseases have been obtained. However, these classification results may have certain errors or uncertainties. Therefore, the output of the GRN-LSTM network is used here to further adjust and optimize the preliminary classification results to improve the accuracy of diagnosis.

[0049] This application also proposes a fundus disease diagnosis system based on deep learning, including: The light field imaging module is used to collect fundus image data and obtain high-quality fundus images through light field imaging technology; the light field imaging module includes: a denoising and contrast enhancement submodule, which uses an advanced hierarchical denoising algorithm based on light field depth estimation to accurately remove image noise according to light information, and intelligently adjusts the contrast according to light field angle information to ensure that key structures and pathological features are clearly visible; a vector quantization encoding submodule, which implements vector quantization encoding on the optimized fundus image to ensure that the data distribution meets the training requirements of the deep learning model; A biologically inspired neural network building module simulates the function and organizational structure of biological retinal cells and builds a neural network model for preliminary classification of fundus diseases; the biologically inspired neural network building module includes: a dynamic synaptic adjustment submodule, which introduces multi-view information of fundus images and dynamically adjusts the neuron connection weights to make the network adapt to the feature changes of fundus images; a pooling and Dropout optimization submodule, which draws on the biological feedback mechanism and self-regulation principle to optimize the pooling strategy and Dropout layer configuration, effectively balance overfitting and model performance, and automatically learn complex features to support accurate classification; The microfluidic chip transfer learning module uses microfluidic chip technology to simulate the microfluidic channel network of fundus disease characteristics, simulates the propagation and change of disease characteristics through fluid flow, and realizes the evaluation and transfer learning of feature importance; the microfluidic chip transfer learning module includes: a layer freezing and fine-tuning submodule, which uses layer freezing and fine-tuning technology to keep the parameters of the general feature simulation channel stable and adjust the parameters of the specific feature simulation channel to meet the needs of different diagnostic tasks; a learning rate dynamic adjustment submodule, which uses the change of fluid pressure in the microfluidic chip to dynamically adjust the learning rate according to the different training stages, thereby reducing the dependence on a large amount of labeled data and improving the generalization ability of the model; The clinical information fusion module integrates the clinical information of fundus images, generates gene coding sequences through gene coding conversion technology based on the mapping relationship between clinical information and gene expression patterns; the clinical information fusion module includes: a result refinement submodule, which inputs the gene coding sequence into the long short-term memory network GRN-LSTM inspired by the gene regulatory network, uses the attention mechanism based on gene interaction to perform sequence modeling, focuses on the parts that have a key impact on the diagnosis results, and deeply refines the preliminary classification results.

[0050] The method of the present invention has the following beneficial effects: Optimization of light field imaging technology: The light field camera simultaneously records the direction and position information of the light, and realizes multi-angle shooting, so as to obtain rich fundus images, which is helpful for comprehensive analysis of fundus structure and lesion characteristics. In the process of collecting images, the clarity of the image is ensured by adjusting parameters and reducing eye movement. Then, a hierarchical denoising method based on light field depth estimation is adopted to avoid destroying the structural information of the image, thereby further improving the image quality. At the same time, a method of adaptively enhancing contrast based on light field angle information is also used to make it more convenient to observe the lesion. Finally, the vector quantization encoding technology of light field data improves the efficiency and stability of model training.

[0051] Bio-inspired neural network construction: The function and organizational form of biological retinal cells are simulated to construct a preliminary classification model for fundus diseases. The model can automatically learn complex features, thereby reducing manual errors and improving classification accuracy and efficiency. In addition, the dynamic synaptic adjustment mechanism driven by multi-view information of fundus images improves the adaptability of the model to different fundus conditions. At the same time, the pooling method based on the biofeedback mechanism and the setting optimization of the Dropout layer effectively balance the overfitting problem and improve the generalization ability of the model.

[0052] Microfluidic chip transfer learning strategy: A microfluidic channel network that simulates the characteristics of fundus diseases is constructed in the microfluidic chip, which helps to deeply understand the development of the disease and the treatment effect. Through layer freezing and fine-tuning operations, the model's ability to recognize specific disease characteristics has been improved. In addition, the learning rate is dynamically adjusted by using the changes in fluid pressure in the microfluidic chip, which not only reduces the dependence on a large amount of labeled data, but also improves the accuracy and generalization ability of the model.

[0053] Fusion of clinical information and gene expression patterns: The patient's various clinical information is integrated and mapped to the gene expression patterns to convert clinical information into gene coding sequences, which improves the comprehensiveness and accuracy of the diagnosis. Then, the GRN-LSTM network is used to process complex data, and the attention mechanism based on gene interaction is used to focus on key factors, further refining the preliminary classification results, thereby improving the accuracy of the diagnosis.

[0054] System modules work together: the light field imaging module, bio-inspired neural network building module, microfluidic chip transfer learning module and clinical information fusion module each complete their own specific functions, forming a complete process from image acquisition to result refinement. This collaborative mechanism improves the accuracy, efficiency and reliability of the entire system in diagnosing fundus diseases.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A deep learning-based fundus disease diagnosis system, characterized in that: include: A light field imaging module, used to collect fundus image data; Bio-inspired neural network building module, simulating the function and organizational structure of biological retinal cells, building a neural network model for the preliminary classification of fundus diseases; The microfluidic chip transfer learning module uses microfluidic chip technology to simulate the microfluidic channel network of fundus disease characteristics, simulates the propagation and change of disease characteristics through fluid flow, and realizes the evaluation and transfer learning of feature importance; The clinical information fusion module integrates the clinical information of fundus images and generates gene coding sequences through gene coding conversion technology based on the mapping relationship between clinical information and gene expression patterns to improve the comprehensiveness of diagnosis.

2. A deep learning-based fundus disease diagnosis method, applied to a deep learning-based fundus disease diagnosis system as claimed in claim 1, characterized in that: include: Step 1, using light field imaging technology to accurately collect and preprocess fundus image data; Step 2: construct a biologically inspired multi-view dynamic neural network model for preliminary classification of fundus diseases; Step 3: Transfer learning and dynamic learning rate adjustment strategy based on microfluidic chip; Step 4: Refine the diagnosis based on the clinical information of gene encoding and GRN-LSTM.

3. The deep learning-based fundus disease diagnosis method according to claim 2, characterized in that: The step 1 comprises: using a light field camera equipped with a microlens array to take fundus images, and the images collected by the light field camera Represented as a vector about the light direction and the position vector Function , and during the acquisition process, the focus and aperture of the camera are adjusted to ensure clear capture of the fundus structure.

4. The deep learning-based fundus disease diagnosis method according to claim 3, characterized in that: The step 1 also includes: using a hierarchical denoising method based on light field depth estimation to estimate the depth of different structures in the fundus image relative to the camera by analyzing the light propagation path and angle differences in the fundus image. Through the function Calculate, where is the light propagation parameter; the fundus image is divided into different depth layers , for each depth layer, the denoising process is expressed as ,in For the The noise standard deviation of the layer, For the The denoising function of the layer, Representative Depth Layer The original fundus image, Representative Depth Layer Fundus image after denoising.

5. The deep learning-based fundus disease diagnosis method according to claim 4, characterized in that: The step 1 also includes: constructing an angle-contrast mapping function by analyzing the brightness and contrast relationship formed by light at different angles on the fundus image ,in is the angle, is a nonlinear function, the original contrast is , the enhanced contrast satisfy .

6. The deep learning-based fundus disease diagnosis method according to claim 5, characterized in that: The step 1 further includes: performing vector quantization encoding of light field data on the processed fundus image, assuming that the fundus image data is , whose distribution mean is , the standard deviation is , the quantization step size is , the codebook size is ,in , , the encoded quantized data By formula calculate.

7. The deep learning-based fundus disease diagnosis method according to claim 2, characterized in that: The step 2 comprises: Step 2.1, study the function and organization of biological retinal cells, simulate their information processing mechanism, construct the hierarchical structure and neuron connection relationship of the neural network, and ensure that the neural network can simulate the hierarchical processing characteristics and intercellular connection patterns of the biological retina; Step 2.2, referring to the functions and structural characteristics of cones, rods, bipolar cells and ganglion cells, according to the response characteristics of the above cells to light signals, set the input characteristics of neurons; according to the connection mode between the above cells, set the connection mode of neurons; according to the integration and processing capabilities of the above cells for information, set the output mode of neurons; Step 2.3, according to the organizational hierarchy and cell distribution of the biological retina, a multi-layer neural network of the neural network model is constructed, where each layer of the network corresponds to a functional level in the retina and contains neurons of the corresponding type; Step 2.4, introduce a dynamic synaptic adjustment mechanism driven by multi-view information of fundus images, detect feature changes by comparing the morphology, direction, thickness of fundus blood vessels at different view angles, as well as the texture and color feature changes of different retinal regions; according to the detected feature changes, adaptively adjust the connection weights between neurons to optimize the neural network's ability to extract and process fundus image features; Step 2.5, optimizing the neural network model using a biofeedback mechanism; Step 2.6, use the neural network model to perform preliminary classification of fundus diseases.

8. The deep learning-based fundus disease diagnosis method according to claim 7, characterized in that: In steps 2.1 to 2.3, the input of the neuron is , the output is , the activation function of the neuron is , the output of the neuron is expressed as , Representative The neuron and The connection weights between the inputs, Representative The bias of each neuron is calculated and the hierarchical structure is constructed according to the multi-layer neural network. The number of neurons in the layer is , No. Layer to The connection weight matrix of the layer is , No. The neuron output of the layer is , Representative The output of the layer neurons, Representative The bias of a neuron.

9. The deep learning-based fundus disease diagnosis method according to claim 2, characterized in that: The step 3 comprises: Step 3.1, using the microchannel structure of the microfluidic chip to simulate the characteristics of fundus blood vessels and different retinal areas, and construct a microfluidic channel network that simulates the characteristics of fundus diseases; Step 3.2, based on the fluid flow simulation results in the microfluidic chip, evaluate the importance of fundus disease features; use transfer learning strategy to freeze and fine-tune the pre-trained model; Step 3.3, use the change of fluid pressure in the microfluidic chip to dynamically adjust the learning rate.

10. The deep learning-based fundus disease diagnosis method according to claim 2, characterized in that: The step 4 comprises: Step 4.1, processing the clinical information of the fundus image to convert it into a gene coding sequence using gene coding conversion technology; Step 4.2, input the converted gene coding sequence into GRN-LSTM; Step 4.3, focusing on the clinical information that has an important impact on the diagnosis results based on the synergistic or inhibitory interaction between genes; In step 4.4, the output of GRN-LSTM is used to refine the preliminary classification results.

Images