A fully self-service fundus photo analysis method, device, storage medium and system

Abstract

The present invention discloses a fully self-service fundus photograph analysis method, device, storage medium and system, which relates to the technical field of image recognition. The present invention includes the following steps: loading an original fundus image for preprocessing; extracting candidate lesion regions of the image, performing morphological operations, threshold segmentation and mean clustering; constructing a U-net model, training the network model with a fundus image training set to obtain a fundus lesion recognition model; using the trained fundus lesion recognition model to segment a test set to obtain a fundus image segmentation result; implementing cross-platform deployment of GPU / CPU in TensorRT or ONNX format, providing two-stage outputs: a rough segmentation result for doctors to quickly screen, and a refined result for diagnosis. The present invention preprocesses and trains a lesion analysis model through an original fundus image, predicts disease risks according to the model analysis results, provides auxiliary diagnosis and decision support for doctors, and improves the accuracy and efficiency of disease analysis.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and particularly relates to a fully automated fundus photo analysis method, device, storage medium and system, which supports fully automated processing from raw image input to quantitative report generation. Through a standardized preprocessing process, an improved deep learning architecture and refined post-processing, efficient and accurate fundus lesion segmentation is achieved, meeting the needs of clinical diagnosis. Background Art

[0002] The morphological structure of the fundus retina blood vessels (such as diameter, branches, angles, etc.) is an important indicator for measuring human health and can reflect various human diseases, such as diabetic retinopathy (DR), glaucoma, hypertension, etc. Therefore, performing fundus image analysis has important clinical significance for assisting doctors in diagnosing and treating various system diseases of the human body. Currently, there are still the following deficiencies in fundus photo analysis;

[0003] High requirement for doctors' professional skills

[0004] To master the funduscopy technique proficiently and accurately identify various fundus lesion characteristics, doctors need long-term professional training and rich clinical practice experience. The cost of training a qualified ophthalmologist is high and the cycle is long, which to a certain extent limits the number of doctors who can perform high-quality fundus examinations.

[0005] Strong subjectivity

[0006] Different doctors may have differences in the interpretation of fundus photos. Even experienced doctors may have inaccurate diagnoses due to subjective factors when faced with complex fundus lesions. For example, for early diabetic retinopathy, the lesion characteristics are relatively subtle, and different doctors may have different judgments on the degree of the lesion.

[0007] Low efficiency

[0008] Manual interpretation of fundus photos takes a long time. Especially when faced with a large number of patients, doctors have a heavy workload and it is difficult to achieve rapid and batch inspections. In some ophthalmic specialty hospitals, hundreds of patients may be received every day, and it is difficult for doctors to conduct a detailed analysis of all fundus photos in a short time, easily resulting in missed diagnoses or misdiagnoses. Summary of the Invention

[0009] The purpose of the present invention is to provide a fully automated fundus photo analysis method, device, storage medium and system. By preprocessing the original fundus image to train a lesion analysis model, disease risk prediction is performed based on the model analysis results, providing auxiliary diagnosis and decision-making support for doctors, and solving the problems of high requirement for doctors' professional skills, strong subjectivity in interpretation, and low efficiency in the existing situation.

[0010] To solve the above technical problems, the present invention is implemented through the following technical solutions:

[0011] The present invention relates to a fully self-service fundus photo analysis method, including the following steps:

[0012] Step S1: Load the original fundus image for preprocessing;

[0013] Step S2: Extract the candidate lesion regions of the image, and perform morphological operations, threshold segmentation, and mean clustering;

[0014] Step S3: Construct a U-net model, train the network model with the fundus image training set, and obtain a fundus lesion recognition model;

[0015] Step S4: Segment the test set with the trained fundus lesion recognition model to obtain the fundus image segmentation result;

[0016] Step S5: Implement GPU / CPU cross-platform deployment using the TensorRT or ONNX format, and provide two-stage outputs: a rough segmentation result for doctors' rapid screening and a refined result for diagnosis.

[0017] As a preferred technical solution, in step S1, in order to eliminate the phenomena of uneven brightness, noise, and poor blood vessel contrast commonly existing in fundus images, the image needs to be preprocessed so that the designed network model can better learn the blood vessel features. The fundus image preprocessing process is as follows:

[0018] Step S11: Since the retinal blood vessel image usually contains red, blue, and green channel images, and the green channel has a higher contrast, select the green channel image for testing, extract the green channel of the original fundus image to enhance the contrast between blood vessels and lesions, and can better display the blood vessel structure;

[0019] Step S12: Perform zero-mean normalization and unit normalization on the image after the green channel to make it conform to the normal distribution. The specific formula is as follows:

[0020] ;

[0021] In the formula, is the original sample data, is the mean of the original sample data, is the standard deviation of the original sample data, is the labeled image data;

[0022] Step S13: Normalize the image. After normalization, the pixel values of the image are in the range of [0, 1]. For OCT images, perform z-score normalization additionally to eliminate the brightness differences collected by different devices. The specific normalization formula is defined as follows:

[0023] ;

[0024] In the formula, is the pixel value of the input image, and are the minimum pixel value and the maximum pixel value in the image sample data respectively, is the pixel value after normalization processing;

[0025] Step S14: Use histogram equalization to enhance the contrast and highlight the lesion edge features. Divide the image into blocks, generally into 64 small blocks of 8 rows and 8 columns, and perform histogram equalization operations on each small block;

[0026] Step S15: Perform median filtering or Gaussian filtering denoising on the noise interference area to reduce noise and improve the contrast and clarity of the fundus image;

[0027] Step S16: Manually mark the lesions by ophthalmologists and perform data augmentation on the fundus image data; To prevent the phenomenon of overfitting during network training and improve the segmentation effect and training learning ability, perform data augmentation on the images in the database training set before network training. The specific data augmentation process is to rotate at 0°, 45°, 90°, 135°, and 180° respectively, and then perform horizontal, vertical, and mirror flips to artificially augment the training set for neural network training, avoid the problem of insufficient image data flow, and improve the segmentation performance;

[0028] Step S17: Divide the preprocessed image into a training set and a test set according to a certain proportion.

[0029] As a preferred technical solution, in the step S2, the specific process of extracting the candidate lesion area is as follows:

[0030] Step S21: Perform opening operation on the preprocessed image, that is, erode first and then dilate; The erosion operation eliminates isolated small noise points or burrs, and the dilation restores the main body size of the lesion; After the edge is eroded and shrunk and then dilated and expanded, the main structure is retained while removing detail interference;

[0031] Step S22: Perform closing operation on the image after the opening operation, that is, dilate first and then erode; The dilation operation expands the lesion area to fill the internal small holes, and the erosion restores the original boundary; The dilation merges adjacent fractured areas, and the erosion avoids over-expansion. After covering the holes by dilation and then retracting by erosion, the overall shape is maintained;

[0032] Step S23: Generate a binary image using an adaptive threshold segmentation algorithm, and mark the regions with abnormal brightness as candidate lesions;

[0033] Step S24: Introduce spatial neighborhood information constraints, initialize the lesion clustering, and iteratively optimize the membership matrix.

[0034] As a preferred technical solution, in the step S24, the specific steps for initializing the lesion clustering centers are as follows:

[0035] Step S241: Select the region with the highest density as the initial clustering center by calculating the local density of each pixel point in the image;

[0036] Step S242: Introduce a spatial weight factor into the objective function to make the membership degrees of adjacent pixels tend to be consistent;

[0037] Step S243: Dynamically correct the membership matrix after each iteration;

[0038] Step S244: Introduce an adaptive distance weight factor to improve the elastic distance metric.

[0039] As a preferred technical solution, in the step S3, the basic network structure of the U-net model includes an encoder, a decoder, and an output layer; the encoder includes 4 downsampling stages, each stage contains two 3×3 convolutional layers, followed by 2×2 max pooling; each stage of the decoder contains two 3×3 convolutional layers, the number of channels is halved step by step, and each stage uses transposed convolution or bilinear interpolation for upsampling; the output layer is a 1×1 convolutional layer, the number of output channels is the number of classes, and the Sigmoid activation function is used to generate a probability map;

[0040] The specific process of training the network model using the training set is as follows:

[0041] Step S31: The input original fundus image extracts features through the encoder, and the decoder restores the resolution and outputs a probability map;

[0042] Step S32: Compare the predicted mask with the true label and calculate the loss function;

[0043] Step S33: Update the weights through backpropagation and use gradient clipping;

[0044] Step S34: Calculate the Dice coefficient, IoU, and sensitivity on the validation set every 1 epoch;

[0045] Step S35: If the validation loss does not decrease for 10 consecutive epochs, terminate the training.

[0046] As a preferred technical solution, in step S5, multi-dimensional data of patients is collected, including demographic information (age, gender), clinical indicators (blood pressure, blood sugar), imaging data (lesion area, location), and laboratory test results. After data cleaning, standardization processing, and data image enhancement, it is input into the trained lesion recognition model. The lesion recognition model sorts variables significantly related to disease risk through LASSO regression or random forest feature importance, adjusts parameters such as learning rate and tree depth using grid search (Grid Search) or Bayesian optimization (Bayesian Optimization) to balance the risks of overfitting and underfitting, calculates the area under the ROC curve, triggers a re-evaluation of the risk level and an early warning, and generates an individual feature contribution waterfall plot to show the quantitative impact of each variable on the current prediction result.

[0047] The present invention is a fully self-service fundus photograph analysis system, including an image acquisition module, an image preprocessing module, a lesion segmentation and annotation module, a U-net model construction module, a model training module, a disease risk prediction module, and an auxiliary diagnosis and decision support module;

[0048] The image acquisition module is used to collect original fundus images;

[0049] The image preprocessing module is used to sequentially perform channel extraction, size standardization, normalization, histogram equalization, and filtering on the original fundus images;

[0050] The lesion segmentation and annotation module is used to manually annotate lesions by ophthalmology experts;

[0051] The U-net model construction module is used to construct a U-net model; the U-net model includes an encoder, a decoder, and a skip connection module; the encoder constitutes a contraction path, the encoder adopts 4 convolutional blocks, each layer contains 2 3*3 convolutional layers, a total of 8 convolutional layers, and each convolutional block is followed by 1 2*2 max pooling layer, a total of 4 pooling layers; the decoder constitutes an expansion path, the decoder adopts 4 transposed convolutional blocks, each layer contains 1 2*2 transposed convolutional layer, a total of 4 transposed convolutional layers, 2 3*3 convolutional layers, a total of eight convolutional layers; the skip connection module is used to splice the low-level features of the encoder and the high-level features of the decoder;

[0052] The model training module is used to train a lesion recognition model through the constructed U-net model;

[0053] The disease risk prediction module is used to input the fundus images of patients into the trained lesion recognition model, and the lesion recognition model outputs the lesion recognition result;

[0054] The auxiliary diagnosis and decision support module is used to generate an auxiliary diagnosis strategy according to the lesion recognition result.

[0055] The present invention relates to a fully self-service fundus photo analysis device, comprising a memory for storing a computer program; and a processor for running the computer program to implement the steps of the fully self-service fundus photo analysis method as described above.

[0056] The present invention relates to a storage medium storing program instructions, wherein when the program instructions are executed, the steps of the fully self-service fundus photo analysis method as described above are implemented.

[0057] The present invention has the following beneficial effects:

[0058] The present invention preprocesses and trains a lesion analysis model through original fundus images, predicts disease risks based on the model analysis results, provides auxiliary diagnosis and decision support for doctors, improves the accuracy and efficiency of disease analysis, and assists doctors in making diagnoses.

[0059] The present invention extracts candidate lesion regions, and by using a serial combination of erosion and dilation, the opening and closing operations effectively solve the problems of noise interference and internal discontinuity while retaining the main structure of the lesions, providing basic support for medical image analysis.

[0060] The present invention converts the distance information of points in the neighborhood into density estimation through exponential weighting, selects the point with the highest density as the initial clustering center to avoid the instability of random initialization; by suppressing the interference of distant noise points and combining spatial neighborhood constraints to smooth the membership matrix, it can more accurately locate the lesion regions, reduce the number of iterations, and improve the robustness to noise.

[0061] The present invention performs triple optimization on the localization of candidate lesion regions, namely density-guided initialization, spatial constraint modeling, and dynamic parameter adjustment, significantly improving the operation efficiency and noise resistance while ensuring the segmentation accuracy, and is applicable to lesion localization in multi-modal medical images such as CT and OCT.

[0062] Of course, it is not necessary for any product implementing the present invention to achieve all the above advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0064] Figure 1 It is a flowchart of a fully self-service fundus photo analysis method of the present invention;

[0065] Figure 2 It is a schematic diagram of the preprocessing of the original fundus image;

[0066] Figure 3 It is a schematic diagram of the structure of a fully self-service fundus photo analysis system according to the present invention;

[0067] Figure 4 It is the effect diagram of the analysis and processing of the fully self-service fundus photography. Specific implementation manners

[0068] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0069] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0070] In order to make the purpose, technical solution and advantages of the present application clearer, the following is a further detailed description of the present application in conjunction with the attached Figures 1-4 and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0071] Embodiment 1. Please refer to Figure 1 As shown, the present invention is a fully self-service fundus photo analysis method, including the following steps:

[0072] Step S1: Load the original fundus image for preprocessing;

[0073] Step S2: Extract the candidate lesion areas of the image, and perform morphological operations, threshold segmentation and mean clustering;

[0074] Step S3: Construct a U-net model, train the network model with the fundus image training set, and obtain a fundus lesion recognition model;

[0075] Step S4: Segment the test set with the trained fundus lesion recognition model to obtain the fundus image segmentation result;

[0076] Step S5: Use the TensorRT or ONNX format to implement cross-platform deployment of GPU / CPU, and provide two-stage outputs: a rough segmentation result for doctors to quickly screen, and a refined result for diagnosis.

[0077] In step S1, in order to eliminate the phenomena such as uneven brightness, noise, and poor blood vessel contrast commonly existing in fundus images, it is necessary to preprocess the images so that the designed network model can better learn the blood vessel features. The preprocessing process of fundus images is as follows:

[0078] Step S11: Since retinal blood vessel images usually contain red, blue, and green channel images, and the green channel has a higher contrast, select the green channel image for testing. Extract the green channel of the original fundus image to enhance the contrast between blood vessels and lesions, and can better display the blood vessel structure;

[0079] Step S12: Perform zero-mean and unit normalization on the image after the green channel to make it conform to the normal distribution. The specific formula is as follows:

[0080] ;

[0081] In the formula, is the original sample data, is the mean of the original sample data, is the standard deviation of the original sample data, is the image data after annotation;

[0082] Step S13: Normalize the image. The pixel value range of the processed image is between [0, 1]. Perform z-score normalization on the OCT image additionally to eliminate the brightness difference collected by different devices. The specific normalization formula is defined as follows:

[0083] ;

[0084] In the formula, is the pixel value of the input image, and are the minimum pixel value and the maximum pixel value in the image sample data respectively, is the pixel value after normalization processing;

[0085] Step S14: Use histogram equalization to enhance the contrast and highlight the lesion edge features. Divide the image into blocks, generally divided into 64 small blocks of 8 rows and 8 columns, and perform histogram equalization operations on each small block;

[0086] Please refer to Figure 2 as shown. Step S15: Perform median filtering or Gaussian filtering denoising on the noise interference area to reduce noise and improve the contrast and clarity of fundus images;

[0087] Step S16: The ophthalmologist manually marks the lesions and performs fundus image data augmentation; to prevent the network from overfitting during training, improve the segmentation effect and training learning ability, before network training, the images in the database training set are augmented. The specific data augmentation process is to perform rotations of 0°, 45°, 90°, 135°, and 180° respectively, and then perform horizontal, vertical, and mirror flips to artificially augment the training set for neural network training, avoiding the problem of insufficient image data flow and improving the segmentation performance;

[0088] Step S17: Divide the preprocessed images into a training set and a test set according to a ratio. Generally, the ratio of the training set to the test set is 8:2.

[0089] In step S2, the specific process of extracting the candidate lesion area is as follows:

[0090] Step S21: Perform an opening operation on the preprocessed image, that is, erode first and then dilate;

[0091] Step S22: Perform a closing operation on the image after the opening operation, that is, dilate first and then erode;

[0092] Step S23: Use an adaptive threshold segmentation algorithm to generate a binary image, and mark the area with abnormal brightness as a candidate lesion;

[0093] Step S24: Introduce spatial neighborhood information constraints, initialize the lesion clustering, and iteratively optimize the membership matrix; the specific calculation formula is as follows:

[0094] ;

[0095] In the formula, represents the objective function, represents the clustering center data, represents the total number of pixels in the image, represents the th pixel belonging to the th clustering center's membership degree, represents the fuzzy factor, represents the th pixel's feature vector, represents the th clustering center's feature vector, represents the th pixel's spatial neighborhood mean value, represents the weight coefficient of the spatial constraint.

[0096] In step S24, the specific steps of initializing the lesion clustering center are as follows:

[0097] Step S241: By calculating the local density of each pixel in the image, select the area with the highest density as the initial clustering center; the specific formula is:

[0098] ;

[0099] In the formula, represents the local density of the data point , reflecting the density of other points in its surrounding neighborhood; is the neighborhood radius, represents the Euclidean distance score between the data point and the data point ; is used to adjust the decay rate of the exponential function and control the contribution weight of points in the neighborhood;

[0100] This formula converts the distance information of points in the neighborhood into density estimation through exponential weighting: For high-density regions (such as the center of the lesion): the surrounding points are close, value increases significantly; for low-density regions (such as noise or background): the surrounding points are sparse or far away, value is low.

[0101] Step S242: Introduce a spatial weight factor into the objective function to make the membership degrees of adjacent pixels tend to be consistent;

[0102] Step S243: After each iteration, dynamically correct the membership matrix to suppress the influence of isolated noise points on clustering. Truncate and correct abnormal membership values (such as deviating more than 2σ from the mean) to improve the robustness of the algorithm;

[0103] Step S244: To address the problem of uneven gray-scale distribution in the lesion area, introduce an adaptive distance weight factor to improve the elastic distance metric; the specific formula is as follows:

[0104] ;

[0105] In the formula, is the covariance matrix dynamically adjusted according to the contrast of the lesion area.

[0106] In step S3, the basic network structure of the U-net model includes an encoder, a decoder, and an output layer; the encoder includes 4 downsampling stages, each stage contains two 3×3 convolutional layers, followed by 2×2 max pooling; each stage of the decoder contains two 3×3 convolutional layers, the number of channels is halved step by step, and each stage uses transposed convolution or bilinear interpolation for upsampling; the output layer is a 1×1 convolutional layer, the number of output channels is the number of categories, and the Sigmoid activation function is used to generate a probability map; an activation function is used to classify the effective pixels of blood vessels and background noise pixels to achieve binary classification of the output feature image, and zero-padding is used for padding in each convolutional layer;

[0107] The specific process of training the network model with the training set is as follows:

[0108] Step S31: The input original fundus image extracts features through the encoder, and the decoder restores the resolution and outputs a probability map;

[0109] Step S32: Compare the predicted mask with the true label and calculate the loss function; the calculation formula of the loss function is as follows:

[0110] ;

[0111] In the formula, represents the label of sample , 1 for blood vessels and 0 for background and interference, represents the probability that sample is predicted to be a blood vessel;

[0112] Step S33: Update the weights through backpropagation, and use gradient clipping to prevent gradient explosion;

[0113] Step S34: Calculate the Dice coefficient, IoU, and sensitivity on the validation set every 1 epoch;

[0114] Step S35: If the validation loss does not decrease for 10 consecutive epochs, terminate the training.

[0115] The bias parameters and weights in the training network are randomly initialized using a normal distribution. The loss function is optimized using binary cross-entropy. The activation function after each convolutional layer during 20 epochs of training iteration is ReLU, and 0.2 dropout is used between two consecutive convolutional layers. The batchsize for model training is set to 4, and the learning rate is set to 0.001.

[0116] In step S5, multi-dimensional data of patients are collected, including demographic information (age, gender), clinical indicators (blood pressure, blood sugar), imaging data (lesion area, location), and laboratory test results. After data cleaning, standardization, and data image enhancement, they are input into the trained lesion recognition model. Among them, for data cleaning and standardization, missing values are filled using multiple imputation methods, and outliers are detected and corrected through box plots or the 3σ principle; continuous variables are normalized (Min-Max or Z-score), and categorical variables are one-hot encoded; for medical image enhancement, for imaging data (such as CT, MRI), enhancement techniques such as histogram equalization, random rotation / flip, etc. are used to improve the robustness of the model to lesion morphology.

[0117] The lesion recognition model ranks variables significantly related to disease risk (p-value < 0.05 or feature weight > threshold) through LASSO regression or random forest feature importance ranking, generates combined features (such as BMI × blood sugar level) based on medical prior knowledge to capture the synergistic effects of multiple factors; uses grid search or Bayesian optimization to adjust parameters such as learning rate and tree depth to balance the risks of overfitting and underfitting. The area under the ROC curve (AUC ≥ 0.75 is the clinical availability standard), with a focus on high sensitivity (> 85%) to reduce missed diagnoses; as well as the Brier score (< 0.1) and calibration curve to evaluate the consistency between predicted probability and actual risk. The generalization of the model is verified in an external dataset (such as data from different hospitals or regions), with the requirement that the decrease in AUC < 5%; trigger a re-evaluation of the risk level and early warning, generate an individual feature contribution waterfall chart to show the quantitative impact of each variable on the current prediction result, and convert the model output into a clinically operable scoring tool. Doctors can quickly estimate the risk probability by superimposing the feature scores.

[0118] The recognition results are divided into:

[0119] Low-risk group (probability < 0.3): Annual regular follow-up;

[0120] Medium-risk group (0.3 - 0.6): Quarterly special examinations (such as tumor marker detection);

[0121] High-risk group (> 0.6): Initiate a multi-disciplinary consultation (MDT) and develop an intervention plan;

[0122] And a dynamic update mechanism is set: Re-enter the latest patient indicators every 3 - 6 months to trigger a re-evaluation of the risk level and early warning (such as automatically pushing an alarm when the risk jump > 20%).

[0123] Example 2, refer to Figures 3-4As shown in the figure, the present invention is a fully self-service fundus photograph analysis system, which can be used to execute the method content of Embodiment 1 of the present invention, including: an image acquisition module, an image preprocessing module, a lesion segmentation and annotation module, a U-net model construction module, a model training module, a disease risk prediction module, and an auxiliary diagnosis and decision support module, characterized in that:

[0124] The image acquisition module is used to acquire the original fundus image;

[0125] The image preprocessing module is used to sequentially perform channel extraction, size normalization, normalization, histogram equalization, and filtering on the original fundus image;

[0126] The lesion segmentation and annotation module is used to manually annotate lesions by ophthalmology experts;

[0127] The U-net model construction module is used to construct a U-net model; the U-net model includes an encoder, a decoder, and a skip connection module; the encoder constitutes a contraction path, the encoder uses 4 convolutional blocks, each layer contains 2 3*3 convolutional layers, and the ReLU activation function is used, with a total of 8 convolutional layers, and each convolutional block is followed by 1 2*2 max pooling layer, with a total of 4 pooling layers for downsampling; after each level of downsampling, the spatial size of the feature map is halved; for example, it decreases from 512*512 to 256*256, and the number of channels doubles, such as doubling from 64 to 128;

[0128] The specific process is as follows:

[0129] First level: input image (such as 572×572×3) → convolution (3×3, 64 channels) → convolution (3×3, 64 channels) → max pooling (output 284×284×64);

[0130] Second to fourth levels: repeat the above operations, and the feature map is gradually reduced to 32×32×1024, and the number of channels increases to 1024;

[0131] The decoder constitutes an expansion path, the decoder uses 4 transposed convolutional blocks, each layer contains 1 2*2 transposed convolutional layer for upsampling, with a total of 4 transposed convolutional layers and 2 3*3 convolutional layers, for a total of eight convolutional layers, which are used to splice the feature maps of the same layer of the encoder, and the feature fusion is optimized through 2 3*3 convolutional layers;

[0132] The specific process is as follows:

[0133] First level: input 32×32×1024 → transposed convolution (output 64×64×512) → splice the fourth-level feature of the encoder (64×64×512) → convolution (3×3, 512 channels) → convolution (3×3, 512 channels);

[0134] Subsequent stage: Repeat the upsampling, concatenation, and convolution operations, and finally restore to the original resolution (such as 568×568×64).

[0135] The skip connection module is used to concatenate the low-level features (such as edge details) of the encoder with the high-level features of the decoder to enhance the boundary segmentation accuracy.

[0136] The model training module is used to train the lesion recognition model through the constructed U-net model.

[0137] The disease risk prediction module is used to input the fundus image of the patient into the trained lesion recognition model, and the lesion recognition model outputs the lesion recognition result.

[0138] The auxiliary diagnosis and decision support module is used to generate an auxiliary diagnosis strategy according to the lesion recognition result.

[0139] It should be noted that in the above system embodiments, the included units are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0140] In addition, those of ordinary skill in the art can understand that all or part of the steps in the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the corresponding program can be stored in a computer-readable storage medium.

[0141] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and utilize the present invention well. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. A fully self-service fundus photo analysis method, characterized in that, It includes the following steps: Step S1: Load the original fundus image for preprocessing; Step S2: Extract the candidate lesion regions of the image, and perform morphological operations, threshold segmentation, and mean clustering; Step S3: Construct a U-net model, train the network model with the fundus image training set to obtain a fundus lesion recognition model; Step S4: The trained fundus lesion recognition model segments the test set to obtain the fundus image segmentation result; Step S5: Implement cross-platform deployment on GPU / CPU using the TensorRT or ONNX format, providing two-stage outputs: a rough segmentation result for doctors' quick screening and a refined result for diagnosis; Among them, in step S2, the specific process of candidate lesion region extraction is as follows: Step S21: Perform an opening operation on the preprocessed image; Step S22: Perform a closing operation on the image after the opening operation; Step S23: Use an adaptive threshold segmentation algorithm to generate a binary image, and mark the regions with abnormal brightness as candidate lesions; Step S24: Introduce spatial neighborhood information constraints, initialize the lesion clustering, and iteratively optimize the membership matrix; the specific calculation formula is as follows: ； In the formula, represents the objective function, represents the cluster center data, represents the total number of pixels in the image, represents the -th pixel belonging to the -th cluster center membership degree, represents the fuzzy factor, represents the -th pixel feature vector, represents the -th cluster center feature vector, represents the -th pixel spatial neighborhood mean, represents the weight coefficient of the spatial constraint; In step S24, the specific steps for initializing the lesion clustering center are as follows: Step S241: By calculating the local density of each pixel point in the image, select the region with the highest density as the initial clustering center; the specific formula is: ； In the formula, represents the local density of the data point , reflecting the density of other points in its surrounding neighborhood; is the neighborhood radius, represents the Euclidean distance score between the data point and the data point ; is used to adjust the decay rate of the exponential function and control the contribution weight of points within the neighborhood; Step S242: Introduce a spatial weight factor into the objective function to make the membership degrees of adjacent pixels tend to be consistent; Step S243: After each iteration, dynamically correct the membership matrix, and truncate and correct the abnormal membership values; Step S244: Introduce an adaptive distance weight factor to improve the elastic distance metric; the specific formula is as follows: ； In the formula, is the covariance matrix dynamically adjusted according to the contrast of the lesion area.

2. The fully automated fundus photo analysis method according to claim 1, characterized in that, In step S1, the fundus image preprocessing process is as follows: Step S11: Extract the green channel of the original fundus image to enhance the contrast between blood vessels and lesions; Step S12: Perform zero-mean normalization and unit normalization on the image after the green channel to make it conform to the normal distribution; Step S13: Normalize the image to eliminate the brightness differences collected by different devices; Step S14: Use histogram equalization to enhance the contrast and highlight the lesion edge features; Step S15: Perform median filtering or Gaussian filtering to reduce noise on the noise interference regions; Step S16: Manually label the lesions by ophthalmology experts and perform data augmentation on the fundus images; Step S17: Divide the preprocessed image into a training set and a test set according to a certain proportion.

3. A fully self-service fundus photo analysis method according to claim 1, characterized in that In step S3, the basic network structure of the U-net model includes an encoder, a decoder, and an output layer; the encoder includes 4 downsampling stages, each stage contains two 3×3 convolutional layers, followed by 2×2 max pooling; each stage of the decoder contains two 3×3 convolutional layers, the number of channels is halved step by step, and each stage uses transposed convolution or bilinear interpolation for upsampling; the output layer is a 1×1 convolutional layer, the number of output channels is the number of categories, and the Sigmoid activation function is used to generate a probability map; The specific process of training the network model with the training set is as follows: Step S31: The input original fundus image passes through the encoder special zone features, and the decoder restores the resolution and outputs the probability map; Step S32: Compare the predicted mask with the true label and calculate the loss function; Step S33: Update the weights through backpropagation and use gradient clipping; Step S34: Calculate the Dice coefficient, IoU, and sensitivity on the validation set every 1 epoch; Step S35: If the validation loss does not decrease for 10 consecutive epochs, terminate the training.

4. A fully self-service fundus photo analysis method according to claim 1, characterized in that In the above Step S4, the fundus lesion recognition model is used to segment the test set to comprehensively evaluate the performance of the lesion recognition model. The accuracy, specificity, sensitivity, and F1 value are used as evaluation criteria. The calculation formulas for relevant indicators are as follows: ； In the formula, The accuracy rate represents the proportion of correctly segmented blood vessel pixels and background pixels in the total pixels of the entire image; Specificity represents the proportion of correctly segmented background pixels to the total number of pixels in the gold standard background; Sensitivity represents the proportion of correctly segmented vessel pixels to the total number of pixels in the gold standard vessels; The value is used to measure the similarity between the segmentation result and the gold standard; represents that vessel pixels are correctly classified as vessels; represents that background pixels are correctly classified as background; represents that background pixels are misclassified as vessels; represents that vessel pixels are misclassified as background.

5. The fully automated fundus photo analysis method according to claim 1, characterized in that In the above Step S5, multi-dimensional data of patients are collected, including demographic information, clinical indicators, imaging data, and laboratory test results. After data cleaning, standardization processing, and data image enhancement, the data are input into the trained lesion recognition model. The lesion recognition model sorts the variables significantly related to the disease risk through LASSO regression or random forest feature importance. The learning rate and tree depth parameters are adjusted using grid search (GridSearch) or Bayesian optimization to balance the risks of overfitting and underfitting. Calculate the area under the ROC curve, trigger the re-evaluation and warning of the risk level, and generate an individual feature contribution waterfall chart to show the quantitative impact of each variable on the current prediction result.

6. A fully self-service fundus photo analysis system, characterized in that, It includes an image acquisition module, an image preprocessing module, a lesion segmentation and annotation module, a U-net model construction module, a model training module, a disease risk prediction module, and an auxiliary diagnosis and decision support module, characterized in that: The image acquisition module is used to acquire the original fundus image; The image preprocessing module is used to sequentially perform channel extraction, size normalization, normalization, histogram equalization, and filtering on the original fundus image. The specific process of channel extraction for the original fundus image is as follows: Step S21: Perform an opening operation on the preprocessed image; Step S22: Perform a closing operation on the image after the opening operation; Step S23: Use an adaptive threshold segmentation algorithm to generate a binary image, and mark the abnormally bright area as a candidate lesion; Step S24: Introduce spatial neighborhood information constraints, initialize the lesion clustering, and iteratively optimize the membership matrix. The specific calculation formula is as follows: ； In the formula, represents the objective function, represents the cluster center data, represents the total number of pixels in the image, represents the th pixel belonging to the th cluster center membership degree, represents the fuzzy factor, represents the th pixel feature vector, represents the th cluster center feature vector, represents the th pixel spatial neighborhood mean value, represents the weight coefficient of the spatial constraint; In the above Step S24, the specific steps for initializing the lesion clustering center are as follows: Step S241: By calculating the local density of each pixel point in the image, select the area with the highest density as the initial clustering center. The specific formula is: ； In the formula, represents the local density of the data point , reflecting the density of other points in its surrounding neighborhood; is the neighborhood radius, represents the Euclidean distance score between the data point and the data point ; is used to adjust the decay rate of the exponential function and control the contribution weight of points within the neighborhood; Step S242: Introduce a spatial weight factor into the objective function to make the membership degrees of adjacent pixels tend to be consistent; Step S243: After each iteration, dynamically correct the membership matrix, and truncate and correct the abnormal membership values; Step S244: Introduce an adaptive distance weight factor to improve the elastic distance metric. The specific formula is as follows: ； In the formula, is the covariance matrix dynamically adjusted according to the contrast of the lesion area; The lesion segmentation and annotation module is used to manually annotate the lesions by ophthalmic experts; The U-net model construction module is used to construct a U-net model; the U-net model includes an encoder, a decoder, and a skip connection module; the encoder constitutes a contraction path, the encoder adopts 4 convolutional blocks, each layer contains 2 3*3 convolutional layers, a total of 8 convolutional layers, and each convolutional block is followed by 1 2*2 max pooling layer, a total of 4 pooling layers; the decoder constitutes an expansion path, the decoder adopts 4 transposed convolutional blocks, each layer contains 1 2*2 transposed convolutional layer, a total of 4 transposed convolutional layers, and 2 3*3 convolutional layers, a total of eight convolutional layers; the skip connection module is used to splice the low-level features of the encoder with the high-level features of the decoder. The model training module is used to train a lesion recognition model through the constructed U-net model. The disease risk prediction module is used to input the fundus image of a patient into the trained lesion recognition model, and the lesion recognition model outputs a lesion recognition result. The auxiliary diagnosis and decision support module is used to generate an auxiliary diagnosis strategy according to the lesion recognition result.

7. A fully self-service fundus photo analysis device, characterized in that, It includes a memory for storing computer programs; a processor for running the computer programs to implement the steps of a fully self-service fundus photo analysis method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, Program instructions are stored, wherein when the program instructions are executed, the steps of a fully self-service fundus photo analysis method as described in any one of claims 1 to 5 are implemented.

Images