Full-self-service fundus photo analysis method, device, storage medium and system

Abstract

The invention discloses a full-self-service fundus photo analysis method and device, a storage medium and a system, and relates to the technical field of image recognition. The method comprises the following steps: loading an original fundus image for preprocessing; extracting a candidate lesion area of the image, and performing morphological operation, threshold segmentation and mean clustering; constructing a U-net model, and training a network model by using the fundus image training set to obtain a fundus focus recognition model; segmenting the test set by the trained fundus focus recognition model to obtain a fundus image segmentation result; a TensorRT or ONNX format is used to realize GPU / CPU cross-platform deployment, and two-stage output is provided: a coarse segmentation result is provided for a doctor to quickly screen, and a refined result is used for diagnosis. The original fundus image is preprocessed to train the focus analysis model, disease risk prediction is performed according to the model analysis result, auxiliary diagnosis and decision support are provided for doctors, and the accuracy and efficiency of disease analysis are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image recognition, and particularly relates to a fully automated fundus photo analysis method, device, storage medium and system, which supports fully automated processing from original image input to quantitative report generation. Through a standardized preprocessing process, an improved deep learning architecture and refined post-processing, efficient and accurate fundus lesion segmentation is achieved, meeting the needs of clinical diagnosis. Background Art

[0002] The morphological structure of fundus retinal blood vessels (such as diameter, branches, angles, etc.) is an important indicator for measuring human health and can reflect various human diseases, such as diabetic retinopathy (DR), glaucoma, hypertension, etc. Therefore, performing fundus image analysis has important clinical significance for assisting doctors in diagnosing and treating various system diseases of the human body.

[0003] Currently, there are still the following deficiencies in fundus photo analysis: High requirements for doctors' professional skills To master funduscopy techniques proficiently and accurately identify various fundus lesion characteristics, doctors need long-term professional training and rich clinical practice experience. The cost of training a qualified ophthalmologist is high and the cycle is long, which to a certain extent limits the number of doctors who can perform high-quality fundus examinations.

[0004] Strong subjectivity Interpretations of fundus photos by different doctors may vary. Even experienced doctors may have inaccurate diagnoses due to subjective factors when faced with complex fundus lesions. For example, for early diabetic retinopathy, the lesion characteristics are relatively subtle, and different doctors may have different judgments on the degree of the lesion.

[0005] Low efficiency Manual interpretation of fundus photos takes a long time. Especially when faced with a large number of patients, doctors have a heavy workload and it is difficult to achieve rapid and batch examinations. In some ophthalmic specialty hospitals, hundreds of patients may be received every day, and it is difficult for doctors to conduct a detailed analysis of all fundus photos in a short time, easily resulting in missed diagnoses or misdiagnoses. Summary of the Invention

[0006] The purpose of the present invention is to provide a fully automated fundus photo analysis method, device, storage medium and system. By preprocessing the original fundus image to train a lesion analysis model, predicting the disease risk based on the model analysis results, and providing auxiliary diagnosis and decision-making support for doctors, the problems of high requirements for doctors' professional skills, strong subjectivity in interpretation, and low efficiency in the existing technology are solved.

[0007] To solve the above technical problems, the present invention is implemented through the following technical solutions: The present invention provides a fully self-service fundus photograph analysis method, including the following steps: Step S1: Load the original fundus image for preprocessing; Step S2: Extract the candidate lesion regions of the image, and perform morphological operations, threshold segmentation, and mean clustering; Step S3: Construct a U-net model, train the network model with the fundus image training set to obtain a fundus lesion recognition model; Step S4: Use the trained fundus lesion recognition model to segment the test set to obtain the fundus image segmentation result; Step S5: Implement cross-platform deployment on GPU / CPU using the TensorRT or ONNX format, providing two-stage outputs: a rough segmentation result for doctors to quickly screen, and a refined result for diagnosis.

[0008] As a preferred technical solution, in step S1, in order to eliminate the phenomena such as uneven brightness, noise, and poor blood vessel contrast commonly existing in fundus images, it is necessary to preprocess the image so that the designed network model can better learn the blood vessel features. The fundus image preprocessing process is as follows: Step S21: Since the retinal blood vessel image usually contains red, blue, and green channel images, and the green channel has a higher contrast, select the green channel image for testing, extract the green channel of the original fundus image to enhance the contrast between blood vessels and lesions, and can better display the blood vessel structure; Step S22: Perform zero-mean and unit normalization on the image after the green channel to make it conform to the normal distribution. The specific formula is as follows: ; In the formula, is the original sample data, is the mean of the original sample data, is the standard deviation of the original sample data, is the labeled image data; Step S23: Perform normalization on the image. The pixel value range of the processed image is between [0,1]. Perform z-score normalization on the OCT image additionally to eliminate the brightness difference collected by different devices. The specific normalization formula is defined as follows: ; In the formula, is the pixel value of the input image, and are the minimum pixel value and the maximum pixel value in the image sample data respectively, is the pixel value after normalization processing; Step S24: Use histogram equalization to enhance the contrast and highlight the edge features of the lesions. Divide the image into blocks, generally into 64 small blocks of 8 rows and 8 columns, and perform histogram equalization operations on each small block; Step S25: Perform median filtering or Gaussian filtering denoising on the noise interference area to reduce noise and improve the contrast and clarity of the fundus image; Step S26: Manually annotate the lesions by ophthalmologists and perform data augmentation on the fundus image data; To prevent the phenomenon of overfitting during network training and improve the segmentation effect and training learning ability, data augmentation is performed on the images in the database training set before network training. The specific data augmentation process is to perform rotations of 0°, 45°, 90°, 135°, and 180° respectively, and then perform horizontal, vertical, and mirror flips respectively to artificially augment the training set for neural network training, avoid the problem of insufficient image data flow, and improve the segmentation performance; Step S27: Divide the preprocessed image into a training set and a test set according to a ratio.

[0009] As a preferred technical solution, in step S2, the specific process of extracting the candidate lesion area is as follows: Step S21: Perform an opening operation on the preprocessed image, that is, erode first and then dilate; The erosion operation eliminates isolated small noise points or burrs, and the dilation restores the main size of the lesion; After the erosion shrinks the edge and then the dilation expands, the main structure is retained while removing detail interference; Step S22: Perform a closing operation on the image after the opening operation, that is, dilate first and then erode; The dilation operation expands the lesion area to fill internal small holes, and the erosion restores the original boundary; The dilation merges adjacent fractured areas, and the erosion avoids over-expansion. After covering the holes by dilation and then retracting by erosion, the overall shape is maintained; Step S23: Use an adaptive threshold segmentation algorithm to generate a binary image and mark the regions with abnormal brightness as candidate lesions; Step S24: Introduce spatial neighborhood information constraints, initialize the lesion clustering, and iteratively optimize the membership matrix.

[0010] As a preferred technical solution, in step S24, the specific steps of initializing the lesion clustering center are as follows: Step S241: Select the area with the highest density as the initial clustering center by calculating the local density of each pixel point in the image; Step S242: Introduce a spatial weight factor into the objective function to make the membership degrees of adjacent pixels tend to be consistent; Step S243: Dynamically correct the membership matrix after each iteration; Step S244: Introduce an adaptive distance weight factor to improve the elastic distance metric.

[0011] As a preferred technical solution, in step S3, the basic network structure of the U-net model includes an encoder, a decoder, and an output layer; the encoder includes 4 downsampling stages, each stage contains two 3×3 convolutional layers, followed by 2×2 max pooling; each stage of the decoder contains two 3×3 convolutional layers, the number of channels is halved step by step, and each stage uses transposed convolution or bilinear interpolation for upsampling; the output layer is a 1×1 convolutional layer, the number of output channels is the number of categories, and the Sigmoid activation function is used to generate a probability map; The specific process of training the network model with the training set is as follows: Step S31: The input original fundus image passes through the encoder to extract features, and the decoder restores the resolution and outputs a probability map; Step S32: Compare the predicted mask with the true label and calculate the loss function; Step S33: Update the weights through backpropagation and use gradient clipping; Step S34: Calculate the Dice coefficient, IoU, and sensitivity on the validation set every 1 epoch; Step S35: If the validation loss does not decrease for 10 consecutive epochs, terminate the training.

[0012] As a preferred technical solution, in step S5, collect multi-dimensional data of patients, including demographic information (age, gender), clinical indicators (blood pressure, blood sugar), imaging data (lesion area, location), and laboratory test results. After data cleaning, standardization processing, and data image enhancement, input them into the trained lesion recognition model. The lesion recognition model uses LASSO regression or random forest feature importance ranking to screen variables significantly related to disease risk, and uses grid search or Bayesian optimization to adjust parameters such as the learning rate and tree depth to balance the risks of overfitting and underfitting, calculate the area under the ROC curve, trigger a re-evaluation and early warning of the risk level, and generate an individual feature contribution waterfall chart to show the quantitative impact of each variable on the current prediction result.

[0013] The present invention is a fully self-service fundus photo analysis system, including an image acquisition module, an image preprocessing module, a lesion segmentation and annotation module, a U-net model construction module, a model training module, a disease risk prediction module, and an auxiliary diagnosis and decision support module; The image acquisition module is used to collect original fundus images; The image preprocessing module is used to perform channel extraction, size normalization, normalization, histogram equalization, and filtering on the original fundus image in sequence; The lesion segmentation and annotation module is used to manually annotate lesions by ophthalmology experts; The U-net model construction module is used to construct a U-net model; the U-net model includes an encoder, a decoder, and a skip connection module; the encoder constitutes a contraction path, the encoder uses 4 convolutional blocks, each layer contains 2 3*3 convolutional layers, a total of 8 convolutional layers, and each convolutional block is followed by 1 2*2 max pooling layer, a total of 4 pooling layers; the decoder constitutes an expansion path, the decoder uses 4 transposed convolutional blocks, each layer contains 1 2*2 transposed convolutional layer, a total of 4 transposed convolutional layers, 2 3*3 convolutional layers, a total of eight convolutional layers; the skip connection module is used to splice the low-level features of the encoder and the high-level features of the decoder; The model training module is used to train a lesion recognition model through the constructed U-net model; The disease risk prediction module is used to input the fundus image of a patient into the trained lesion recognition model, and the lesion recognition model outputs the lesion recognition result; The auxiliary diagnosis and decision support module is used to generate an auxiliary diagnosis strategy according to the lesion recognition result.

[0014] The present invention is a fully self-service fundus photo analysis device, including a memory for storing a computer program; a processor for running the computer program to implement the steps of a fully self-service fundus photo analysis method as described above.

[0015] The present invention is a storage medium storing program instructions, wherein the program instructions, when executed, implement the steps of a fully self-service fundus photo analysis method as described above.

[0016] The present invention has the following beneficial effects: The present invention preprocesses the original fundus image to train a lesion analysis model, predicts the disease risk according to the model analysis result, provides auxiliary diagnosis and decision support for doctors, improves the accuracy and efficiency of disease analysis, and assists doctors in diagnosis.

[0017] The present invention extracts candidate lesion regions, and uses a serialized combination of erosion and dilation. The opening and closing operations effectively solve the problems of noise interference and internal discontinuity while retaining the main structure of the lesions, providing basic support for medical image analysis.

[0018] The present invention converts the distance information of points in the neighborhood into density estimation through exponential weighting, and selects the point with the highest density as the initial clustering center to avoid the instability of random initialization. By suppressing the interference of distant noise points and combining spatial neighborhood constraints to smooth the membership matrix, the lesion area can be more accurately located, while reducing the number of iterations and enhancing the robustness to noise.

[0019] The present invention performs triple optimizations on the localization of candidate lesion areas, namely density-guided initialization, spatial constraint modeling, and dynamic parameter adjustment, which significantly improves the operation efficiency and anti-noise ability while ensuring the segmentation accuracy, and is applicable to the lesion localization of multi-modal medical images such as CT and OCT. Of course, it is not necessary for any product implementing the present invention to achieve all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 It is a flowchart of a fully self-service fundus photo analysis method of the present invention; Figure 2 It is a schematic diagram of the preprocessing of the original fundus image; Figure 3 It is a schematic diagram of the structure of a fully self-service fundus photo analysis system of the present invention; Figure 4 It is an effect diagram of the analysis and processing of fully self-service fundus photos. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0023] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0024] In order to make the purpose, technical solutions and advantages of the present application more clear, the following will be combined with the attached Figures 1-4And the embodiments are used to further elaborate on this application in detail. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0025] Example 1. Please refer to Figure 1 As shown, the present invention is a fully self-service fundus photograph analysis method, including the following steps: Step S1: Load the original fundus image for preprocessing; Step S2: Extract the candidate lesion regions of the image, and perform morphological operations, threshold segmentation, and mean clustering; Step S3: Construct a U-net model, train the network model with the fundus image training set, and obtain a fundus lesion recognition model; Step S4: Use the trained fundus lesion recognition model to segment the test set to obtain the fundus image segmentation result; Step S5: Implement GPU / CPU cross-platform deployment using the TensorRT or ONNX format, and provide two-stage outputs: a rough segmentation result for doctors to quickly screen, and a refined result for diagnosis.

[0026] In step S1, in order to eliminate the phenomena such as uneven brightness, noise, and poor blood vessel contrast commonly existing in fundus images, it is necessary to preprocess the images so that the designed network model can better learn the blood vessel features. The fundus image preprocessing process is as follows: Step S21: Since the retinal blood vessel image usually contains red, blue, and green channel images, and the green channel has a higher contrast, select the green channel image for testing, extract the green channel of the original fundus image to enhance the contrast between blood vessels and lesions, and can better display the blood vessel structure; Step S22: Perform zero-mean and unit normalization processing on the image after the green channel to make it conform to the normal distribution. The specific formula is as follows: ; In the formula, is the original sample data, is the mean of the original sample data, is the standard deviation of the original sample data, is the labeled image data; Step S23: Perform normalization processing on the image. The pixel value range of the processed image is between [0,1]. Perform z-score normalization on the OCT image additionally to eliminate the brightness differences collected by different devices. The specific normalization formula is defined as follows: ; In the formula, is the pixel value of the input image, and are the minimum and maximum pixel values in the image sample data respectively, is the pixel value after normalization; Step S24: Use histogram equalization to enhance the contrast and highlight the edge features of the lesions. The image is divided into blocks, generally into 64 small blocks of 8 rows and 8 columns, and histogram equalization operation is performed on each small block; Please refer to Figure 2 As shown, Step S25: Perform median filtering or Gaussian filtering denoising on the noise interference area to reduce noise and improve the contrast and clarity of the fundus image; Step S26: Ophthalmic experts manually mark the lesions and perform data augmentation on the fundus image data; In order to prevent the phenomenon of overfitting during network training and improve the segmentation effect and training learning ability, data augmentation is performed on the images in the database training set before network training. The specific data augmentation process is to rotate at 0°, 45°, 90°, 135°, and 180° respectively, and then perform horizontal, vertical, and mirror flips respectively to artificially augment the training set for neural network training, avoiding the problem of insufficient image data flow and improving the segmentation performance; Step S27: Divide the preprocessed image into a training set and a test set according to a ratio. Generally, the ratio of the training set to the test set is 8:2.

[0027] In Step S2, the specific process of extracting the candidate lesion area is as follows: Step S21: Perform an opening operation on the preprocessed image, that is, erode first and then dilate; Step S22: Perform a closing operation on the image after the opening operation, that is, dilate first and then erode; Step S23: Use an adaptive threshold segmentation algorithm to generate a binary image and mark the abnormally bright area as a candidate lesion; Step S24: Introduce spatial neighborhood information constraints, initialize the lesion clustering, and iteratively optimize the membership matrix; The specific calculation formula is as follows: ; In the formula, represents the objective function, represents the clustering center data, represents the total number of pixels in the image, represents the th pixel belongs to the th clustering center's membership degree, represents the fuzzy factor, represents the th pixel's feature vector, represents the th clustering center's feature vector, represents the The spatial neighborhood mean of a pixel represents the weight coefficient for spatial constraint.

[0028] In step S24, the specific steps for initializing the lesion clustering center are as follows: Step S241: By calculating the local density of each pixel point in the image, select the area with the highest density as the initial clustering center; the specific formula is: ; In the formula, represents the local density of the data point and reflects the density of other points in its surrounding neighborhood; is the neighborhood radius, represents the data point and the data point is the Euclidean distance score between them; is used to adjust the decay rate of the exponential function and control the contribution weight of points in the neighborhood; This formula converts the distance information of points in the neighborhood into density estimation through exponential weighting: in high-density regions (such as the lesion center), the distance to surrounding points is close, the value increases significantly; in low-density regions (such as noise or background), the surrounding points are sparse or far away, the value is low.

[0029] Step S242: Introduce a spatial weight factor into the objective function to make the membership degrees of adjacent pixels tend to be consistent; Step S243: After each iteration, dynamically correct the membership matrix to suppress the influence of isolated noise points on clustering. Truncate and correct abnormal membership values (such as those deviating more than 2σ from the mean) to improve the robustness of the algorithm; Step S244: To address the problem of uneven gray distribution in the lesion area, introduce an adaptive distance weight factor to improve the elastic distance metric; the specific formula is as follows: ; In the formula, is the covariance matrix dynamically adjusted according to the contrast of the lesion area.

[0030] In step S3, the basic network structure of the U-net model includes an encoder, a decoder, and an output layer; the encoder includes 4 downsampling stages, each stage contains two 3×3 convolutional layers, followed by 2×2 max pooling; each stage of the decoder contains two 3×3 convolutional layers, the number of channels is halved step by step, and each stage uses transposed convolution or bilinear interpolation for upsampling; the output layer is a 1×1 convolutional layer, the number of output channels is the number of classes, and the Sigmoid activation function is used to generate a probability map; an activation function is used to classify effective vascular pixels and background noise pixels to achieve binary classification of the output feature image, and zero padding is used for padding in each convolutional layer; The specific process of training the network model with the training set is as follows: Step S31: The input original fundus image extracts features through the encoder, and the decoder restores the resolution and outputs a probability map; Step S32: Compare the predicted mask with the true label and calculate the loss function; the calculation formula of the loss function is as follows: ; In the formula, represents the label of sample , 1 for blood vessels and 0 for background and interference, represents the probability that sample is predicted to be a blood vessel; Step S33: Update the weights through backpropagation, and use gradient clipping to prevent gradient explosion; Step S34: Calculate the Dice coefficient, IoU, and sensitivity on the validation set every 1 epoch; Step S35: If the validation loss does not decrease for 10 consecutive epochs, terminate the training.

[0031] The bias parameters and weights in the training network are randomly initialized using a normal distribution, the loss function is optimized using binary cross-entropy, the activation function after each convolutional layer during 20 epochs of training is ReLU, and 0.2 dropout is used between two consecutive convolutional layers. The batchsize of model training is set to 4, and the learning rate is set to 0.001.

[0032] In step S5, multi-dimensional data of patients are collected, including demographic information (age, gender), clinical indicators (blood pressure, blood sugar), imaging data (lesion area, location), and laboratory test results. After data cleaning, standardization, and data image enhancement, they are input into the trained lesion recognition model. Among them, for data cleaning and standardization, missing values are filled using multiple imputation methods, and outliers are detected and corrected through box plots or the 3σ principle; continuous variables are normalized (Min-Max or Z-score), and categorical variables are one-hot encoded; for medical image enhancement, for imaging data (such as CT, MRI), enhancement techniques such as histogram equalization, random rotation / flip are used to improve the robustness of the model to lesion morphology. The lesion recognition model ranks the variables significantly related to disease risk through LASSO regression or random forest feature importance (p-value < 0.05 or feature weight > threshold), generates combined features (such as BMI × blood sugar level) based on medical prior knowledge to capture the synergistic effects of multiple factors; uses grid search or Bayesian optimization to adjust parameters such as the learning rate and tree depth to balance the risks of overfitting and underfitting. The area under the ROC curve (AUC ≥ 0.75 is the clinical availability standard), with a focus on high sensitivity (> 85%) to reduce missed diagnoses. And the Brier score (< 0.1) and calibration curve are used to evaluate the consistency between the predicted probability and the actual risk. The generalization of the model is verified in an external dataset (such as data from different hospitals or regions), requiring the AUC decline to be < 5%; triggering a re-evaluation of the risk level and early warning, generating an individual feature contribution waterfall plot to show the quantitative impact of each variable on the current prediction result, and converting the model output into a clinically actionable scoring tool, allowing doctors to quickly estimate the risk probability by superimposing the feature scores.

[0033] The recognition results are divided into: Low-risk group (probability < 0.3): Annual regular follow-up; Medium-risk group (0.3 - 0.6): Quarterly special examinations (such as tumor marker detection); High-risk group (> 0.6): Initiate multi-disciplinary consultation (MDT) and formulate an intervention plan; And a dynamic update mechanism is set: Re-enter the latest indicators of the patient every 3 - 6 months, triggering a re-evaluation of the risk level and early warning (such as automatically pushing an alarm when the risk jump > 20%).

[0034] Example 2, refer to Figures 3-4As shown in the figure, the present invention is a fully self-service fundus photo analysis system, which can be used to execute the method content of Embodiment 1 of the present invention, including: an image acquisition module, an image preprocessing module, a lesion segmentation and annotation module, a U-net model construction module, a model training module, a disease risk prediction module, and an auxiliary diagnosis and decision support module, characterized in that: The image acquisition module is used to collect the original fundus images; The image preprocessing module is used to sequentially perform channel extraction, size normalization, normalization, histogram equalization, and filtering on the original fundus images; The lesion segmentation and annotation module is used to manually annotate the lesions by ophthalmic experts; The U-net model construction module is used to construct a U-net model; the U-net model includes an encoder, a decoder, and a skip connection module; the encoder constitutes a contraction path, and the encoder adopts 4 convolutional blocks. Each level contains 2 3×3 convolutional layers, and the ReLU activation function is used for all of them, with a total of 8 convolutional layers. And after each convolutional block, there is a 2×2 max pooling layer, with a total of 4 pooling layers for downsampling; after each level of downsampling, the spatial size of the feature map is halved; for example, it decreases from 512×512 to 256×256, and the number of channels doubles, such as doubling from 64 to 128; The specific process is as follows: The first level: input image (such as 572×572×3) → convolution (3×3, 64 channels) → convolution (3×3, 64 channels) → max pooling (output 284×284×64); The second to fourth levels: repeat the above operations, and the feature map is gradually reduced to 32×32×1024, and the number of channels increases to 1024; The decoder constitutes an expansion path, and the decoder adopts 4 transposed convolutional blocks. Each level contains 1 2×2 transposed convolutional layer for upsampling, with a total of 4 transposed convolutional layers and 2 3×3 convolutional layers, for a total of eight convolutional layers, which are used to splice with the feature maps of the same layer of the encoder, and optimize and fuse the features through 2 3×3 convolutional layers; The specific process is as follows: The first level: input 32×32×1024 → transposed convolution (output 64×64×512) → splice the fourth-level feature of the encoder (64×64×512) → convolution (3×3, 512 channels) → convolution (3×3, 512 channels); For subsequent levels: repeat the upsampling, splicing, and convolution operations, and finally restore to the original resolution (such as 568×568×64); The skip connection module is used to splice the low-level features (such as edge details) of the encoder with the high-level features of the decoder to enhance the boundary segmentation accuracy; The model training module is used to train the lesion recognition model through the constructed U-net model; The disease risk prediction module is used to input the fundus image of a patient into a trained lesion recognition model, and the lesion recognition model outputs the lesion recognition result; The auxiliary diagnosis and decision support module is used to generate an auxiliary diagnosis strategy according to the lesion recognition result.

[0035] It should be noted that in the above system embodiments, the included units are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0036] In addition, those of ordinary skill in the art can understand that all or part of the steps in the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the corresponding program can be stored in a computer-readable storage medium.

[0037] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and utilize the present invention well. The present invention is only limited by the claims and their full scope and equivalents..

Claims

1. A fully self-service fundus photography analysis method, characterized in that: The steps include: Step S1: loading the original fundus image for preprocessing; Step S2: extracting candidate lesion areas of the image, performing morphological operations, threshold segmentation and mean clustering; Step S3: constructing a U-net model, training the network model with the fundus image training set, and obtaining a fundus lesion recognition model; Step S4: The trained fundus lesion recognition model segments the test set to obtain fundus image segmentation results; Step S5: Use TensorRT or ONNX format to achieve GPU / CPU cross-platform deployment and provide two-stage output: coarse segmentation results for doctors to quickly screen, and refined results for diagnosis.

2. A fully automated fundus photography analysis method according to claim 1, characterized in that: In step S1, the fundus image preprocessing process is as follows: Step S11: extracting the green channel of the original fundus image to enhance the contrast between blood vessels and lesions; Step S12: performing zero mean conversion and unit standardization processing on the image after the green channel to make it conform to the normal distribution; Step S13: normalize the image to eliminate the brightness difference collected by different devices; Step S14: using histogram equalization to enhance contrast and highlight the edge features of the lesion; Step S15: performing median filtering or Gaussian filtering noise reduction processing on the noise interference area; Step S16: The ophthalmologist manually marks the lesions and performs fundus image data amplification; Step S17: Divide the preprocessed images into a training set and a test set according to the ratio.

3. A fully automated fundus photography analysis method according to claim 1, characterized in that: In step S2, the specific process of extracting the candidate lesion region is as follows: Step S21: performing an opening operation on the preprocessed image, i.e., first erosion and then expansion; Step S22: performing a closing operation on the image after the opening operation, that is, first dilating and then corroding; Step S23: using an adaptive threshold segmentation algorithm to generate a binary image, marking the abnormal brightness area as a candidate lesion; Step S24: Introduce spatial neighborhood information constraints, initialize lesion cluster centers, and iteratively optimize the membership matrix.

4. A fully automated fundus photography analysis method according to claim 3, characterized in that: In step S24, the specific steps of initializing the lesion cluster center are as follows: Step S241: Calculate the local density of each pixel in the image and select the area with the highest density as the initial cluster center; Step S242: introducing a spatial weight factor into the objective function to make the membership of neighboring pixels tend to be consistent; Step S243: After each iteration, dynamically modify the membership matrix; Step S244: introducing an adaptive distance weight factor to improve the elastic distance metric.

5. The fully automated fundus photography analysis method according to claim 1, characterized in that: In step S3, the basic network structure of the U-net model includes an encoder, a decoder and an output layer; the encoder includes 4 downsampling stages, each of which includes two 3×3 convolutional layers followed by 2×2 maximum pooling; each stage of the decoder includes two 3×3 convolutional layers, the number of channels is halved step by step, and each stage uses transposed convolution or bilinear interpolation for upsampling; the output layer is a 1×1 convolutional layer, the number of output channels is the number of categories, and the Sigmoid activation function is used to generate a probability map; The specific process of training the network model with the training set is as follows: Step S31: input the original fundus image through the encoder special area features, and the decoder restores the resolution and outputs a probability map; Step S32: Compare the predicted mask with the true label and calculate the loss function; Step S33: Update weights by back propagation, using gradient clipping; Step S34: Calculate the Dice coefficient, IoU and sensitivity on the validation set every epoch; Step S35: If the verification loss does not decrease for 10 consecutive epochs, the training is terminated.

6. A fully automated fundus photography analysis method according to claim 1, characterized in that: In step S4, the fundus lesion recognition model segments the test set for comprehensive evaluation of the performance of the lesion recognition model, using accuracy, specificity, sensitivity and F1 value as evaluation criteria, and the calculation formulas for the relevant indicators are as follows: ； In the formula, the accuracy Indicates the ratio of correctly segmented blood vessel pixels and background pixels to the total pixels of the entire image; Specificity Indicates the ratio of correctly segmented background pixels to the total pixels of the gold standard background; sensitivity It indicates the ratio of correctly segmented blood vessel pixels to the total pixels of the gold standard blood vessels; The value is used to measure the similarity between the segmentation result and the gold standard; indicates that the vessel pixels are correctly classified as vessels; indicates that the background pixels are correctly classified as background; indicates that background pixels are misclassified as blood vessels; Indicates that the vessel pixels are misclassified as background.

7. The fully automated fundus photography analysis method according to claim 1, characterized in that: In step S5, multi-dimensional data of patients are collected, including demographic information, clinical indicators, imaging data and laboratory test results. After data cleaning, standardization and data image enhancement, the data is input into the trained lesion recognition model. The lesion recognition model uses LASSO regression or random forest feature importance sorting to screen variables significantly related to disease risk, and uses grid search (GridSearch) or Bayesian optimization to adjust the learning rate and tree depth parameters, balance the risks of overfitting and underfitting, calculate the area under the ROC curve, trigger risk level reassessment and early warning, generate an individualized feature contribution waterfall chart, and display the quantitative impact of each variable on the current prediction result.

8. A fully self-service fundus photo analysis system, characterized in that: It includes an image acquisition module, an image preprocessing module, a lesion segmentation and annotation module, a U-net model construction module, a model training module, a disease risk prediction module, and an auxiliary diagnosis and decision support module, and is characterized by: The image acquisition module is used to acquire original fundus images; The image preprocessing module is used to perform channel extraction, size standardization, normalization, histogram equalization and filtering processing on the original fundus image in sequence; The lesion segmentation and labeling module is used for manually labeling lesions by ophthalmologists; The U-net model construction module is used to construct a U-net model; the U-net model includes an encoder, a decoder and a jump connection module; the encoder constitutes a contraction path, the encoder uses 4 convolution blocks, each level includes 2 3*3 convolution layers, a total of 8 convolution layers, and each convolution block is followed by a 2*2 maximum pooling layer, a total of 4 pooling layers; the decoder constitutes an expansion path, the decoder uses 4 deconvolution blocks, each level includes 1 2*2 transposed convolution layer, a total of 4 deconvolution layers, 2 3*3 convolution layers, a total of eight convolution layers; the jump connection module is used to splice the low-level features of the encoder with the high-level features of the decoder; The model training module is used to train the lesion recognition model through the constructed U-net model; The disease risk prediction module is used to input the patient's fundus image into a trained lesion recognition model, and the lesion recognition model outputs a lesion recognition result; The auxiliary diagnosis and decision support module is used to generate an auxiliary diagnosis strategy based on the lesion identification result.

9. A fully self-service fundus photography analysis device, characterized in that: It comprises a memory for storing a computer program; and a processor for running the computer program to implement the steps of a fully self-service fundus photography analysis method as claimed in any one of claims 1 to 7.

10. A storage medium, characterized in that: Program instructions are stored, wherein when the program instructions are executed, the steps of a fully self-service fundus photography analysis method as claimed in any one of claims 1 to 7 are implemented.

Images