Mobile terminal wide-angle laser fundus image cup-to-disk ratio detection method and system based on deep learning

Abstract

The invention provides a mobile terminal wide-angle laser fundus image cup-to-disk ratio detection method based on deep learning. The method comprises the following steps: acquiring a fundus image data set; performing eye fundus image optic cup and optic disk target detection training on the eye fundus image data set through a deep learning network structure to obtain a high-precision optic cup and optic disk target detection model; based on a high-precision optic cup and optic disk target detection model, recognition training is carried out on the fundus image data set through a lightweight deep learning network structure, and a lightweight mobile terminal deployment model is obtained; and processing the fundus image through a model deployed by the lightweight mobile terminal to obtain fundus optic cup and optic disk target data. According to the invention, through training and lightweight processing, the efficient model suitable for the mobile device is obtained. According to the method, the eye fundus image is accurately recognized on the mobile device, target data of the optic cup and the optic disk are provided, the real-time performance and portability of examination are improved, and meanwhile the detection precision is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of ophthalmic image analysis, and in particular to a method and system for detecting the cup-to-disc ratio of a mobile wide-angle laser fundus image based on deep learning. Background Art

[0002] Glaucoma is the second leading cause of blindness in the world. Early detection and treatment can effectively reduce the risk of blindness. Although Heidelberg retinal tomography (HRT) and ocular coherence tomography (OCT) can provide accurate test results, the operation is time-consuming and requires professional skills. In contrast, digital fundus images are more economical and suitable for basic ophthalmic diagnosis, but manual detection methods are not suitable for large-scale screening. Therefore, it is particularly important to design a reliable glaucoma detection system. Among them, the cup-to-disc ratio (CDR) expanded in the vertical direction is an important indicator for evaluating glaucoma. Accurate calculation of CDR requires accurate measurement of the optic disc and optic cup.

[0003] Traditional automatic cup-to-disc ratio measurement methods rely on classic image processing techniques, such as Canny edge detection and Hough transform. These methods segment the optic disc and cup by manual feature extraction and rule design. Although they improve efficiency, they still have limitations in accuracy and robustness. Another method is to use threshold methods and level set techniques to extract the boundaries of the optic disc and cup and calculate the cup-to-disc ratio through an adaptive model, but due to the accumulation of segmentation errors, the prediction reliability of this method is also low. Summary of the invention

[0004] In view of this, the present invention is committed to providing a mobile wide-angle laser fundus image cup-to-disc ratio detection method and system based on deep learning, which is used to solve the technical problems of the prior art methods for measuring the cup-to-disc ratio, such as low timeliness, low accuracy, low robustness and large resource consumption.

[0005] The present invention provides a method for detecting cup-to-disc ratio of mobile wide-angle laser fundus images based on deep learning, the method comprising:

[0006] Acquire a fundus image dataset;

[0007] Performing fundus image optic cup and optic disc target detection training on the fundus image dataset through a deep learning network structure to obtain a high-precision optic cup and optic disc target detection model;

[0008] Based on the high-precision optic cup and optic disc target detection model, the fundus image dataset is trained for recognition through a lightweight deep learning network structure to obtain a lightweight mobile terminal deployment model;

[0009] The fundus image is processed by the model deployed on the lightweight mobile terminal to obtain fundus optic cup and optic disc target data.

[0010] Optionally, the method further includes performing data enhancement on the fundus image dataset to obtain an enhanced fundus image dataset.

[0011] Optionally, performing data enhancement on the fundus image dataset to obtain the enhanced fundus image dataset includes mosaic transformation, movement, contraction, rotation, cropping, horizontal and vertical flipping, and hue, saturation and value enhancement on the fundus image dataset.

[0012] Optionally, the method further includes performing model pruning, quantization and optimization processing on the model deployed on the lightweight mobile terminal.

[0013] Optionally, the fundus optic cup and optic disc target data include the coordinates, width and height of the center point of the bounding box, which are obtained by the following formula:

[0014]

[0015]

[0016]

[0017]

[0018] in, Respectively represent the number of grids on the y and x axes; H and W are the height and width of the input fundus image, are the width and height of the jth anchor point respectively; the bounding box predicted by the jth anchor point in the kth row and lth column grid of the i-th batch index image can be expressed as the center point coordinate x P ,y p , the width of the bounding box w P and height h p To express.

[0019] Optionally, the processing of the fundus image by the model deployed by the lightweight mobile terminal to obtain fundus optic cup and optic disc target data includes:

[0020] Acquire the image to be detected;

[0021] Preprocessing the image to be detected;

[0022] Inputting the preprocessed image data into the model deployed on the lightweight mobile terminal for inference calculation to obtain the model output result;

[0023] Decode the model output and extract the detected object boxes, categories, and confidences.

[0024] Optionally, the method further comprises identifying and correcting the fundus optic cup and the optic disc target data to obtain more accurate cup-to-disc ratio data.

[0025] Optionally, the identifying and correcting the fundus optic cup and the optic disc target data comprises:

[0026] Screening the optic cup and optic disc detection frames to obtain the optic cup and optic disc detection frames with the highest confidence;

[0027] Determining that the optic cup in the detection frame is surrounded by the optic disc;

[0028] Determining that the sizes of the optic cup and the optic disc meet the requirements;

[0029] The coordinates and size information of the optic cup and optic disc are output.

[0030] In a second aspect, the present invention provides a fundus image cup-to-disc ratio detection system based on a mobile wide-angle laser fundus image cup-to-disc ratio detection method based on deep learning, the system comprising:

[0031] A basic network training module is configured to perform fundus image optic cup and optic disc target detection training on the fundus image dataset through a deep learning network structure to obtain a high-precision optic cup and optic disc target detection model;

[0032] The mobile terminal network integration module is configured to perform recognition training on the fundus image data set based on the high-precision optic cup and optic disc target detection model through a lightweight deep learning network structure to obtain a lightweight mobile terminal deployed model.

[0033] Optionally, the system further comprises:

[0034] A training data enhancement module is configured to perform data enhancement on the fundus image dataset to obtain an enhanced fundus image dataset;

[0035] The cup-to-disc ratio measurement module is configured to identify and correct the fundus optic cup and the optic disc target data to obtain more accurate cup-to-disc ratio data.

[0036] The present invention trains the fundus image dataset through a deep learning network structure to obtain a high-precision optic cup and optic disc target detection model, and trains the model through a lightweight deep learning network structure to achieve lightweight mobile terminal deployment. This method can not only efficiently process fundus images on mobile devices, but also accurately identify optic cup and optic disc target data, improve the accuracy and efficiency of detection, and is suitable for real-time applications. The lightweight model design ensures that high-performance detection effects can still be provided on resource-constrained mobile devices, which is suitable for large-scale screening and telemedicine scenarios.

[0037] Furthermore, the present invention effectively increases the diversity of training samples and improves the generalization and robustness of the model by performing data enhancement on the fundus image dataset. At the same time, model pruning, quantization, and optimization processing further reduce the computational complexity and storage requirements of the model, enabling it to be efficiently deployed on mobile devices. By accurately obtaining the coordinates, width, and height of the center point of the bounding box of the fundus optic cup and optic disc, and preprocessing and decoding the results of the images to be detected, accurate and efficient cup and disc target detection is achieved. In addition, by identifying the correction step, the accuracy of the cup-to-disc ratio data is ensured, which is suitable for clinical and telemedicine applications.

[0038] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic flow chart of a method for detecting cup-to-disc ratio of wide-angle laser fundus images on a mobile terminal based on deep learning according to an embodiment of the present invention is shown;

[0040] Figure 2 Shows Figure 1 A schematic flow chart of a method for processing a fundus image by a model deployed on a lightweight mobile terminal to obtain fundus optic cup and optic disc target data in step S400;

[0041] Figure 3 A schematic flow chart showing a method for identifying and correcting fundus optic cup and optic disc target data to obtain more accurate cup-to-disc ratio data according to an embodiment of the present invention;

[0042] FIG4 is a schematic diagram showing a cup-to-disc ratio detection result according to an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of the PR accuracy result of cup-to-disc ratio detection according to an embodiment of the present invention is shown;

[0044] Figure 6 A schematic diagram of the accuracy result of cup-to-disc ratio detection F1 according to an embodiment of the present invention is shown;

[0045] Figure 7 A structural block diagram of a fundus image cup-to-disc ratio detection system of a mobile-end wide-angle laser fundus image cup-to-disc ratio detection method based on deep learning according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some structures related to the present application are shown in the accompanying drawings, rather than all structures. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0047] The terms "including" and "having" and any variations thereof in this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0048] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0049] Figure 1 FIG. 1 is a schematic flow chart of a method for detecting cup-to-disc ratio of mobile wide-angle laser fundus images based on deep learning according to an embodiment of the present invention. Figure 1 As shown, the fundus image cup-to-disc ratio detection method includes the following steps:

[0050] Step S100, acquiring a fundus image dataset.

[0051] In some embodiments, lasers with wavelengths of 532nm and 785nm are used to generate multi-color photos by combining laser images of these two wavelengths, which significantly improves the penetration ability of the image and reduces the absorption and reflection of the refractive medium, thereby enhancing the imaging effect of the lesion area. Since lasers of different wavelengths have different tissue penetration capabilities, the generated multi-color photos can show different levels of characteristics of fundus lesion tissue in the same image, which is helpful for more accurate disease identification and diagnosis. At the same time, through the multi-angle wide-angle lens optical design, fundus images of different angles and regions are collected. For example, using 15°, 30°, and 60° lenses to shoot the macular area, with a lesion resolution of 5 microns, small lesions can be displayed, ensuring high image clarity and rich details, which is suitable for deep learning model training. At the same time, by collecting images at an ultra-wide angle of 165° and stitching the images with lenses to achieve a 240° field of view, more than 90% of the retina can be covered, the detection rate of peripheral lesions can be improved, and lesions can be avoided from being missed. The multi-angle wide-angle lens design fully covers the retinal area, improves the detection rate of peripheral lesions, avoids missing lesions, ensures the comprehensiveness and diversity of the data set, and does not require mydriasis and dark rooms. It is suitable for a wider range of patients, has a faster detection speed, and facilitates the rapid collection of large amounts of clear and available fundus image data.

[0052] Step S200, training fundus image optic cup and optic disc target detection on a fundus image dataset through a deep learning network structure to obtain a high-precision optic cup and optic disc target detection model.

[0053] The fundus image dataset obtained above is used as a training sample, and a deep learning algorithm is used to train the fundus image optic cup and optic disc target detection, identifying and annotating the optic cup and optic disc areas in the image. By continuously optimizing the model parameters, a high-precision optic cup and optic disc target detection model is finally obtained. This high-precision optic cup and optic disc target detection model can accurately detect and identify the optic cup and optic disc in fundus images, laying the foundation for subsequent model lightweighting and mobile terminal deployment. This step ensures the high accuracy and reliability of the model and is the key to achieving accurate detection.

[0054] Step S300, based on the high-precision optic cup and optic disc target detection model, the fundus image dataset is trained for recognition through a lightweight deep learning network structure to obtain a lightweight model deployed on a mobile terminal.

[0055] Based on the high-precision optic cup and optic disc target detection model, the fundus image dataset is trained again using a lightweight deep learning network structure. By using a more efficient network architecture, the model can adapt to the computing resource limitations of mobile devices while maintaining detection accuracy. By using a lightweight deep learning network structure, the computational complexity and storage requirements of the model can be effectively reduced. This enables the generated model to run efficiently on resource-constrained mobile devices, ensuring real-time and portability, and is suitable for clinical and telemedicine applications.

[0056] Step S400, processing the fundus image by the model deployed on the lightweight mobile terminal to obtain fundus optic cup and optic disc target data.

[0057] The lightweight model is deployed on a mobile device and the fundus image is processed. The model extracts the coordinates, width, and height of the center point of the bounding box of the optic cup and optic disc through inference calculation. Efficient and real-time optic cup and optic disc detection on a mobile device can quickly provide analysis results of the fundus image, making it easier for clinicians to make diagnosis and treatment decisions on-site or in remote medical care, and improving the convenience and practicality of fundus image detection.

[0058] Through the above embodiments, the present invention trains the fundus image data set through a deep learning network structure to obtain a high-precision optic cup and optic disc target detection model, and performs recognition training on the model through a lightweight deep learning network structure, thereby realizing lightweight mobile terminal deployment. This method can not only efficiently process fundus images on mobile devices, but also accurately identify optic cup and optic disc target data, thereby improving the accuracy and efficiency of detection and being suitable for real-time applications. The lightweight model design ensures that high-performance detection effects can still be provided on resource-constrained mobile devices, and is suitable for large-scale screening and telemedicine scenarios.

[0059] In some embodiments, the above-mentioned mobile-end wide-angle laser fundus image cup-to-disc ratio detection method based on deep learning also includes the step of performing data enhancement on the fundus image dataset to obtain an enhanced fundus image dataset.

[0060] In some embodiments, data enhancement of the fundus image dataset includes mosaic transformation, movement, shrinkage, rotation, cropping, horizontal and vertical flipping, and enhancement of hue, saturation, and value. This process generates diverse image variants, expands the training sample size, and improves the richness of the dataset. Data enhancement also improves the robustness and generalization ability of the model, prevents overfitting, and enhances the detection accuracy of the model under various conditions. By expanding and diversifying the dataset, the model can handle different image transformations more accurately, ensuring higher detection performance.

[0061] In some embodiments, the above-mentioned deep learning-based mobile terminal wide-angle laser fundus image cup-to-disc ratio detection method also includes the steps of pruning, quantizing and optimizing the model deployed on the lightweight mobile terminal.

[0062] Model pruning reduces the complexity of the model by removing redundant network parameters and connections. Quantization converts the weights and activation values ​​of the model from high-precision floating-point numbers to low-precision integers, reducing storage and computing requirements. Optimization includes improving network structure and algorithms to improve operating efficiency. Model pruning, quantization, and optimization processes significantly reduce the computing and storage requirements of the model, allowing it to run more efficiently on mobile devices. This not only improves the real-time and responsiveness of the model, but also extends the battery life of the device, making it suitable for clinical and telemedicine applications. While ensuring detection accuracy, these processing steps improve the practicality and adaptability of the model.

[0063] Figure 2 Shows Figure 1 A schematic flow chart of a method for processing the fundus image by using a model deployed on a lightweight mobile terminal to obtain fundus optic cup and optic disc target data in step S400. Figure 2 As shown, step S400 includes:

[0064] S410, obtaining an image to be detected.

[0065] S420, preprocessing the image to be detected.

[0066] The acquired images to be detected are preprocessed, such as image denoising, enhancement, normalization and other operations. Preprocessing can improve the quality of the image, reduce noise interference, and enhance image features, so that the model can more accurately identify and analyze the target during the inference calculation process, thereby improving the detection accuracy.

[0067] S430: input the preprocessed image data into the model deployed on the lightweight mobile terminal for inference calculation to obtain the model output result.

[0068] The processed image data is input into the optimized lightweight deep learning model for inference calculation to obtain the output of the model. Efficient inference calculation is performed on the mobile device to ensure real-time and accuracy. The efficient operation of the lightweight model speeds up the processing speed, is suitable for the mobile environment, and improves the practicality and responsiveness of the model.

[0069] S440, decode the model output result and extract the detected target box, category and confidence.

[0070] By decoding the output results, the optic cup and optic disc targets in the image can be clearly identified, providing the specific location, type and detection confidence of the target. By extracting the location information of the target box, that is, the coordinates of the bounding box, the category label and the confidence score, the user can accurately identify the optic cup and optic disc targets in the fundus image. This process provides detailed target positioning and classification information, making the detection results not only spatially accurate, but also clear in category recognition.

[0071] In some embodiments, the above-mentioned mobile wide-angle laser fundus image cup-to-disc ratio detection method based on deep learning also includes the step of identifying and correcting the fundus optic cup and optic disc target data to obtain more accurate cup-to-disc ratio data.

[0072] Figure 3 FIG. 1 is a schematic flow chart of a method for identifying and correcting fundus optic cup and optic disc target data to obtain more accurate cup-to-disc ratio data according to an embodiment of the present invention. Figure 3 As shown, the step includes:

[0073] S510, screening the optic cup and optic disc detection frames to obtain the optic cup and optic disc detection frames with the highest confidence.

[0074] By selecting the detection frame with the highest confidence, the possibility of false positives and missed detections can be reduced, and the reliability of the detection results can be improved. This step ensures that the target data used is the most reliable, thereby improving the accuracy of the final cup-to-disc ratio calculation and the performance of the overall system.

[0075] S520, determining that the optic cup is surrounded by the optic disc in the detection frame.

[0076] Check the selected optic cup and optic disc detection frames to verify whether the optic cup is completely surrounded by the optic disc. If the optic cup is not surrounded by the optic disc, the optic cup detection result is marked as invalid. Ensuring that the optic cup is completely surrounded by the optic disc helps to exclude detection results where the position relationship does not conform to the actual situation and avoid calculation deviations caused by incorrect target position. This step improves the accuracy of the cup-to-disc ratio data and ensures the rationality of the actual detection results.

[0077] S530, determining whether the sizes of the optic cup and the optic disc meet the requirements.

[0078] Verify that the cup and disc sizes are within a reasonable physiological range. If the size does not meet the expected standard, it will be marked as an invalid test. By ensuring that the cup and disc sizes meet reasonable physiological sizes, detection errors due to abnormal sizes can be avoided. This step ensures the physiological rationality of the target data, supports more accurate cup-to-disc ratio calculations and effective medical analysis.

[0079] S540, output the coordinates and size information of the optic cup and optic disc.

[0080] According to the results of the above screening and verification steps, the optimized optic cup and optic disc test data are output. If both the optic cup and optic disc are effectively detected, their coordinates and size information are provided, otherwise, an invalid detection prompt is output. By outputting the optimization results, accurate optic cup and optic disc coordinates and size information are provided to support subsequent clinical evaluation and analysis. If the test is invalid, timely prompts can avoid misleading the user. This process improves the practicality and reliability of the system, making the final test results more accurate and useful.

[0081] By identifying and correcting the fundus optic cup and optic disc target data, the accuracy of the cup-to-disc ratio data is significantly improved. By screening and verifying the test results, it is ensured that only high-confidence target boxes are used, which reduces the possibility of false detection. Ensuring that the optic cup is surrounded by the optic disc and meets the size requirements helps to accurately calculate the cup-to-disc ratio, thereby providing more reliable test results. This refined processing is crucial for cup-to-disc ratio measurement in clinical diagnosis and telemedicine, which can support more accurate diagnosis and improve the overall reliability and practicality of the detection system.

[0082] According to the above embodiments, the present invention effectively increases the diversity of training samples and improves the generalization and robustness of the model by performing data enhancement on the fundus image dataset. At the same time, model pruning, quantization and optimization processing further reduce the computational complexity and storage requirements of the model, so that it can be efficiently deployed on mobile devices. By accurately obtaining the coordinates, width and height of the center point of the bounding box of the fundus optic cup and optic disc, and preprocessing and decoding the results of the image to be detected, accurate and efficient detection of the optic cup and optic disc target is achieved. In addition, by identifying the correction step, the accuracy of the cup-to-disc ratio data is ensured, which is suitable for clinical and telemedicine applications.

[0083] The following is a specific implementation process to further explain the mobile wide-angle laser fundus image cup-to-disc ratio detection method based on deep learning of the present invention.

[0084] Fundus image data is collected by using 532nm wavelength laser and 785nm wavelength laser and wide-angle lens optical design. The collected fundus image data is randomly mosaic transformed, moved, shrunk, rotated, cropped, horizontally and vertically flipped, and enhanced in hue, saturation and value to expand the training samples. The fundus image data after data enhancement is used as a training sample, and the fundus image optic cup and optic disc target detection training is performed using a deep learning network structure to obtain a high-precision optic cup and optic disc target detection model. Among the many deep learning target detection algorithms and network structures currently available, based on the real-time requirements of mobile terminal applications, the YOLO framework network is finally selected in the embodiment of the present invention. The core mechanism of YOLO is to apply multi-scale grids on the same image, and each grid generates three predictions based on predefined anchor points. During the training process, only those grids whose anchor points are close to the target bounding box size are retained to participate in forward propagation and back propagation. In addition, in order to increase the number of positive samples, each target grid selects two adjacent grids and assigns the same target label, thereby increasing the probability of correctly predicting the bounding box. Specifically, the training of fundus image optic cup and optic disc target detection on the fundus image dataset through the YOLO framework network includes the following process:

[0085] The final output at each grid scale is a matrix Where N B is the batch size, N A is the number of anchor points for each grid, are the number of grids on the y and x axes, respectively, C is the number of categories, N D is the rotation degree. Given H and W as the height and width of the input image, the horizontal rectangle predicted by the jth anchor point in the kth row and lth column grid of the i-th batch index image can be expressed as P ,y p , the width of the bounding box w P and height h p To express.

[0086]

[0087]

[0088]

[0089]

[0090] In the above formula, σ represents the sigmoid function σ(x) = 1 / (1+exp(-x)), are the width and height of the jth anchor point respectively. Let IoU represent the intersection of unions, then the bounding box loss value of each grid is:

[0091]

[0092] L_box is all L_boxes that are not empty ijkl The average value of Indicates whether there is an object in the current grid.

[0093]

[0094] The target loss value is the binary cross entropy between P and T, as shown below:

[0095]

[0096] For each target label, there is a c t Indicates the category index of the optic cup / disc. Assume there are n t A target frame, Written as the following one-hot label matrix form, it improves model performance by providing the model with more information about the categorical variables.

[0097]

[0098] where i∈[0,n t -1],j∈[0,N C -1]. The corresponding selected output can be expressed as: Each row represents The 5th to (Nth C -1) item. The classification loss value is:

[0099]

[0100] Similarly, to predict the rotation θ, a one-hot label matrix is ​​introduced As shown below

[0101]

[0102] where i∈[0,n t -1],j∈[0,N D -1]. is N selected from P C to (N D -1) The corresponding row of the column. The rotation angle loss is:

[0103]

[0104] The total loss value on the entire grid scale is the weighted sum of these four losses. sgrid scale, let L_box(i), L_obj(i), L_cls(i), L_θ(i) represent the loss value in the i-th space, and the total loss is:

[0105]

[0106] Where ξ represents the weight of the object loss at different grid scales, γ is used to balance the four loss values, γ box , γ obj , γ cls , γ θ They are weight coefficients of four losses, which are 0.3, 0.25, 0.25, and 0.2 respectively in the embodiment of the present invention.

[0107] During the training process, the model optimizes the network according to the total loss error of the output, and saves the final fitted weight model file. Each target object contains horizontal and rotated bounding box labels. All images in the dataset are in RGB format, with pixel values ​​ranging from 0 to 255. The size of the input data is 640×640. For each sample, it is first normalized by 255, and then the data is further standardized by the minimum-maximum normalization method to ensure the stability and accuracy of the model training.

[0108] Export the trained model results to TorchScript format, and convert the TorchScript model to ONNX format. Through this conversion process, the compatibility of the model on different platforms and hardware frameworks is achieved. The converted ONNX model is lightweight, including model pruning, quantization, and optimization. Among them, model pruning is to remove redundant neurons or convolution kernels to reduce the model size and computational complexity. Model quantization is to convert the model weights and activations from 32-bit floating point numbers to 8-bit integers to reduce the model's storage requirements and computational complexity. After lightweight processing of the ONNX model, install and configure the ONNX Runtime library on the mobile device to ensure that it is compatible with the reasoning requirements of the YOLO model. Use C++ programming language to write loading code, and load the YOLO model in ONNX format into memory by calling the API function of ONNX Runtime. The loading code includes initializing the ONNX Runtime environment and session, and specifying the path of the model file. After obtaining the image to be detected, convert the image pre-export to a format that meets the input requirements of the YOLO model, including adjusting the order of RGB channels, normalizing, and resizing to the model input size. Store the preprocessed image data as a floating point array and prepare it for input into the model for inference. Create an ONNX Runtime tensor and pass the preprocessed image data as an input tensor to the ONNX Runtime for inference calculation. Call the inference function of the ONNX Runtime to perform the forward inference calculation of the YOLO model and obtain the model output results. Decode the output results of the model to extract the detected target boxes, categories, and confidences. Perform further post-processing on the model output results, including non-maximum suppression (NMS) operations to remove redundant detection boxes. Finally, format the decoded optic cup and optic disc target detection results for subsequent processing or display.

[0109] After obtaining the fundus optic cup and optic disc target data output by the model, further identification and correction are performed. First, the optic cup and optic disc detection frames are screened and sorted by confidence, and the one with the highest confidence is selected. The encirclement relationship of the selected optic cup and optic disc detection frames is judged to ensure that the optic cup is surrounded by the optic disc. If it is not surrounded, the optic cup detection is marked as invalid. In addition, the size rationality of the selected optic cup and optic disc detection frames is judged to ensure that the size of the optic cup and optic disc meets the reasonable physiological size. If not, the optic cup detection is marked as invalid. Finally, if both the optic cup and optic disc are valid detections, their coordinates and size information are output, otherwise an invalid detection prompt is output.

[0110] Fig. 4 is a schematic diagram of cup-to-disc ratio detection results according to an embodiment of the present invention. Fig. 4(a), Fig. 4(b), Fig. 4(c) and Fig. 4(d) are example diagrams of cup-to-disc ratio detection results under different fundus images. Figure 5A schematic diagram of the PR accuracy results of cup-to-disc ratio detection according to an embodiment of the present invention is shown. Figure 6 It shows a schematic diagram of the accuracy result of cup-to-disc ratio detection F1 according to an embodiment of the present invention. Figure 5 and Figure 6 As shown, the network learning and preservation weight model is used to detect the optic cup and optic disc of the fundus image to obtain the cup-to-disc ratio detection result. The above processing flow is implemented on a mobile terminal based on NVIDIA TX2. After testing, when processing an image with a resolution of 2048*2048, the inference time for each image is about 0.5s. Therefore, the technical solution of the present invention has high timeliness and accuracy for the cup-to-disc ratio detection of mobile wide-angle laser fundus images.

[0111] The embodiment of the present invention also provides a fundus image cup-to-disc ratio detection system based on a mobile terminal wide-angle laser fundus image cup-to-disc ratio detection method based on deep learning. Figure 7 As shown, the fundus image cup-to-disc ratio detection system of the mobile wide-angle laser fundus image cup-to-disc ratio detection method based on deep learning includes a training data enhancement module 101, a basic network training module 102, a mobile network integration module 103 and a cup-to-disc ratio measurement module 104.

[0112] The training data enhancement module 101 is configured to perform data enhancement on the fundus image dataset to obtain an enhanced fundus image dataset.

[0113] The basic network training module 102 is configured to perform fundus image optic cup and optic disc target detection training on the fundus image dataset through a deep learning network structure to obtain a high-precision optic cup and optic disc target detection model.

[0114] The mobile terminal network integration module 103 is configured to perform recognition training on the fundus image dataset based on a high-precision optic cup and optic disc target detection model through a lightweight deep learning network structure to obtain a lightweight mobile terminal deployed model.

[0115] The cup-to-disc ratio measurement module 104 is configured to identify and correct the fundus optic cup and optic disc target data to obtain more accurate cup-to-disc ratio data.

[0116] In the fundus image cup-to-disc ratio detection system of the above-mentioned mobile wide-angle laser fundus image cup-to-disc ratio detection method based on deep learning, the specific implementation method of obtaining the fundus optic cup and optic disc target data refers to the relevant content of the embodiments in the above-mentioned method, and will not be elaborated here.

[0117] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting cup-to-disc ratio of mobile wide-angle laser fundus images based on deep learning, characterized in that: The method comprises: Acquire a fundus image dataset; Performing fundus image optic cup and optic disc target detection training on the fundus image dataset through a deep learning network structure to obtain a high-precision optic cup and optic disc target detection model; Based on the high-precision optic cup and optic disc target detection model, the fundus image dataset is trained for recognition through a lightweight deep learning network structure to obtain a lightweight mobile terminal deployment model; The fundus image is processed by the model deployed on the lightweight mobile terminal to obtain fundus optic cup and optic disc target data.

2. The method for detecting cup-to-disc ratio of mobile wide-angle laser fundus images based on deep learning according to claim 1, characterized in that: The method further includes performing data enhancement on the fundus image dataset to obtain an enhanced fundus image dataset.

3. The method for detecting cup-to-disc ratio of mobile wide-angle laser fundus images based on deep learning according to claim 2, characterized in that: The data enhancement of the fundus image dataset to obtain the enhanced fundus image dataset includes mosaic transformation, movement, contraction, rotation, cropping, horizontal and vertical flipping, and hue, saturation and value enhancement of the fundus image dataset.

4. The method for detecting cup-to-disc ratio of mobile wide-angle laser fundus images based on deep learning according to claim 1, characterized in that: The method also includes performing model pruning, quantization and optimization processing on the model deployed on the lightweight mobile terminal.

5. The method for detecting cup-to-disc ratio of mobile wide-angle laser fundus images based on deep learning according to claim 1, characterized in that: The fundus optic cup and optic disc target data include the coordinates, width and height of the center point of the bounding box, which are obtained by the following formula: in, Respectively represent the number of grids on the y and x axes; H and W are the height and width of the input fundus image, are the width and height of the jth anchor point respectively; the bounding box predicted by the jth anchor point in the kth row and lth column grid of the i-th batch index image can be expressed as the center point coordinate x P ,y p , the width of the bounding box w P and height h p To express.

6. The method for detecting cup-to-disc ratio of mobile wide-angle laser fundus images based on deep learning according to claim 1, characterized in that: The method of processing the fundus image by the model deployed by the lightweight mobile terminal to obtain the fundus optic cup and optic disc target data includes: Acquire the image to be detected; Preprocessing the image to be detected; Inputting the preprocessed image data into the model deployed on the lightweight mobile terminal for inference calculation to obtain the model output result; Decode the model output and extract the detected object boxes, categories, and confidences.

7. The method for detecting cup-to-disc ratio of mobile wide-angle laser fundus images based on deep learning according to claim 1, characterized in that: The method further includes identifying and correcting the fundus optic cup and the optic disc target data to obtain more accurate cup-to-disc ratio data.

8. The method for detecting cup-to-disc ratio of mobile wide-angle laser fundus images based on deep learning according to claim 7, characterized in that: The identifying and correcting the fundus optic cup and the optic disc target data comprises: Screening the optic cup and optic disc detection frames to obtain the optic cup and optic disc detection frames with the highest confidence; Determining that the optic cup in the detection frame is surrounded by the optic disc; Determining that the sizes of the optic cup and the optic disc meet the requirements; The coordinates and size information of the optic cup and optic disc are output.

9. A fundus image cup-to-disc ratio detection system using the mobile terminal wide-angle laser fundus image cup-to-disc ratio detection method based on deep learning as described in claims 1-8, characterized in that: The system comprises: A basic network training module is configured to perform fundus image optic cup and optic disc target detection training on the fundus image dataset through a deep learning network structure to obtain a high-precision optic cup and optic disc target detection model; The mobile terminal network integration module is configured to perform recognition training on the fundus image data set based on the high-precision optic cup and optic disc target detection model through a lightweight deep learning network structure to obtain a lightweight mobile terminal deployed model.

10. The fundus image cup-to-disc ratio detection system according to claim 9, characterized in that: The system further comprises: A training data enhancement module is configured to perform data enhancement on the fundus image dataset to obtain an enhanced fundus image dataset; The cup-to-disc ratio measurement module is configured to identify and correct the fundus optic cup and the optic disc target data to obtain more accurate cup-to-disc ratio data.

Images