Fundus image evaluation method based on fusion of digital twin and optic nerve imaging

Abstract

The present invention discloses a fundus image evaluation method based on the fusion of digital twin and optic nerve imaging, which relates to the field of image processing technology, and comprises the following steps: constructing a fundus digital twin model; dividing the acquired OCT and OCTA fundus images into a number of grid areas and performing pixel fusion to obtain a fundus fusion image; inputting the fundus fusion image into the fundus digital twin model for superposition and feature segmentation of key areas, and calculating the image quality coefficients of a number of key area grid images and ordinary area grid images based on the image quality evaluation index by the fundus fusion key area detection large model, and calculating the quality deviation coefficient between the key area image and the ordinary area image; comprehensively evaluating the fundus fusion image based on the quality deviation coefficient and a preset quality threshold, thereby solving the problems of uneven quality of fundus images, insufficient accuracy of key area detection and insufficient evaluation of the overall image quality.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a fundus image evaluation method based on the fusion of digital twins and optic nerve imaging. Background Art

[0002] Fundus images are acquired using dedicated fundus cameras or optical coherence tomography (OCT) technology, primarily including key anatomical structures such as the retina, optic disc, macula, and retinal blood vessels. The optic disc, the most prominent bright yellow area in fundus images, is located at the source of fundus blood vessels and retinal nerves and is often used to assist in the diagnosis of diseases such as glaucoma. The macula, located in the center of the retina and the area with the highest visual acuity, is the area with the highest visual acuity. Diseases of the macula can lead to decreased vision and other visual impairments and is often used to diagnose age-related macular degeneration (AMD). Retinal blood vessels, on the other hand, help identify signs of cardiovascular disease.

[0003] For example, the invention patent with announcement number: CN118196218B discloses a fundus image processing method, device and equipment, which performs a first analysis and processing on the fundus image to calculate and determine the clarity of the fundus image; performs a second analysis and processing on the fundus image to calculate and determine the chromaticity distribution of the fundus image; obtains a clear target scene image; performs style transfer processing on the target scene image based on the fundus image to form a blurred version of the target scene image; and performs processing based on the clarity, chromaticity distribution and blurred version of the scene image to form a simulated image for simulating the imaging effect when the patient looks at the target scene.

[0004] During fundus image acquisition, image quality may degrade due to factors such as uneven illumination, motion blur, and inaccurate focus, affecting the visibility and clarity of key structures such as the optic disc, macula, and retinal vessels. Furthermore, fundus image quality can vary between different regions, particularly between critical and non-critical areas. The present invention proposes a solution to these problems. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a fundus image evaluation method based on the fusion of digital twins and optic nerve imaging. The fundus image evaluation method based on the fusion of digital twins and optic nerve imaging is used to solve the current problems of fundus images due to uneven quality, accuracy of key area detection and insufficient assessment of the overall image quality.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] The fundus image evaluation method based on the fusion of digital twins and optic nerve imaging includes the following steps: dividing the acquired OCT and OCTA fundus images into several grid areas, calculating the pixel values ​​of the OCT and OCTA fundus images in each grid area and fusing them to obtain the fundus fusion image and perform preprocessing;

[0008] The fundus fusion image is input into the fundus digital twin model for superposition and feature segmentation of key areas to obtain a fundus fusion image dataset, and the fundus fusion image dataset is input into a preset target detection large model for training to obtain a fundus fusion key area detection large model; based on the fundus fusion key area detection large model, the key areas of the fundus fusion image are detected, and the fundus fusion image is divided into key areas and ordinary areas, and the key areas and ordinary areas are divided into several key area grid images and ordinary area grid images; based on the image quality evaluation index, the image quality coefficients of the several key area grid images and ordinary area grid images are calculated, and the quality deviation coefficient between the key area image and the ordinary area image is calculated; based on the quality deviation coefficient between the key area image and the ordinary area image and the preset quality threshold, the fundus fusion image is comprehensively evaluated.

[0009] In a preferred embodiment, the acquired OCT and OCTA fundus images are divided into several grid areas, the pixel values ​​of the OCT and OCTA fundus images in each grid area are calculated and fused to obtain a fused fundus image and preprocessed, specifically as follows: the acquired OCT and OCTA fundus images are pixel-aligned using a registration method to obtain aligned OCT and OCTA fundus images; the aligned OCT and OCTA fundus images are divided into several grid areas in the same manner to obtain several aligned OCT and OCTA fundus grid images; the pixel values ​​of the several aligned OCT and OCTA fundus grid images are respectively calculated; the pixel values ​​of the several aligned OCT and OCTA fundus grid images are weightedly fused to obtain fused pixel values ​​of the several grid areas; and the fused pixel values ​​of each grid area are spliced ​​grid by grid to obtain a fused fundus image.

[0010] In a preferred embodiment, the fused pixel values ​​of each grid area are spliced ​​grid by grid to obtain a fundus fusion image. Specifically, for each row of the grid area, the fused pixel values ​​of all grids in the row are horizontally spliced ​​in order from left to right to obtain a complete fusion area of ​​each row; the complete fusion areas of each row are vertically superimposed to obtain a fundus fusion image.

[0011] In a preferred embodiment, the fundus fusion image is input into the fundus digital twin model for superposition and feature segmentation of key areas to obtain a fundus fusion image dataset, and the fundus fusion image dataset is input into a preset target detection large model for training to obtain a fundus fusion key area detection large model, specifically: the fundus fusion image is combined with the patient's actual image to obtain the patient's fundus twin virtual body; the geometric functional feature database of the patient's fundus twin virtual body is used to match the key areas in the fundus fusion image for superposition of key areas, and the key areas refer to the macular area, optic nerve head area and blood vessel dense area; a deep learning segmentation algorithm is used in the fundus digital twin model to perform fine-grained segmentation of the key areas to obtain precise bounding boxes of the key areas, and the precise bounding boxes of the key areas are saved as a preset training set format to obtain a fundus fusion image dataset; the fundus fusion image dataset is divided into a training set and a test set according to a preset ratio, and the training set is input into the target detection large model for training the large model to obtain a trained target detection large model;

[0012] In a preferred embodiment, the image quality coefficients of the several key area grid images and the ordinary area grid images are calculated based on the image quality evaluation index, specifically: the clarity evaluation values, edge clarity evaluation values, and brightness evaluation values ​​of the several key area grid images and the ordinary area grid images are calculated respectively based on the image quality evaluation index; the clarity evaluation values, edge clarity evaluation values, and brightness evaluation values ​​are standardized, and weighted summation is performed to obtain the image quality coefficients of the several key area grid images and the ordinary area grid images.

[0013] In a preferred embodiment, the calculation of the quality deviation coefficient between the key area image and the ordinary area image is specifically as follows: based on the image quality coefficients of several key area grid images and ordinary area grid images, a geometric mean method is used to calculate the first quality coefficient of the key area image and the first quality coefficient of the ordinary area; based on the first quality coefficient of the key area image and the first quality coefficient of the ordinary area, the average quality coefficient of the key area image and the average quality coefficient of the ordinary area image are calculated; the image quality similarity index of the key area image and the ordinary area image is calculated, and based on the image quality similarity index, the quality deviation coefficient between the key area image and the ordinary area image is obtained.

[0014] In a preferred embodiment, the fundus fusion image is comprehensively evaluated based on the quality deviation coefficient between the key area image and the ordinary area image and a preset quality threshold, specifically: when the quality deviation coefficient between the key area image and the ordinary area image is less than the preset quality threshold, the image quality is qualified; when the quality deviation coefficient between the key area image and the ordinary area image is greater than the preset quality threshold, the image quality is unqualified.

[0015] The technical effects and advantages of the fundus image evaluation method based on the fusion of digital twin and optic nerve imaging of the present invention are as follows:

[0016] 1. The present invention integrates the OCT and OCTA fundus images and pathological data collected from patients multiple times, and combines them with a deep learning large model to construct a fundus digital twin model, which has significant advantages. First, the model realizes personalized dynamic modeling of patients, can accurately simulate the anatomical structure and lesion characteristics of the retina, and provide individualized diagnosis and treatment support; second, digital twin technology generates a variety of simulation scenarios and data enhancements, which helps to improve the generalization ability of the model; third, the twin realizes automatic labeling and feature segmentation during data processing, significantly improving labeling efficiency and accuracy and reducing manual intervention; in addition, the model optimizes diagnostic and prediction results through a real-time feedback mechanism, supporting the dynamic evaluation of complex lesions; finally, the fundus digital twin model not only improves the scientificity and reliability of clinical decision-making, but also provides simulation support for disease progression monitoring and treatment plan design. It is an important innovative tool in the field of intelligent diagnosis and treatment in ophthalmology.

[0017] 2. This method fuses OCT and OCTA images to generate high-quality fused fundus images, further dividing them into grid regions to provide richer and more accurate detailed information in each region. Furthermore, the annotation of key regions and the training of a large detection model enhance the large model's ability to identify fundus lesions. During the detection phase, this method automatically identifies and distinguishes key regions from common regions in the image, processing them by region for more targeted analysis. By finely dividing the grid images of key regions into grid images of common regions, the large model's focus on key regions is enhanced, facilitating early detection of lesions and high-precision lesion localization. Secondly, image quality coefficients are calculated for the grid images of key and common regions, accurately quantifying image quality in different regions and helping to identify the clarity and detail retention of key regions. Further calculation of the quality deviation coefficient allows analysis of quality consistency between key and common regions, helping to ensure higher and more reliable imaging quality in important areas. Combining the image quality coefficient and deviation coefficient with a pre-set quality threshold for comprehensive evaluation, fundus fused images with satisfactory overall quality and good regional quality consistency can be effectively selected. This evaluation method not only enhances the scientific nature and reliability of image quality control but also provides a stable and clear image foundation for subsequent disease diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a structural schematic diagram of the fundus image evaluation method based on the fusion of digital twins and optic nerve imaging of the present invention. DETAILED DESCRIPTION

[0019] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0020] Example 1, Figure 1 The present invention provides a fundus image evaluation method based on the fusion of digital twins and optic nerve imaging, which includes the following steps:

[0021] S1, dividing the acquired OCT and OCTA fundus images into several grid areas, calculating the pixel values ​​of the OCT and OCTA fundus images in each grid area and fusing them to obtain the fundus fusion image and perform preprocessing;

[0022] In this example, the acquired OCT and OCTA fundus images are divided into several grid areas. The pixel values ​​of the OCT and OCTA fundus images in each grid area are calculated and fused to obtain a fused fundus image. The image is then preprocessed as follows:

[0023] The acquired OCT and OCTA fundus images are pixel-aligned using a registration method to obtain aligned OCT and OCTA fundus images;

[0024] Dividing the aligned OCT and OCTA fundus images into a plurality of grid areas in the same manner to obtain a plurality of aligned OCT and OCTA fundus grid images;

[0025] Calculating pixel values ​​of the plurality of aligned OCT and OCTA fundus grid images respectively;

[0026] performing weighted fusion on the pixel values ​​of the plurality of aligned OCT and OCTA fundus grid images to obtain fused pixel values ​​of the plurality of grid areas;

[0027] The fused pixel values ​​of each grid area are spliced ​​grid by grid to obtain the fundus fusion image.

[0028] In this example, the acquired OCT and OCTA fundus images are pixel-aligned using a registration method to obtain aligned OCT and OCTA fundus images, specifically:

[0029] Convert the OCT and OCTA images into grayscale images, and use the SURF algorithm to extract key feature points in the grayscale images;

[0030] Use the KNN algorithm to perform feature matching on key feature points in the grayscale image and find similar point pairs in the OCT image and OCTA image;

[0031] Estimate the transformation matrix from OCT image to OCTA image based on similar point pairs in OCT image and OCTA image;

[0032] The transformation matrix from the OCT image to the OCTA image is applied to the OCT and OCTA images for alignment to obtain aligned OCT and OCTA fundus images.

[0033] It should be noted that the SURF algorithm is an efficient and robust feature point detection and description method suitable for applications with high real-time requirements. SURF detects feature points through the Hessian matrix, and its calculation speed is faster than SIFT. It also has scale and rotation invariance, making it highly robust to image rotation, scaling, and partial occlusion. SURF uses Haar wavelet responses for feature description, which speeds up the descriptor calculation process and improves matching accuracy. In addition, the feature points calculated by the SURF algorithm are relatively few and representative, making feature matching efficient. It is particularly suitable for medical image processing tasks that require fast and accurate registration, such as fundus OCT and OCTA image registration. Overall, SURF achieves a good balance between efficiency and accuracy.

[0034] The KNN (K-nearest neighbor) algorithm offers high accuracy and simplicity in image feature matching. It is an effective method for finding similar point pairs based on feature distance. For feature points extracted from OCT and OCTA images, KNN can quickly find multiple nearest neighbors (typically the first two neighbors) for each feature point in the other image, thereby improving matching accuracy. Furthermore, by using a distance ratio test in matching, incorrect matching points can be effectively filtered out, thereby improving the reliability of feature matching. KNN boasts high computational efficiency in multidimensional feature spaces and is well-suited for use with large amounts of image data. Therefore, in medical image processing and multimodal image registration, KNN matching provides a solid foundation for subsequent image alignment.

[0035] In this example, the pixel values ​​of the aligned OCT and OCTA fundus grid images are calculated respectively. The specific formula is:

[0036]

[0037] in, and are the pixel values ​​of the aligned OCT and OCTA fundus grid images, and are the pixel values ​​of the position (x, y) in a certain grid area of ​​the OCT and OCTA fundus grid images, S is the side length of the grid area, and x and y are the horizontal and vertical coordinates of the position in a certain grid area, respectively.

[0038] In this example, the fused pixel values ​​of each grid area are spliced ​​grid by grid to obtain the fundus fusion image, specifically:

[0039] For each row of the grid area, horizontally splice the fused pixel values ​​of all grids in the row from left to right to obtain the complete fused area of ​​each row;

[0040] The complete fusion areas of each row are vertically superimposed to obtain a fundus fusion image.

[0041] It should be noted that grid-by-grid splicing of the fused pixel values ​​of each grid area can effectively integrate the information in the OCT and OCTA images, thereby generating a fundus fusion image with rich information. This grid-by-grid splicing method first horizontally splices each row to merge the pixel information within the same row, ensuring that the spatial information of each grid area is accurately transmitted. Subsequently, the fusion areas of each row are vertically superimposed to maintain the overall structure and spatial continuity of the image. This method not only retains the details and important features in images of different modalities, but also ensures a seamless transition of the spliced ​​images, avoiding the boundary distortion problem in traditional splicing methods. The fundus fusion image obtained in this way has higher resolution and more diagnostic information, providing more powerful image support for the early detection and accurate diagnosis of fundus diseases.

[0042] In this example, weighted fusion is performed on the pixel values ​​of the aligned OCT and OCTA fundus grid images to obtain fused pixel values ​​of several grid areas, specifically:

[0043] The selection of weights in the weighted fusion is based on the doctor's diagnostic experience data and historical fundus images through machine learning training. For example, the blood vessel-related areas have higher weights in the OCTA fundus grid image, and the tissue thickness-related areas have higher weights in the OCT fundus grid image.

[0044] S2, detecting key areas of the fundus fusion image based on a preset fundus fusion key area detection model, dividing the fundus fusion image into key areas and common areas, and further dividing the key areas and common areas into a plurality of key area grid images and common area grid images;

[0045] In this example, the preset fundus fusion key area detection model is as follows:

[0046] Combining the fundus fusion image with the patient's actual image to obtain the patient's fundus twin virtual body;

[0047] Using a geometric functional feature database of the patient's fundus twin virtual body, key areas in the fundus fusion image are matched and superimposed, where the key areas are the macula, the optic nerve head, and the vascular dense area;

[0048] Using a deep learning segmentation algorithm in the fundus digital twin model to perform fine-grained segmentation on the key areas, obtaining precise bounding boxes of the key areas, and saving the precise bounding boxes of the key areas in a preset training set format to obtain a fundus fusion image dataset;

[0049] The fundus fusion image dataset is divided into a training set and a test set according to a preset ratio, and the training set is input into the target detection large model for training the large model to obtain the trained target detection large model;

[0050] In this example, the fundus fusion image is combined with the patient's actual image to obtain the patient's fundus twin virtual body, specifically:

[0051] The patient's fundus twin is a digital simulation model constructed based on the patient's actual fundus images (such as OCT and OCTA fusion images) as well as multimodal data such as pathology, anatomy, and physiology. This virtual model can truly reflect the patient's fundus health and is used for dynamic monitoring, prediction, diagnosis, and treatment optimization. Specifically, the patient's fundus twin includes the following key components:

[0052] Patient fundus images (OCT and OCTA fused images), fundus disease data (such as diabetic retinopathy, macular degeneration, etc.), and anatomical data involving the structure of the eyeball.

[0053] In this example, the key areas of the fundus fusion image are the macula, optic nerve head, and vascular dense areas, specifically:

[0054] The macula is located in the central part of the retina. It is the area with the highest visual acuity and is responsible for the visual perception of details. The macula is crucial for the clarity and precision of vision. Diseases such as macular degeneration can lead to severe vision loss. Therefore, the health of the macula is very important for ophthalmological diagnosis.

[0055] Optic disc: Located on the nasal side of the retina, the optic disc is where the optic nerve fibers transmit signals from the eye. Often referred to as the "optic disc," the optic disc is crucial for the eye's neurological function. Diseases such as glaucoma can damage or atrophy the optic disc, affecting vision.

[0056] Densely vascularized areas: These areas are typically found in the retina and contain a dense concentration of blood vessels, primarily the retinal vascular plexus and central retinal artery. These densely vascularized areas provide blood supply to the fundus, and any vascular disease (such as diabetic retinopathy and retinal hemorrhage) may manifest in these areas, making monitoring them crucial.

[0057] In this example, the fundus fusion image dataset is divided into a training set and a test set according to a preset ratio, and the training set is input into the target detection large model for training the large model to obtain the trained target detection large model, specifically:

[0058] The target detection large model can select target detection algorithms such as Faster R-CNN, YOLO and SSD, and can add an attention mechanism to improve the detection accuracy of the large model.

[0059] It's important to note that labeling key regions in fused fundus images and inputting them into a pre-defined large-scale object detection model for training offers significant advantages. First, the large-scale object detection model can automatically identify and locate key regions in fundus images, such as the macula, optic nerve head, and vascular-dense areas. These regions are crucial for diagnosing ocular diseases. Through dataset annotation, the large-scale model learns the spatial characteristics of key regions, enhancing its adaptability to a variety of fundus images. Second, the large-scale object detection model exhibits strong robustness and accuracy, capable of handling differences between modalities (such as OCT and OCTA) and providing consistent results. The trained large-scale detection model enables efficient and automated image analysis, reducing manual labeling workload and improving diagnostic efficiency and accuracy. Furthermore, the trained large-scale model can rapidly locate and analyze key regions in large-scale fundus images, providing ophthalmologists with more accurate and rapid decision support.

[0060] In this example, the key areas of the fundus fusion image are detected based on the fundus fusion key area detection model, and the fundus fusion image is divided into key areas and common areas, specifically:

[0061] A large-scale model for fundus fusion key region detection can accurately detect key regions in fundus images (such as the macula, optic nerve head, and densely vascularized areas) and divide the images into key and common regions, offering significant advantages. First, this method automatically identifies key regions in fundus images, avoiding the subjectivity and inefficiency of manual labeling and improving processing speed and accuracy. Second, by further dividing key and common regions into grid images, more detailed local analysis is possible, allowing every detail in the image to be fully evaluated. This is of great significance for the early detection, monitoring, and evaluation of treatment efficacy of ophthalmic diseases. Furthermore, refined region segmentation helps improve the accuracy and stability of image quality assessment and enhances the adaptability of the large-scale model to fundus images from different patients and modalities (such as OCT and OCTA). Therefore, the entire method improves the automation level of fundus image analysis and diagnostic accuracy.

[0062] S3, calculating image quality coefficients of the plurality of key area grid images and the common area grid images based on the image quality evaluation index, and calculating a quality deviation coefficient between the key area images and the common area images;

[0063] In this example, the image quality coefficients of the plurality of key area grid images and the common area grid images are calculated based on the image quality evaluation index, specifically:

[0064] Calculating the clarity evaluation values, edge clarity evaluation values, and brightness evaluation values ​​of the plurality of key area grid images and the common area grid images based on the image quality evaluation index;

[0065] The clarity evaluation value, edge clarity evaluation value, and brightness evaluation value are standardized, and weighted summed to obtain image quality coefficients of a plurality of key area grid images and common area grid images.

[0066] In this example, the clarity evaluation values, edge clarity evaluation values, and brightness evaluation values ​​of the plurality of key area grid images and the common area grid images are calculated based on the image quality evaluation index. The specific formula is:

[0067]

[0068]

[0069]

[0070] in, is the clarity evaluation value of the grid area, and are the gradient values ​​of the key area grid image and the common area grid image in the horizontal and vertical directions respectively, N is the total pixels in the grid area, is the edge clarity evaluation value of the grid area, is the Laplace response value of the key area grid image and the common area grid image at position (x, y), is the brightness evaluation value of the grid area, is the grayscale value of the i-th pixel, is the average brightness value of the grid area, is the brightness standard deviation of the grid area, A small positive number.

[0071] In this example, the clarity evaluation value, edge clarity evaluation value, and brightness evaluation value are specifically:

[0072] Clarity evaluation value: The clarity evaluation value measures the overall clarity of an image and reflects the sharpness of image details based on the amount of local gradient change. Images with higher clarity generally display more delicate structures and richer detail information.

[0073] Edge clarity evaluation value: The edge clarity evaluation value is used to measure the clarity of the edge area in the image. It mainly extracts important edge information in the image through the edge detection algorithm and reflects the accuracy of the image boundary.

[0074] Brightness evaluation value: reflects the overall brightness level of the image, mainly measuring the brightness and darkness of the image. It is usually obtained by calculating the average brightness or brightness distribution of the image.

[0075] It should be noted that the clarity, edge clarity, and brightness metrics are key indicators in image quality assessment, each with its own unique advantages. The clarity quantifies the image's ability to render detail, which is crucial for identifying subtle lesions or structural changes. It helps accurately capture details in fundus images, particularly in fundus images, and improves diagnostic accuracy. The edge clarity focuses on the sharpness of image edges. In fundus images, it effectively identifies the edges of key areas such as the macula and optic nerve head, helping doctors detect the specific location of blood vessels and lesions. The brightness ensures a balanced distribution of light and dark in the image, avoiding information loss caused by overly bright or dark images and ensuring the visibility of important details. Combining these three metrics allows for a comprehensive assessment of fundus image quality, enabling optimization and analysis from multiple dimensions to enhance image analysis accuracy and diagnostic value. These metrics not only help automated systems improve detection efficiency but also support doctors in making more accurate decisions in actual clinical diagnosis.

[0076] In this example, the quality deviation coefficient between the key area image and the normal area image is calculated as follows:

[0077] Based on the image quality coefficients of the plurality of key area grid images and the common area grid images, a geometric mean method is used to calculate the first quality coefficient of the key area image and the first quality coefficient of the common area;

[0078] Based on the first quality coefficient of the key area image and the first quality coefficient of the common area, an average quality coefficient of the key area image and an average quality coefficient of the common area image are calculated;

[0079] An image quality similarity index between the key region image and the common region image is calculated, and a quality deviation coefficient between the key region image and the common region image is obtained based on the image quality similarity index.

[0080] In this example, the first quality coefficient of the key area image and the first quality coefficient of the common area are calculated based on the image quality coefficients of several key area grid images and common area grid images using the geometric mean method. The specific formula is as follows:

[0081]

[0082] in, is the first quality coefficient, is the inverse function of the natural logarithm, is the weight of the i-th grid area, is the image quality coefficient of the i-th grid area, the total number of grids divided in the T key area, is the natural logarithm of the image quality coefficient of the i-th grid area.

[0083] In this example, the image quality similarity index of the key area image and the general area image is calculated, and the specific formula is as follows:

[0084]

[0085] in, is the image quality similarity index between the key area image and the common area image, and are the average quality coefficients of key area images and the average quality coefficients of common area images, and are the variances of the image quality coefficients of the key area grid image and the normal area grid image, are the covariances of the image quality coefficients of the key area grid image and the normal area grid image, respectively.

[0086] It should be noted that through grid-based image quality coefficient calculation, a more detailed quality analysis of key areas and ordinary areas can be performed, avoiding local quality differences that may be masked by the overall evaluation, and improving the accuracy of the evaluation; the use of the geometric mean method to calculate the first quality coefficients of key areas and ordinary areas reduces the impact of extreme values ​​on the overall evaluation, thereby providing more stable and fair evaluation results; by calculating the average quality coefficient, the overall image quality in the area can be comprehensively measured, reflecting the relative quality level between areas, and providing data support for image quality optimization; the image quality similarity index is used to measure the overall consistency between key areas and ordinary areas, and the deviation coefficient is combined to quantitatively describe the quality differences, providing a clear goal for image correction and optimization.

[0087] In this example, the image quality similarity index is specifically:

[0088] The Image Quality Similarity Index (IQSI) effectively assesses image quality and structural consistency by quantifying the similarity between images. Its advantage lies in its ability to objectively and accurately compare differences between different images or image regions, avoiding subjective bias. In fundus image processing, the IQSI helps identify quality consistency between critical and common areas, ensuring that image quality meets diagnostic requirements. This method not only improves the reliability of automated image assessment but also supports the accuracy of image optimization and processing, providing strong data support for clinical decision-making.

[0089] In this example, the quality deviation coefficient between the key area image and the common area image is obtained based on the image quality similarity index, specifically:

[0090]

[0091] in, is the image quality similarity index, is the quality deviation coefficient between the key area image and the normal area image.

[0092] S4, comprehensively evaluating the fundus fusion image based on the image quality coefficient, the quality deviation coefficient and a preset quality threshold.

[0093] In this example, the fundus fusion image is comprehensively evaluated based on the quality deviation coefficient between the key area image and the ordinary area image and a preset quality threshold, specifically:

[0094] When the quality deviation coefficient between the key area image and the general area image is less than the preset quality threshold, the image quality is excellent;

[0095] When the quality deviation coefficient between the key area image and the general area image is greater than a preset quality threshold, the image quality is unqualified.

[0096] It should be noted that the method for comprehensively evaluating fundus fusion images based on image quality coefficients, quality deviation coefficients, and preset quality thresholds offers several advantages. First, by setting reasonable quality coefficient and quality deviation thresholds, image quality assessment can be quantified and standardized, reducing the influence of subjective factors and ensuring the objectivity and consistency of image evaluation. Second, this method effectively distinguishes images of varying quality levels: by setting different conditions (such as quality coefficient and quality deviation thresholds), it can accurately identify images as qualified, acceptable, or unqualified. This graded evaluation helps optimize the image processing process and further correct and optimize low-quality images. Finally, this evaluation method combines quality assessments of both critical and common areas, ensuring that the quality of important areas in fundus images is fully protected. This method is suitable for the high-precision requirements of clinical diagnosis and improves the reliability and practicality of automated systems.

[0097] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by technicians in this field according to actual conditions.

[0098] The above embodiments may be implemented in whole or in part through software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product.

[0099] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0100] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0101] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0102] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A fundus image evaluation method based on the fusion of digital twin and optic nerve imaging, characterized in that: The following steps are involved: The acquired OCT and OCTA fundus images are divided into several grid areas, the pixel values ​​of the OCT and OCTA fundus images in each grid area are calculated and fused to obtain the fundus fusion image and perform preprocessing; Construct a fundus digital twin model, detect the key areas of the fundus fusion image based on the preset fundus fusion key area detection model, and divide the fundus fusion image into several key area grid images and common area grid images; Combining the fundus fusion image with the actual image of the patient to obtain a fundus twin virtual body; Based on the feature database of the fundus twin virtual body, matching and superimposing the key areas in the fundus fusion image; Using a deep learning segmentation algorithm to perform fine-grained segmentation on the key area, generate an accurate bounding box, and obtain a fundus fusion image dataset based on the accurate bounding box; The target detection model is trained and verified based on the fundus fusion image data set to obtain a large model for fundus fusion key area detection; Inputting the fundus fusion image to be predicted into the fundus fusion key area detection large model to obtain the key area of ​​the fundus fusion image; Calculating image quality coefficients of the plurality of key area grid images and common area grid images based on the image quality evaluation index, and calculating quality deviation coefficients between the key area images and the common area images; Calculate the quality deviation coefficient between the key area image and the common area image, specifically: Based on the image quality coefficients of the plurality of key area grid images and the common area grid images, a first quality coefficient of the key area image and a first quality coefficient of the common area are calculated by using a geometric mean method; Based on the first quality coefficient of the key area image and the first quality coefficient of the common area, an average quality coefficient of the key area image and an average quality coefficient of the common area image are obtained by an average calculation method; Calculating an image quality similarity index between the key area image and the common area image, and obtaining a quality deviation coefficient between the key area image and the common area image based on the image quality similarity index; The fundus fusion image is comprehensively evaluated based on the quality deviation coefficient between the key area image and the common area image and a preset quality threshold.

2. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 1 is characterized in that: The acquired OCT and OCTA fundus images are divided into a number of grid areas, the pixel values ​​of the OCT and OCTA fundus images in each grid area are calculated and fused, and the fundus fusion image is obtained and preprocessed, specifically: The acquired OCT and OCTA fundus images are pixel-aligned using a registration method to obtain aligned OCT and OCTA fundus images; Evenly dividing the aligned OCT and OCTA fundus images into a plurality of grid areas to obtain a plurality of aligned OCT and OCTA fundus grid images; Calculating pixel values ​​of the plurality of aligned OCT and OCTA fundus grid images respectively; Performing weighted fusion on the pixel values ​​of the plurality of aligned OCT and OCTA fundus grid images to obtain fused pixel values ​​of a plurality of grid areas; The fused pixel values ​​of each grid area are spliced ​​grid by grid to obtain the fundus fusion image.

3. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 2 is characterized in that: The fusion pixel values ​​of each grid area are spliced ​​grid by grid to obtain the fundus fusion image, specifically: For each row of the grid area, horizontally splice the fused pixel values ​​of all grids in each row from left to right to obtain the complete fused area of ​​each row; The complete fusion area of ​​each row is vertically superimposed to obtain a fundus fusion image.

4. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 3 is characterized in that: The image quality coefficients of the plurality of key area grid images and the common area grid images are calculated based on the image quality evaluation index, specifically: Calculating the clarity evaluation value, edge clarity evaluation value, and brightness evaluation value of the plurality of key area grid images and the common area grid images respectively based on the image quality evaluation index; The clarity evaluation value, edge clarity evaluation value, and brightness evaluation value are standardized, and weighted summed to obtain image quality coefficients of a plurality of key area grid images and common area grid images.

5. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 4 is characterized in that: The fundus fusion image is comprehensively evaluated based on the quality deviation coefficient between the key area image and the common area image and a preset quality threshold, specifically: When the quality deviation coefficient between the key area image and the common area image is less than the preset quality threshold, the image quality is qualified; When the quality deviation coefficient between the key area image and the common area image is greater than a preset quality threshold, the image quality is unqualified.

6. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 5 is characterized in that: The pixel values ​​of the aligned OCT and OCTA fundus grid images are calculated respectively, and the specific formula is: in, and are the pixel values ​​of the aligned OCT and OCTA fundus grid images, and are the pixel values ​​of the OCT and OCTA fundus grid images at positions (x, y) within a certain grid area, S is the side length of the grid area, and x and y are the horizontal and vertical coordinates of the position within a certain grid area, respectively.

7. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 6 is characterized in that: The definition evaluation value, edge definition evaluation value, and brightness evaluation value of the plurality of key area grid images and the common area grid images are respectively calculated based on the image quality evaluation index, and the specific formula is: in, is the clarity evaluation value of the grid area, and are the gradient values ​​of the key area grid image and the common area grid image in the horizontal and vertical directions respectively, N is the total pixels in the grid area, is the edge clarity evaluation value of the grid area, is the Laplace response value of the key area grid image and the common area grid image at position (x, y), is the brightness evaluation value of the grid area, is the gray value of the i-th pixel, is the average brightness value of the grid area, is the brightness standard deviation of the grid area, is a small positive number.

8. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 7 is characterized in that: The image quality similarity index of the key area image and the common area image is calculated, and the specific formula is: in, is the image quality similarity index between the key area image and the common area image, and are the average quality coefficients of key area images and the average quality coefficients of common area images, and are the variances of the image quality coefficients of the key area grid image and the common area grid image, is the covariance of the image quality coefficients of the key area grid image and the common area grid image.

Images