Eye fundus image evaluation method based on digital twinborn and optic nerve imaging fusion

Abstract

The invention discloses a fundus image evaluation method based on digital twinning and optic nerve imaging fusion, and relates to the technical field of image processing, comprising the following steps: constructing a fundus digital twinning model; dividing the obtained OCT and OCTA fundus images into a plurality of grid regions and carrying out pixel fusion to obtain a fundus fusion image; inputting the fundus fusion image into a fundus digital twin model to carry out key region superposition and feature segmentation, and calculating image quality coefficients of a plurality of key region grid images and common region grid images by a fundus fusion key region detection large model based on an image quality evaluation index; calculating a quality deviation coefficient between the key area image and the common area image; and comprehensively evaluating the fundus fusion image based on the quality deviation coefficient and the preset quality threshold, thereby solving the problems of insufficient evaluation of the detection accuracy of the key region and the overall quality of the image due to non-uniform quality of the fundus image at present.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a fundus image evaluation method based on the fusion of digital twin and optic nerve imaging. Background Art

[0002] Fundus images are acquired by dedicated fundus cameras or optical coherence tomography (OCT) technology, and mainly include key anatomical structures such as the retina, optic disc, macula, and retinal blood vessels. The optic disc is the most prominent bright yellow area in the fundus image, located at the source of fundus blood vessels and retinal nerves, and is often used to assist in the diagnosis of diseases such as glaucoma; the macula is located in the center of the retina and is the area with the sharpest vision. Its lesions may cause vision loss and other visual impairments, and are often used to diagnose age-related macular degeneration; and retinal blood vessels help identify signs of cardiovascular disease.

[0003] For example, the invention patent with announcement number: CN118196218B discloses a fundus image processing method, device and equipment, which performs a first analysis and processing on the fundus image to calculate and determine the clarity of the fundus image; performs a second analysis and processing on the fundus image to calculate and determine the chromaticity distribution of the fundus image; obtains a clear target scene image; performs style transfer processing on the target scene image based on the fundus image to form a blurred version of the target scene image; and performs processing based on the clarity, chromaticity distribution and blurred version of the scene image to form a simulated image for simulating the imaging effect when the patient looks at the target scene.

[0004] During the acquisition process, fundus images may suffer from image quality degradation due to factors such as uneven illumination, motion blur, and inaccurate focus, which may affect the visibility and clarity of key structures (such as the optic disc, macula, and retinal blood vessels). In addition, different areas of fundus images may have quality differences, especially between key areas and ordinary areas. In view of the above problems, the present invention proposes a solution. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a fundus image evaluation method based on the fusion of digital twin and optic nerve imaging, so as to solve the current problems of fundus image uneven quality, key area detection accuracy and insufficient assessment of the overall image quality.

[0006] To achieve the above object, the present invention provides the following technical solutions: The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging includes the following steps: dividing the acquired OCT and OCTA fundus images into a number of grid areas, calculating the pixel values ​​of the OCT and OCTA fundus images in each grid area and fusing them, obtaining the fundus fusion image and performing preprocessing; The fundus fusion image is input into the fundus digital twin model for superposition and feature segmentation of key areas to obtain a fundus fusion image dataset, and the fundus fusion image dataset is input into a preset target detection large model for training to obtain a fundus fusion key area detection large model; based on the fundus fusion key area detection large model, the key areas of the fundus fusion image are detected, and the fundus fusion image is divided into key areas and ordinary areas, and the key areas and ordinary areas are divided into a number of key area grid images and ordinary area grid images; based on the image quality evaluation index, the image quality coefficients of the several key area grid images and the ordinary area grid images are calculated, and the quality deviation coefficient between the key area image and the ordinary area image is calculated; based on the quality deviation coefficient between the key area image and the ordinary area image and the preset quality threshold, the fundus fusion image is comprehensively evaluated.

[0007] In a preferred embodiment, the acquired OCT and OCTA fundus images are divided into several grid areas, the pixel values ​​of the OCT and OCTA fundus images in each grid area are calculated and fused to obtain a fused fundus image and perform preprocessing, specifically: the acquired OCT and OCTA fundus images are pixel-aligned using a registration method to obtain aligned OCT and OCTA fundus images; the aligned OCT and OCTA fundus images are divided into several grid areas in the same manner to obtain several aligned OCT and OCTA fundus grid images; the pixel values ​​of the several aligned OCT and OCTA fundus grid images are calculated respectively; the pixel values ​​of the several aligned OCT and OCTA fundus grid images are weightedly fused to obtain fused pixel values ​​of several grid areas; the fused pixel values ​​of each grid area are spliced ​​grid by grid to obtain a fused fundus image.

[0008] In a preferred embodiment, the fused pixel values ​​of each grid area are spliced ​​grid by grid to obtain a fundus fusion image, specifically: for each row of the grid area, the fused pixel values ​​of all grids in the row are horizontally spliced ​​in order from left to right to obtain a complete fusion area of ​​each row; the complete fusion area of ​​each row is vertically superimposed to obtain a fundus fusion image.

[0009] In a preferred embodiment, the fundus fusion image is input into the fundus digital twin model to perform superposition and feature segmentation of key areas to obtain a fundus fusion image data set, and the fundus fusion image data set is input into a preset target detection large model for training to obtain a fundus fusion key area detection large model, specifically: the fundus fusion image is combined with the patient's actual image to obtain the patient's fundus twin virtual body; the geometric function feature database of the patient's fundus twin virtual body is used to match the key areas in the fundus fusion image for superposition of key areas, and the key areas refer to the macular area, the optic nerve head area, and the vascular dense area; a deep learning segmentation algorithm is used in the fundus digital twin model to perform fine-grained segmentation of the key areas to obtain precise bounding boxes of the key areas, and the precise bounding boxes of the key areas are saved in a preset training set format to obtain a fundus fusion image data set; the fundus fusion image data set is divided into a training set and a test set according to a preset ratio, and the training set is input into the target detection large model to train the large model to obtain a trained target detection large model; In a preferred embodiment, the image quality coefficients of the several key area grid images and the ordinary area grid images are calculated based on the image quality evaluation index, specifically: the clarity evaluation values, edge clarity evaluation values, and brightness evaluation values ​​of the several key area grid images and the ordinary area grid images are calculated respectively based on the image quality evaluation index; the clarity evaluation values, edge clarity evaluation values, and brightness evaluation values ​​are standardized, and weighted summation is performed to obtain the image quality coefficients of the several key area grid images and the ordinary area grid images.

[0010] In a preferred embodiment, the calculation of the quality deviation coefficient between the key area image and the ordinary area image is specifically as follows: based on the image quality coefficients of several key area grid images and the ordinary area grid images, a first quality coefficient of the key area image and a first quality coefficient of the ordinary area are calculated using the geometric mean method; based on the first quality coefficient of the key area image and the first quality coefficient of the ordinary area, an average quality coefficient of the key area image and an average quality coefficient of the ordinary area image are calculated; the image quality similarity index of the key area image and the ordinary area image is calculated, and based on the image quality similarity index, the quality deviation coefficient between the key area image and the ordinary area image is obtained.

[0011] In a preferred embodiment, the fundus fusion image is comprehensively evaluated based on the quality deviation coefficient between the key area image and the ordinary area image and a preset quality threshold, specifically: when the quality deviation coefficient between the key area image and the ordinary area image is less than the preset quality threshold, the image quality is qualified; when the quality deviation coefficient between the key area image and the ordinary area image is greater than the preset quality threshold, the image quality is unqualified.

[0012] The technical effects and advantages of the fundus image evaluation method based on the fusion of digital twin and optic nerve imaging of the present invention are as follows: 1. The present invention integrates the OCT and OCTA fundus images and pathological data collected by the patient multiple times, and combines the deep learning large model to build a fundus digital twin model, which has significant advantages. First, the model realizes personalized dynamic modeling of patients, can accurately simulate the anatomical structure and lesion characteristics of the retina, and provide individualized diagnosis and treatment support; second, the digital twin technology generates a variety of simulation scenarios and data enhancements, which helps to improve the generalization ability of the model; third, the twin realizes automatic labeling and feature segmentation during data processing, which significantly improves the labeling efficiency and accuracy and reduces manual intervention; in addition, the model optimizes the diagnosis and prediction results through a real-time feedback mechanism, and supports the dynamic evaluation of complex lesions; finally, the fundus digital twin model not only improves the scientificity and reliability of clinical decision-making, but also provides simulation support for disease progression monitoring and treatment plan design. It is an important innovative tool in the field of intelligent diagnosis and treatment of ophthalmology.

[0013] 2. The present invention generates a high-quality fundus fusion image by fusing OCT and OCTA images, and further divides them into grid areas, so that the detailed information of each area is richer and more accurate. At the same time, the annotation of key areas and the training of the detection large model improve the recognition ability of the large model for fundus lesions. In the detection stage, the method can automatically identify and distinguish the key areas and ordinary areas in the image, and process them by area, so that the analysis is more targeted. By finely dividing the grid images of key areas and the grid images of ordinary areas, the attention of the large model to key areas is improved, which is helpful for early detection of lesions and high-precision lesion localization; secondly, the image quality coefficients of the grid images of key areas and ordinary areas are calculated, and the image quality of different areas is accurately quantified to help identify the clarity and detail retention of key areas. By further calculating the quality deviation coefficient, the quality consistency between key areas and ordinary areas can be analyzed, which helps to ensure that the imaging quality of important areas is higher and more reliable. Combined with the comprehensive evaluation of the image quality coefficient and the deviation coefficient with the preset quality threshold, the fundus fusion images with qualified overall quality and good regional quality consistency can be effectively screened out. This evaluation method not only enhances the scientificity and reliability of image quality control, but also provides a stable and clear image basis for subsequent disease diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a structural schematic diagram of the fundus image evaluation method based on the fusion of digital twin and optic nerve imaging of the present invention. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] Embodiment 1, Figure 1 The present invention provides a fundus image evaluation method based on the fusion of digital twin and optic nerve imaging, which includes the following steps: S1, dividing the acquired OCT and OCTA fundus images into several grid areas, calculating the pixel values ​​of the OCT and OCTA fundus images in each grid area and fusing them to obtain the fundus fusion image and perform preprocessing; In this example, the acquired OCT and OCTA fundus images are divided into several grid areas, the pixel values ​​of the OCT and OCTA fundus images in each grid area are calculated and fused, and the fundus fusion image is obtained and preprocessed, specifically: The acquired OCT and OCTA fundus images are aligned with pixels by using a registration method to obtain aligned OCT and OCTA fundus images; Dividing the aligned OCT and OCTA fundus images into a plurality of grid areas in the same manner to obtain a plurality of aligned OCT and OCTA fundus grid images; Calculating pixel values ​​of the plurality of aligned OCT and OCTA fundus grid images respectively; Performing weighted fusion on the pixel values ​​of the plurality of aligned OCT and OCTA fundus grid images to obtain fused pixel values ​​of a plurality of grid areas; The fused pixel values ​​of each grid area are spliced ​​grid by grid to obtain the fundus fusion image.

[0017] In this example, the acquired OCT and OCTA fundus images are pixel-aligned using a registration method to obtain aligned OCT and OCTA fundus images, specifically: Convert the OCT and OCTA images into grayscale images, and use the SURF algorithm to extract key feature points in the grayscale images; Use the KNN algorithm to perform feature matching on key feature points in the grayscale image and find similar point pairs in the OCT image and OCTA image; Estimate the transformation matrix from the OCT image to the OCTA image based on similar point pairs in the OCT image and the OCTA image; The transformation matrix from the OCT image to the OCTA image is applied to the OCT and OCTA images for alignment to obtain aligned OCT and OCTA fundus images.

[0018] It should be noted that the SURF algorithm is an efficient and robust feature point detection and description method suitable for applications with high real-time requirements. SURF detects feature points through the Hessian matrix, and its calculation speed is faster than SIFT. It also has scale and rotation invariance, so it is very robust to image rotation, scaling and partial occlusion. SURF uses Haar wavelet response in feature description, which speeds up the descriptor calculation process and improves the matching accuracy. In addition, the feature points calculated by the SURF algorithm are relatively few and representative, making the feature matching efficient, especially suitable for medical image processing tasks that require fast and accurate registration, such as fundus OCT and OCTA image registration. Overall, SURF achieves a good balance between efficiency and accuracy.

[0019] The KNN (K nearest neighbor) algorithm has high accuracy and simplicity in image feature matching and is an effective method for finding similar point pairs by feature distance. For feature points extracted from OCT and OCTA images, KNN can quickly find multiple nearest points (usually the first two nearest points) of each feature point in another image, thereby improving the accuracy of matching. In addition, by using the distance ratio test in matching, erroneous matching points can be effectively filtered out, thereby improving the reliability of feature matching. KNN has high computational efficiency in multidimensional feature space and is suitable for situations with more image data. Therefore, in medical image processing and multimodal image registration, KNN matching can provide a solid foundation for subsequent image alignment.

[0020] In this example, the pixel values ​​of the aligned OCT and OCTA fundus grid images are calculated respectively, and the specific formula is:

[0021] in, and are the pixel values ​​of the aligned OCT and OCTA fundus grid images, and are the pixel values ​​of the OCT and OCTA fundus grid images at positions (x, y) within a certain grid area, S is the side length of the grid area, and x and y are the horizontal and vertical coordinates of the position within a certain grid area, respectively.

[0022] In this example, the fused pixel values ​​of each grid area are spliced ​​grid by grid to obtain the fundus fusion image, specifically: For each row of the grid area, horizontally splice the fused pixel values ​​of all grids in the row from left to right to obtain the complete fused area of ​​each row; The complete fusion area of ​​each row is vertically superimposed to obtain a fundus fusion image.

[0023] It should be noted that grid-by-grid splicing of the fused pixel values ​​of each grid area can effectively integrate the information in the OCT and OCTA images, thereby generating a fundus fusion image with rich information. This grid-by-grid splicing method first horizontally splices each row to merge the pixel information in the same row, ensuring that the spatial information of each grid area is accurately transmitted. Subsequently, the fusion areas of each row are vertically superimposed to maintain the overall structure and spatial continuity of the image. This method not only retains the details and important features in images of different modalities, but also ensures a seamless transition of the spliced ​​images, avoiding the boundary distortion problem in traditional splicing methods. The fundus fusion image obtained in this way has higher resolution and more diagnostic information, providing more powerful image support for the early detection and accurate diagnosis of fundus diseases.

[0024] In this example, the pixel values ​​of the several aligned OCT and OCTA fundus grid images are weightedly fused to obtain fused pixel values ​​of several grid areas, specifically: The selection of weights in the weighted fusion is based on the doctor's diagnostic experience data and historical fundus images through machine learning training. For example, the blood vessel-related area has a higher weight in the OCTA fundus grid image, and the tissue thickness-related area has a higher weight in the OCT fundus grid image.

[0025] S2, detecting the key area of ​​the fundus fusion image based on a preset fundus fusion key area detection large model, dividing the fundus fusion image into a key area and a common area, and dividing the key area and the common area into a plurality of key area grid images and common area grid images; In this example, the preset fundus fusion key area detection model is as follows: Combining the fundus fusion image with the actual image of the patient to obtain a fundus twin virtual body of the patient; Using the geometric function feature database of the patient's fundus twin virtual body, matching the key areas in the fundus fusion image to perform key area superposition, the key areas refer to the macular area, the optic nerve head area, and the blood vessel dense area; Using a deep learning segmentation algorithm in the fundus digital twin model to perform fine-grained segmentation on the key area, obtaining an accurate bounding box of the key area, and saving the accurate bounding box of the key area in a preset training set format to obtain a fundus fusion image dataset; The fundus fusion image data set is divided into a training set and a test set according to a preset ratio, and the training set is input into the target detection large model for training the large model to obtain the trained target detection large model; In this example, the fundus fusion image is combined with the actual image of the patient to obtain the patient's fundus twin virtual body, specifically: The patient's fundus twin virtual body is a digital simulation model based on the patient's actual fundus images (such as OCT and OCTA fusion images) and multimodal data such as pathology, anatomy, and physiology. This virtual body can truly reflect the patient's fundus health status and is used for dynamic monitoring, prediction, diagnosis, and treatment optimization. Specifically, the patient's fundus twin virtual body includes the following key components: Patient fundus images (OCT and OCTA fused images), fundus pathology data (such as diabetic retinopathy, macular degeneration, etc.), and anatomical data involving the structure of the eyeball.

[0026] In this example, the key areas of the fundus fusion image are the macula, the optic nerve head, and the blood vessel dense area, specifically: Macular area: The macula is located in the central part of the retina. It is the area with the highest visual acuity and is responsible for the visual perception of details. The macular area is crucial to the clarity and accuracy of vision. Pathologies such as macular degeneration may lead to severe vision loss. Therefore, the health of the macular area is very important for ophthalmic diagnosis; Optic nerve head area: The optic nerve head is located on the nasal side of the retina. It is the site where the optic nerve fibers transmit signals from the eyeball. It is also commonly known as the "optic disc." The optic nerve head area is very important for the neurological function of the eye. Diseases such as glaucoma may cause damage or atrophy of the optic nerve head, thereby affecting vision.

[0027] Densely vascularized areas: These areas are usually distributed in the retina, where the distribution of blood vessels is very dense, mainly including the retinal vascular plexus and the central retinal artery. Densely vascularized areas can provide blood supply to the fundus, and any vascular lesions (such as diabetic retinopathy, retinal hemorrhage, etc.) may appear in these areas, so monitoring these areas is crucial.

[0028] In this example, the fundus fusion image data set is divided into a training set and a test set according to a preset ratio, and the training set is input into the target detection large model for training the large model to obtain the trained target detection large model, specifically: The target detection large model can select target detection algorithms such as Faster R-CNN, YOLO and SSD, and can add an attention mechanism to improve the detection accuracy of the large model.

[0029] It should be noted that there are significant advantages in labeling the key areas in the fundus fusion image and inputting them into the preset target detection large model for training. First, the target detection large model can automatically identify and locate key areas from the fundus image, such as the macula area, the optic nerve head area, and the vascular dense area, which are crucial for diagnosing eye diseases. Through data set labeling, the large model can learn the spatial characteristics of the key areas and enhance its adaptability to various fundus images. Secondly, the target detection large model has strong robustness and accuracy, and can handle the differences between different modalities (such as OCT and OCTA) and provide consistent results. The large detection model obtained through training can perform image analysis efficiently and automatically, reduce the workload of manual labeling, and improve the efficiency and accuracy of diagnosis. In addition, the trained large model can quickly locate and analyze key areas when processing large-scale fundus images, thereby providing ophthalmologists with more accurate and rapid auxiliary decision support.

[0030] In this example, the key area of ​​the fundus fusion image is detected based on the fundus fusion key area detection large model, and the fundus fusion image is divided into key areas and common areas, specifically: Based on the fundus fusion key area detection large model, it can accurately detect the key areas in the fundus image (such as the macula, optic nerve head and vascular dense areas), and divide the image into key areas and common areas, which has significant advantages. First, this method can automatically identify the important areas in the fundus image, avoid the subjectivity and inefficiency of manual annotation, and improve the processing speed and accuracy. Secondly, by further dividing the key areas and common areas into grid images, more detailed local analysis can be achieved, so that every detail in the image can be fully evaluated. This is of great significance for the early detection of ophthalmic diseases, disease monitoring and evaluation of treatment effects. In addition, refined regional division helps to improve the accuracy and stability of image quality assessment, and enhances the adaptability of the large model to fundus images of different patients and different modalities (such as OCT, OCTA). Therefore, the whole method improves the automation level of fundus image analysis and the accuracy of diagnosis.

[0031] S3, calculating the image quality coefficients of the plurality of key area grid images and the common area grid images based on the image quality evaluation index, and calculating the quality deviation coefficient between the key area images and the common area images; In this example, the image quality coefficients of the plurality of key area grid images and the common area grid images are calculated based on the image quality evaluation index, specifically: Calculating the clarity evaluation value, edge clarity evaluation value, and brightness evaluation value of the plurality of key area grid images and the common area grid images respectively based on the image quality evaluation index; The clarity evaluation value, edge clarity evaluation value, and brightness evaluation value are standardized, and weighted summed to obtain image quality coefficients of a plurality of key area grid images and common area grid images.

[0032] In this example, the clarity evaluation values, edge clarity evaluation values, and brightness evaluation values ​​of the plurality of key area grid images and the common area grid images are calculated based on the image quality evaluation index, and the specific formula is:

[0033]

[0034]

[0035] in, is the clarity evaluation value of the grid area, and are the gradient values ​​of the key area grid image and the common area grid image in the horizontal and vertical directions respectively, N is the total pixels in the grid area, is the edge clarity evaluation value of the grid area, is the Laplace response value of the key area grid image and the common area grid image at position (x, y), is the brightness evaluation value of the grid area, is the gray value of the i-th pixel, is the average brightness value of the grid area, is the brightness standard deviation of the grid area, is a small positive number.

[0036] In this example, the clarity evaluation value, edge clarity evaluation value, and brightness evaluation value are specifically: Clarity evaluation value: The clarity evaluation value measures the overall clarity of the image and reflects the sharpness of the image details based on the local gradient change. Images with higher clarity usually show more delicate structures and richer detail information. Edge clarity evaluation value: The edge clarity evaluation value is used to measure the clarity of the edge area in the image. It mainly extracts important edge information in the image through the edge detection algorithm and reflects the accuracy of the image boundary.

[0037] Brightness evaluation value: reflects the overall brightness level of the image, mainly measuring the brightness of the image. It is usually obtained by calculating the average brightness or brightness distribution of the image.

[0038] It should be noted that the clarity evaluation value, edge clarity evaluation value and brightness evaluation value are key indicators in image quality evaluation, each with unique advantages. The clarity evaluation value can quantify the image's ability to present details, which is crucial for identifying tiny lesions or structural changes, especially in fundus images, where it can help accurately capture details and improve the accuracy of diagnosis. The edge clarity evaluation value focuses on the sharpness of the image edge, especially in fundus images, where it can effectively identify the edges of key areas such as the macula and optic nerve head, helping doctors detect the specific locations of blood vessels and lesions. The brightness evaluation value ensures that the image has a reasonable distribution of light and dark, avoids information loss caused by images that are too bright or too dark, and ensures visibility of important details. Combining these three evaluation values, the quality of fundus images can be comprehensively measured, optimized and analyzed from different dimensions, and the accuracy and diagnostic value of image analysis can be improved. These indicators not only help automated systems improve detection efficiency, but also support doctors in making more accurate decisions in actual clinical diagnosis.

[0039] In this example, the quality deviation coefficient between the key area image and the common area image is calculated, specifically: Based on the image quality coefficients of the plurality of key area grid images and the common area grid images, a first quality coefficient of the key area image and a first quality coefficient of the common area are calculated by using a geometric mean method; Based on the first quality coefficient of the key area image and the first quality coefficient of the common area, an average quality coefficient of the key area image and an average quality coefficient of the common area image are calculated; An image quality similarity index between the key region image and the common region image is calculated, and a quality deviation coefficient between the key region image and the common region image is obtained based on the image quality similarity index.

[0040] In this example, the first quality coefficient of the key area image and the first quality coefficient of the common area are calculated based on the image quality coefficients of several key area grid images and common area grid images by using the geometric mean method. The specific formula is as follows:

[0041] in, is the first quality coefficient, is the inverse function of the natural logarithm, is the weight of the ith grid area, is the image quality coefficient of the i-th grid area, the total number of grids divided in the T key area, is the natural logarithm of the image quality coefficient of the i-th grid area.

[0042] In this example, the image quality similarity index of the key area image and the general area image is calculated, and the specific formula is as follows:

[0043] in, is the image quality similarity index between the key area image and the common area image, and are the average quality coefficients of key area images and the average quality coefficients of common area images, and are the variances of the image quality coefficients of the key area grid image and the common area grid image, are the covariances of the image quality coefficients of the key area grid image and the normal area grid image, respectively.

[0044] It should be noted that through the grid-based calculation of image quality coefficients, a more detailed quality analysis of key areas and ordinary areas can be performed, avoiding local quality differences that may be concealed by the overall evaluation, and improving the accuracy of the evaluation; the first quality coefficients of key areas and ordinary areas are calculated using the geometric mean method, which reduces the impact of extreme values ​​on the overall evaluation, thereby providing more stable and fair evaluation results; by calculating the average quality coefficient, the overall image quality in the area can be comprehensively measured, reflecting the relative quality level between areas, and providing data support for image quality optimization; the overall consistency of key areas and ordinary areas is measured through the image quality similarity index, and the quality differences are quantitatively described in combination with the deviation coefficient, providing a clear goal for image correction and optimization.

[0045] In this example, the image quality similarity index is specifically: The image quality similarity index can effectively evaluate image quality and structural consistency by quantifying the similarity between images. Its advantage is that it can objectively and accurately compare the differences between different images or image regions, avoiding subjective bias. In fundus image processing, the image quality similarity index helps identify the quality consistency between key areas and ordinary areas, ensuring that the image quality meets diagnostic requirements. This method not only improves the reliability of automated image evaluation, but also supports the accuracy of image optimization and processing, providing strong data support for clinical decision-making.

[0046] In this example, the quality deviation coefficient between the key area image and the common area image is obtained based on the image quality similarity index, specifically:

[0047] in, is the image quality similarity index, is the quality deviation coefficient between the key area image and the common area image.

[0048] S4, comprehensively evaluating the fundus fusion image based on the image quality coefficient, the quality deviation coefficient and a preset quality threshold.

[0049] In this example, the fundus fusion image is comprehensively evaluated based on the quality deviation coefficient between the key area image and the common area image and a preset quality threshold, specifically: When the quality deviation coefficient between the key area image and the general area image is less than the preset quality threshold, the image quality is excellent; When the quality deviation coefficient between the key area image and the common area image is greater than a preset quality threshold, the image quality is unqualified.

[0050] It should be noted that the method of comprehensively evaluating fundus fusion images based on image quality coefficient, quality deviation coefficient and preset quality threshold has many advantages. First, by setting reasonable quality coefficient threshold and quality deviation threshold, the image quality assessment can be quantified and standardized, the interference of subjective factors can be reduced, and the objectivity and consistency of image assessment can be ensured. Secondly, this method effectively distinguishes images of different quality levels: by setting different conditions (such as quality coefficient and quality deviation threshold), it can accurately identify the state of qualified, qualified or unqualified image quality. This hierarchical evaluation helps to optimize the image processing process and further correct and optimize low-quality images. Finally, this evaluation method combines the quality assessment of key areas and ordinary areas to ensure that the quality of important areas in fundus images is fully guaranteed, which is suitable for the high-precision requirements in clinical diagnosis and improves the reliability and practicality of the automation system.

[0051] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0052] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product.

[0053] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0054] In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0055] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0056] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A fundus image evaluation method based on the fusion of digital twin and optic nerve imaging, characterized in that: The following steps are involved: The acquired OCT and OCTA fundus images are divided into several grid areas, the pixel values ​​of the OCT and OCTA fundus images in each grid area are calculated and fused to obtain the fundus fusion image and perform preprocessing; Construct a fundus digital twin model, detect the key areas of the fundus fusion image based on the preset fundus fusion key area detection model, and divide the fundus fusion image into several key area grid images and common area grid images; Calculating image quality coefficients of the plurality of key area grid images and common area grid images based on the image quality evaluation index, and calculating quality deviation coefficients between the key area images and the common area images; The fundus fusion image is comprehensively evaluated based on the quality deviation coefficient between the key area image and the common area image and a preset quality threshold.

2. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 1 is characterized in that: The acquired OCT and OCTA fundus images are divided into a number of grid areas, the pixel values ​​of the OCT and OCTA fundus images in each grid area are calculated and fused, and the fundus fusion image is obtained and preprocessed, specifically: The acquired OCT and OCTA fundus images are pixel-aligned using a registration method to obtain aligned OCT and OCTA fundus images; Evenly dividing the aligned OCT and OCTA fundus images into a plurality of grid areas to obtain a plurality of aligned OCT and OCTA fundus grid images; Calculating pixel values ​​of the plurality of aligned OCT and OCTA fundus grid images respectively; Performing weighted fusion on the pixel values ​​of the plurality of aligned OCT and OCTA fundus grid images to obtain fused pixel values ​​of a plurality of grid areas; The fused pixel values ​​of each grid area are spliced ​​grid by grid to obtain the fundus fusion image.

3. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 2 is characterized in that: The fusion pixel values ​​of each grid area are spliced ​​grid by grid to obtain the fundus fusion image, specifically: For each row of the grid area, horizontally splice the fused pixel values ​​of all grids in each row from left to right to obtain the complete fused area of ​​each row; The complete fusion area of ​​each row is vertically superimposed to obtain a fundus fusion image.

4. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 3 is characterized in that: The fundus digital twin model is constructed to detect the key areas of the fundus fusion image based on the preset fundus fusion key area detection model, specifically: Combining the fundus fusion image with the actual image of the patient to obtain a fundus twin virtual body of the patient; Using the geometric function feature database of the patient's fundus twin virtual body, matching the key areas in the fundus fusion image to perform key area superposition, the key areas refer to the macular area, the optic nerve head area, and the blood vessel dense area; Using a deep learning segmentation algorithm in the fundus digital twin model to perform fine-grained segmentation on the key area, obtaining an accurate bounding box of the key area, and saving the accurate bounding box of the key area in a preset training set format to obtain a fundus fusion image dataset; The fundus fusion image data set is divided into a training set and a test set according to a preset ratio, and the training set is input into the target detection large model for training the large model to obtain the trained target detection large model; The trained target detection large model is verified by using a test set to obtain a fundus fusion key area detection large model; The fundus image to be predicted is input into the fundus fusion key area detection large model to obtain the key area of ​​the fundus fusion image.

5. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 4 is characterized in that: The image quality coefficients of the plurality of key area grid images and the common area grid images are calculated based on the image quality evaluation index, specifically: Calculating the clarity evaluation value, edge clarity evaluation value, and brightness evaluation value of the plurality of key area grid images and the common area grid images respectively based on the image quality evaluation index; The clarity evaluation value, edge clarity evaluation value, and brightness evaluation value are standardized, and weighted summed to obtain image quality coefficients of a plurality of key area grid images and common area grid images.

6. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 5 is characterized in that: The quality deviation coefficient between the key area image and the common area image is calculated as follows: Based on the image quality coefficients of the plurality of key area grid images and the common area grid images, a first quality coefficient of the key area image and a first quality coefficient of the common area are calculated by using a geometric mean method; Based on the first quality coefficient of the key area image and the first quality coefficient of the common area, an average quality coefficient of the key area image and an average quality coefficient of the common area image are obtained by an average calculation method; An image quality similarity index between the key region image and the common region image is calculated, and a quality deviation coefficient between the key region image and the common region image is obtained based on the image quality similarity index.

7. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 6 is characterized in that: The fundus fusion image is comprehensively evaluated based on the quality deviation coefficient between the key area image and the common area image and a preset quality threshold, specifically: When the quality deviation coefficient between the key area image and the common area image is less than the preset quality threshold, the image quality is qualified; When the quality deviation coefficient between the key area image and the common area image is greater than a preset quality threshold, the image quality is unqualified.

8. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 7 is characterized in that: The pixel values ​​of the aligned OCT and OCTA fundus grid images are calculated respectively, and the specific formula is: in, and are the pixel values ​​of the aligned OCT and OCTA fundus grid images, and are the pixel values ​​of the OCT and OCTA fundus grid images at positions (x, y) within a certain grid area, S is the side length of the grid area, and x and y are the horizontal and vertical coordinates of the position within a certain grid area, respectively.

9. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 8 is characterized in that: The definition evaluation value, edge definition evaluation value, and brightness evaluation value of the plurality of key area grid images and the common area grid images are respectively calculated based on the image quality evaluation index, and the specific formula is: in, is the clarity evaluation value of the grid area, and are the gradient values ​​of the key area grid image and the common area grid image in the horizontal and vertical directions respectively, N is the total pixels in the grid area, is the edge clarity evaluation value of the grid area, is the Laplace response value of the key area grid image and the common area grid image at position (x, y), is the brightness evaluation value of the grid area, is the gray value of the i-th pixel, is the average brightness value of the grid area, is the brightness standard deviation of the grid area, is a small positive number.

10. The fundus image evaluation method based on the fusion of digital twin and optic nerve imaging according to claim 9, characterized in that: The image quality similarity index of the key area image and the common area image is calculated, and the specific formula is: in, is the image quality similarity index between the key area image and the common area image, and are the average quality coefficients of key area images and the average quality coefficients of common area images, and are the variances of the image quality coefficients of the key area grid image and the common area grid image, are the covariances of the image quality coefficients of the key area grid image and the normal area grid image, respectively.

Images