A three-dimensional image processing method for fundus examination

Abstract

The present invention relates to the field of image processing technology, and in particular to a three-dimensional image processing method for fundus examination, the method comprising: obtaining an initial OCT image and an initial OCTA image; processing the initial OCT image based on threshold segmentation to extract an initial retinal image and an initial choroidal image; identifying an image signal corresponding to the initial retinal image, performing algorithm fitting on the retinal image signal through a multi-scale context aggregation network to obtain a fitted retinal image; reconstructing the fitted retinal image based on a retinal curvature and a preset curvature value to obtain a reconstructed retinal image; drawing a fundus three-dimensional structure image based on the reconstructed retinal image and the initial choroidal image; identifying the optic nerve head in the initial OCTA image and the fundus three-dimensional structure image, fitting the initial OCTA image with the fundus three-dimensional structure image with the optic nerve head as the center to obtain a final fundus three-dimensional image. The present invention improves the accuracy of the imaging result of the fundus three-dimensional image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a three-dimensional image processing method for fundus examination. Background Art

[0002] Optical coherence tomography (OCT) is a part of tissue optics. OCT technology utilizes the light transmittance of biological structures and uses photodetectors to detect reflection, scattering and other signals of biological tissues, and converts them into electrical signals to reconstruct the image structure of biological tissues through computers.

[0003] The patent document with Chinese patent publication number CN106683080A discloses a retinal fundus image preprocessing method, which is characterized by comprising the following steps: 1) reading in the original image: using the green channel to read in the original retinal fundus image; 2) removing the central light reflection of the blood vessels; 3) removing the salt and pepper noise; 4) smoothing the noise; 5) background extraction; 6) obtaining a shadow-corrected image; 7) obtaining a homogenized image; 8) obtaining a complementary image; 9) obtaining a blood vessel enhanced image; 10) outputting the enhanced image.

[0004] The prior art mainly focuses on the processing of two-dimensional images for fundus examination. Due to the complexity and diversity of fundus tissue, a single image processing method is often unable to cope with all situations, resulting in inaccurate image processing results, thereby causing the problem of low fundus image imaging accuracy. Summary of the invention

[0005] To this end, the present invention provides a three-dimensional image processing method for fundus examination, which constructs a final fundus three-dimensional image by reconstructing the three-dimensional structure of the retina and choroid and fusing OCTA data, thereby solving the problem of low fundus image imaging accuracy caused by the limitations of existing two-dimensional image processing methods in fundus examination.

[0006] To achieve the above object, the present invention provides a three-dimensional image processing method for fundus examination, comprising:

[0007] Acquire an initial OCT image and an initial OCTA image of the target to be detected;

[0008] Processing the initial OCT image based on threshold segmentation, and extracting an initial retinal image and an initial choroidal image based on the processing result;

[0009] Identify the image signal corresponding to the initial retinal image, obtain the retinal image signal, perform algorithm fitting on the retinal image signal through a multi-scale context aggregation network to obtain a fitted image signal, and obtain a fitted retinal image based on the fitted image signal;

[0010] Reconstructing the fitted retinal image based on the retinal curvature in the fitted retinal image and a preset curvature value to obtain a reconstructed retinal image;

[0011] Drawing a fundus three-dimensional structure image based on the reconstructed retinal image and the initial choroidal image;

[0012] The optic nerve head in the initial OCTA image and the fundus three-dimensional structure image is identified, and the initial OCTA image and the fundus three-dimensional structure image are fitted with the optic nerve head as the center to obtain a final fundus three-dimensional image.

[0013] Furthermore, the step of processing the initial OCT image based on threshold segmentation includes:

[0014] Dividing the initial OCT image into a plurality of grids by preset pixel blocks;

[0015] Identify the brightness values ​​corresponding to the grids, and obtain actual brightness values;

[0016] Determine the average of several historical brightness values ​​corresponding to the historical retinal image as the retinal brightness threshold, and determine the average of several historical brightness values ​​corresponding to the historical choroidal image as the choroid brightness threshold;

[0017] Mark the actual brightness value with the smallest difference with the retinal brightness threshold value among the plurality of actual brightness values ​​to obtain a first marked brightness value, and mark the actual brightness value with the smallest difference with the choroidal brightness threshold among the plurality of actual brightness values ​​to obtain a second marked brightness value;

[0018] A first central area is determined based on the first marker brightness value, and a second central area is determined based on the second marker brightness value.

[0019] Furthermore, the step of extracting the initial retinal image and the initial choroidal image based on the processing result includes:

[0020] Determining a retinal brightness range based on a historical maximum value and a historical minimum value of a plurality of historical brightness values ​​corresponding to the historical retinal image;

[0021] Determine a choroid brightness range based on a historical maximum value and a historical minimum value of a plurality of historical brightness values ​​corresponding to the historical choroid image;

[0022] determining an initial retinal region based on the first central region and the retinal brightness range;

[0023] determining an initial choroid region based on the second central region and the choroid brightness range;

[0024] Extracting continuous areas in the initial retinal area to obtain the initial retinal image;

[0025] Continuous regions in the initial choroid region are extracted to obtain the initial choroid image.

[0026] Furthermore, the step of performing algorithm fitting on the retinal digital signal through the multi-scale context aggregation network includes:

[0027] Establish operator fitting network;

[0028] Acquire a number of historical retinal images and their corresponding historical image signals, and divide them into a training data set and a test data set;

[0029] Preprocessing the training data set based on the auxiliary function, performing training based on the preprocessed training data set and the multi-scale CAN layer to obtain an initial operator fitting network;

[0030] Constructing a test operator fitting network based on the test data set and the initial operator fitting network;

[0031] Algorithmic fitting of retinal image signals through a multi-scale context aggregation network;

[0032] Inputting the retinal image signal into the test operator fitting network to obtain a fitting feature vector;

[0033] The fitting feature vector is converted into the fitting image signal, and image reconstruction is performed based on the fitting image signal to obtain a fitting retinal image.

[0034] Furthermore, the step of reconstructing the fitted retinal image based on the retinal curvature in the fitted retinal image and a preset curvature value comprises:

[0035] Identifying the retinal edge in the fitted retinal image, calculating the curvature of the retinal edge, and obtaining a number of actual curvature values;

[0036] Comparing the actual bending radian values ​​with the preset bending radian values ​​to obtain a plurality of radian difference values;

[0037] elastically transforming the fitting retinal image according to the plurality of curvature difference values ​​so that the actual curvature value of the fitting retinal image matches the preset curvature value;

[0038] The elastically transformed image is filled at the pixel level by an interpolation algorithm to obtain the reconstructed retinal image.

[0039] Furthermore, the step of calculating the curvature of the retinal edge includes:

[0040] Acquire an edge pixel point set based on the retinal edge;

[0041] Fitting the edge pixel point set based on a curve fitting algorithm to obtain an edge fitting curve;

[0042] The actual curvature values ​​corresponding to each point on the edge fitting curve are calculated to obtain a plurality of actual curvature values, and the plurality of actual curvature values ​​are used as the plurality of actual bending radian values.

[0043] Furthermore, the step of elastically transforming the fitted retinal image according to the radian difference value comprises:

[0044] Determine a plurality of deformation parameters corresponding to each pixel point at the edge of the retina based on the plurality of curvature difference values;

[0045] Applying the deformation parameters to the pixel points corresponding to the retinal edge to locally deform the retinal edge;

[0046] The deformation parameters are gradually adjusted in an iterative manner until the actual bending curvature value matches the preset bending curvature value.

[0047] Furthermore, the step of drawing a fundus three-dimensional structure image based on the reconstructed retinal image and the initial choroidal image comprises:

[0048] identifying a first key point and a second key point in the reconstructed retinal image and the initial choroidal image, respectively;

[0049] Matching the first key point and the second key point, and determining an image matching point based on the matching result;

[0050] Detecting a first target contour in the reconstructed retinal image and detecting a second target contour in the initial choroidal image;

[0051] Calculate a first minimum bounding rectangle based on the first object outline, and calculate a second minimum bounding rectangle based on the second object outline;

[0052] Defining four correction vertex positions of the initial choroidal image according to the historical fundus three-dimensional structure image characteristics;

[0053] A perspective transformation matrix is ​​calculated based on the corrected vertex positions and the image matching points, and the initial choroidal image is perspectively transformed based on the perspective transformation matrix to align the initial choroidal image with the reconstructed retinal image, so as to draw a three-dimensional fundus structure image based on the aligned initial choroidal image and the reconstructed retinal image.

[0054] Furthermore, the step of defining the positions of four correction vertices of the initial choroidal image according to the historical fundus three-dimensional structure image characteristics comprises:

[0055] Identify historical choroidal edge contours in historical fundus three-dimensional structural images;

[0056] The mean edge width was calculated based on the historical choroidal edge contour;

[0057] Two correction vertex positions are defined outside and inside the historical choroidal edge contour based on the average edge width, so that the distance between the two inner correction vertices and the distance between the two outer correction vertices are both in a preset proportion to the average edge width.

[0058] Furthermore, the step of fitting the initial OCTA image with the fundus three-dimensional structure image with the optic nerve head as the center includes:

[0059] Identify the position of the nerve papilla in the initial OCTA image based on a feature detection algorithm to obtain a first position, and identify the position of the nerve papilla in the fundus three-dimensional structure image based on a feature detection algorithm to obtain a second position;

[0060] Acquire a two-dimensional projection surface of the three-dimensional fundus structure image, identify four vertex positions of the two-dimensional projection surface, and calculate a first spacing value between the second position and the four vertex positions;

[0061] Identify the edge contour of the initial OCTA image, select a third minimum circumscribed rectangle based on the edge contour, identify four vertex positions of the third minimum circumscribed rectangle, and calculate a second spacing value between the first position and the four vertex positions;

[0062] Aligning the first position and the second position based on an image transformation technology, and rotating the initial OCTA image so that the first spacing value is the same as the second spacing value;

[0063] The curvature in the initial OCTA image is aligned with the curvature in the fundus three-dimensional structure image to obtain a final fundus three-dimensional image.

[0064] Compared with the prior art, the beneficial effect of the present invention lies in that the initial OCT image is processed by threshold segmentation, the retinal and choroidal images are effectively separated, the interference of background noise and irrelevant information is reduced, thereby improving the accuracy of subsequent image processing and analysis, and the retinal image signal is algorithmically fitted using a multi-scale context aggregation network, which can further optimize the image quality and generate a clearer fitted retinal image, and the fitted retinal image is reconstructed based on the retinal curvature, so that the reconstructed retinal image is closer to the real fundus structure, which not only improves the authenticity of the image, but also provides a more reliable basis for subsequent analysis, and the reconstructed retinal image is combined with the initial choroidal image to draw a three-dimensional fundus structure image, thereby realizing three-dimensional visualization of fundus tissue, and the initial OCTA image is fitted with the three-dimensional fundus structure image with the optic nerve head as the center, thereby ensuring the high consistency of the two images in spatial position and obtaining an accurate three-dimensional structure map.

[0065] In particular, the threshold determination method based on historical data can more accurately identify the retinal and choroidal areas in the image, improve the accuracy and reliability of segmentation, and simplify the complex OCT image into several central areas through gridding and threshold segmentation, which greatly simplifies the subsequent analysis and processing procedures and improves processing efficiency. The mean of historical data is used as the threshold to reduce the impact of noise and individual differences on the segmentation results to a certain extent, thereby enhancing the robustness of the algorithm.

[0066] In particular, accurate description of retinal morphology is ensured by precisely identifying the retinal edge and calculating its curvature. Comparison and adjustment with the preset curvature value make the reconstructed retinal image closer to the ideal or standardized morphology, thereby improving the accuracy and reliability of image analysis. The automated curvature matching and image reconstruction process speeds up image processing. Elastic transformation and pixel-level interpolation filling can smoothly adjust the morphology of retinal images, reduce image distortion and blurring caused by shooting angle, eye movement and other factors, improve the visual effect of the image, and provide a better basis for subsequent image processing and analysis. Using the preset curvature value as a standard, retinal images from different patients or from the same patient at different time points are reconstructed, making these images more consistent in morphology, facilitating more accurate comparison and analysis in different situations, and improving the consistency of results. The reconstructed retinal image has a better standardized morphology, enabling image-based analysis and algorithms to run more efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 A schematic flow chart of a three-dimensional image processing method for fundus examination provided by an embodiment of the present invention;

[0068] Figure 2A schematic diagram of an operator fitting network structure of a three-dimensional image processing method for fundus examination provided by an embodiment of the present invention;

[0069] Figure 3 A second method flow chart of the three-dimensional image processing method for fundus examination provided by an embodiment of the present invention;

[0070] Figure 4 A third method flow chart of the three-dimensional image processing method for fundus examination provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0072] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0073] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be understood as a limitation on the present invention.

[0074] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0075] See also Figure 1 As shown, an embodiment of the present invention provides a three-dimensional image processing method for fundus examination, the method comprising:

[0076] Step S100, acquiring an initial OCT image and an initial OCTA image of the target to be detected;

[0077] Step S200, processing the initial OCT image based on threshold segmentation, and extracting an initial retinal image and an initial choroidal image based on the processing result;

[0078] Step S300, identifying an image signal corresponding to the initial retinal image, obtaining the retinal image signal, performing algorithm fitting on the retinal image signal through a multi-scale context aggregation network to obtain a fitted image signal, and obtaining a fitted retinal image based on the fitted image signal;

[0079] Step S400, reconstructing the fitted retinal image based on the retinal curvature in the fitted retinal image and a preset curvature value to obtain a reconstructed retinal image;

[0080] Step S500, drawing a fundus three-dimensional structure image based on the reconstructed retinal image and the initial choroidal image;

[0081] Step S600, identifying the optic nerve head in the initial OCTA image and the fundus 3D structure image, fitting the initial OCTA image and the fundus 3D structure image with the optic nerve head as the center, and obtaining a final fundus 3D image.

[0082] Specifically, the steps of obtaining the initial OCT image and the initial OCTA image of the target to be detected in the embodiment of the present invention include:

[0083] Detecting the target to be detected by an OCT device, acquiring an initial OCT optical path, identifying the interference light signals of the reference arm and the sample arm in the initial OCT optical path, converting the interference light signals into electrical signals and then into digital signals by a signal receiving device and a signal acquisition device, inputting the digital signals into an OCT algorithm program in a computer, and obtaining an OCT image;

[0084] The light on the sample arm in the OCT optical path is irradiated onto the same section of the bottom of the eye for multiple times to scan and obtain scanning signals. The scanning signals are converted into electrical signals through a signal receiving device and a signal acquisition device, and then converted into digital signals. The digital signals are input into the OCTA algorithm program to obtain OCTA images.

[0085] Specifically, in the embodiment of the present invention, light propagates in the reference arm and returns to the interferometer after passing through a series of optical elements (such as mirrors, lenses, etc.); the length of the reference arm can be precisely controlled for comparison with the light signal in the sample arm;

[0086] After entering the sample arm, the light hits the target to be detected (such as fundus tissue) and is then scattered or reflected back to the interferometer.

[0087] Specifically, the target to be detected in the embodiment of the present invention is an eye.

[0088] Specifically, the signal receiving device in the embodiment of the present invention is a photoelectric detector, and the signal acquisition device is an analog-to-digital converter.

[0089] Specifically, the embodiment of the present invention processes the initial OCT image through threshold segmentation, effectively separates the retinal and choroidal images, reduces the interference of background noise and irrelevant information, thereby improving the accuracy of subsequent image processing and analysis, and uses a multi-scale context aggregation network to perform algorithmic fitting on the retinal image signal, which can further optimize the image quality and generate a clearer fitted retinal image. By reconstructing the fitted retinal image based on the radian of the retina, the reconstructed retinal image is closer to the real fundus structure, which not only improves the authenticity of the image, but also provides a more reliable basis for subsequent analysis. By combining the reconstructed retinal image with the initial choroidal image to draw a three-dimensional fundus structure image, three-dimensional visualization of the fundus tissue is achieved, and the initial OCTA image is fitted with the three-dimensional fundus structure image with the optic nerve head as the center, ensuring the high consistency of the two images in spatial position, and obtaining an accurate three-dimensional structure map.

[0090] Specifically, the step of processing the initial OCT image based on threshold segmentation includes:

[0091] Dividing the initial OCT image into a plurality of grids by preset pixel blocks;

[0092] Identify the brightness values ​​corresponding to the grids, and obtain actual brightness values;

[0093] Determine the average of several historical brightness values ​​corresponding to the historical retinal image as the retinal brightness threshold, and determine the average of several historical brightness values ​​corresponding to the historical choroidal image as the choroid brightness threshold;

[0094] Mark the actual brightness value with the smallest difference with the retinal brightness threshold value among the plurality of actual brightness values ​​to obtain a first marked brightness value, and mark the actual brightness value with the smallest difference with the choroidal brightness threshold among the plurality of actual brightness values ​​to obtain a second marked brightness value;

[0095] A first central area is determined based on the first marker brightness value, and a second central area is determined based on the second marker brightness value.

[0096] Specifically, the preset pixel block in the embodiment of the present invention is a 16x16 pixel matrix. This division method not only ensures processing accuracy, but also effectively improves the efficiency of image processing.

[0097] Specifically, in the embodiment of the present invention, the first central area is the corresponding retinal area in the initial OCT image, the second central area is the corresponding choroid area in the initial OCT image, and the central area includes at least one pixel block.

[0098] Specifically, the embodiments of the present invention more accurately identify the retinal and choroidal regions in the image through a threshold determination method based on historical data, thereby improving the accuracy and reliability of segmentation, simplifying the complex OCT image into several central areas through gridding and threshold segmentation, greatly simplifying the subsequent analysis and processing procedures, and improving processing efficiency. The mean of historical data is used as the threshold to reduce the impact of noise and individual differences on the segmentation results to a certain extent, thereby enhancing the robustness of the algorithm.

[0099] Specifically, the step of extracting the initial retinal image and the initial choroidal image based on the processing result includes:

[0100] Determining a retinal brightness range based on a historical maximum value and a historical minimum value of a plurality of historical brightness values ​​corresponding to the historical retinal image;

[0101] Determine a choroid brightness range based on a historical maximum value and a historical minimum value of a plurality of historical brightness values ​​corresponding to the historical choroid image;

[0102] determining an initial retinal region based on the first central region and the retinal brightness range;

[0103] determining an initial choroid region based on the second central region and the choroid brightness range;

[0104] Extracting continuous areas in the initial retinal area to obtain the initial retinal image;

[0105] Continuous regions in the initial choroid region are extracted to obtain the initial choroid image.

[0106] Specifically, the determining of the initial retinal area based on the first central area and the retinal brightness range in the embodiment of the present invention includes:

[0107] Expanding the first central area, identifying brightness values ​​around the first central area, and adding the surrounding brightness values ​​to an area within the retinal brightness range as the initial retinal area;

[0108] When the surrounding brightness value does not fall within the retinal brightness range, the expansion of the first central area is stopped.

[0109] Specifically, the embodiment of the present invention also includes filtering the initial retinal image through a smoothing filter after extracting the initial retinal image to eliminate noise and detail texture in the image and improve the signal-to-noise ratio and clarity of the image.

[0110] Specifically, the step of performing algorithm fitting on retinal digital signals through a multi-scale context aggregation network includes:

[0111] Establish operator fitting network;

[0112] Acquire a number of historical retinal images and their corresponding historical image signals, and divide them into a training data set and a test data set;

[0113] Preprocessing the training data set based on the auxiliary function, performing training based on the preprocessed training data set and the multi-scale CAN layer to obtain an initial operator fitting network;

[0114] Constructing a test operator fitting network based on the test data set and the initial operator fitting network;

[0115] Algorithmic fitting of retinal image signals through a multi-scale context aggregation network;

[0116] Inputting the retinal image signal into the test operator fitting network to obtain a fitting feature vector;

[0117] The fitting feature vector is converted into the fitting image signal, and image reconstruction is performed based on the fitting image signal to obtain a fitting retinal image.

[0118] Specifically, the multi-scale context aggregation network (CAN) described in the embodiment of the present invention is a deep learning network structure that can capture context information of different scales and perform feature learning and fitting on images. During the training process, historical retinal images and their corresponding image signals are used as the data basis, and feature learning and network parameter optimization are performed through the multi-scale CAN layer, and finally an operator fitting network that can accurately fit the retinal image signal is obtained.

[0119] Specifically, the data volume ratio of the training data set and the test data set described in the embodiment of the present invention is 3:1.

[0120] Specifically, the auxiliary function described in the embodiment of the present invention is bilateralFilterDataset.

[0121] See also Figure 2 As shown, it is a schematic diagram of the operator fitting network structure.

[0122] See also Figure 3 As shown, the step of reconstructing the fitted retinal image based on the retinal curvature in the fitted retinal image and a preset curvature value includes:

[0123] Step S410, identifying the retinal edge in the fitted retinal image, calculating the curvature of the retinal edge, and obtaining a number of actual curvature values;

[0124] Step S420, comparing the actual bending radian values ​​with the preset bending radian values ​​to obtain a plurality of radian difference values;

[0125] Step S430, elastically transforming the fitted retinal image according to the plurality of curvature difference values, so that the actual curvature value of the fitted retinal image matches the preset curvature value;

[0126] Step S440, performing pixel-level filling on the elastically transformed image by using an interpolation algorithm to obtain the reconstructed retinal image.

[0127] Specifically, the embodiments of the present invention ensure accurate description of retinal morphology by accurately identifying the retinal edge and calculating its curvature. The comparison and adjustment with the preset curvature value makes the reconstructed retinal image closer to the ideal or standardized morphology, thereby improving the accuracy and reliability of image analysis. The automated curvature matching and image reconstruction process speeds up the image processing. Elastic transformation and pixel-level interpolation filling can smoothly adjust the morphology of the retinal image, reduce image distortion and blurring caused by shooting angle, eye movement and other factors, improve the visual effect of the image, and provide a better basis for subsequent image processing and analysis. Using the preset curvature value as a standard, retinal images from different patients or the same patient at different time points are reconstructed, so that these images are more consistent in morphology, which helps to make more accurate comparisons and analyses in different situations and improve the consistency of the results. The reconstructed retinal image has a better standardized morphology, so that image-based analysis and algorithms can run more efficiently.

[0128] Specifically, the preset curvature value described in the embodiment of the present invention is the average curvature of a normal fundus retina, which is obtained based on statistics of a large number of healthy fundus images.

[0129] Specifically, the embodiment of the present invention can identify the retinal edge in the fitted retinal image through an edge detection algorithm.

[0130] Specifically, the step of calculating the curvature of the retinal edge includes:

[0131] Step S411, acquiring an edge pixel point set based on the retinal edge;

[0132] Step S412, fitting the edge pixel point set based on a curve fitting algorithm to obtain an edge fitting curve;

[0133] Step S413, calculating the actual curvature value corresponding to each point on the edge fitting curve, obtaining a plurality of actual curvature values, and using the plurality of actual curvature values ​​as a plurality of actual bending radian values.

[0134] Specifically, the embodiment of the present invention obtains a set of pixel points at the edge of the retina and applies a curve fitting algorithm to fit the set, thereby accurately depicting the outline of the retinal edge, improving the precision and accuracy of the edge outline, calculating the actual curvature value corresponding to each point on the edge fitting curve, and quantifying the curvature of the retinal edge into a specific numerical value. The quantification process enables direct comparison and analysis of the curvatures between different images, providing strong data support for subsequent image reconstruction and diagnosis, and more accurately evaluating the morphological changes of the retina based on the precise curvature values, which helps to improve the accuracy and consistency of the results. Through the preset curve fitting algorithm and curvature calculation method, the processing process between different images is ensured to be consistent and standardized, thereby improving processing efficiency.

[0135] Specifically, the curve fitting algorithm described in the embodiment of the present invention may be least squares fitting or polynomial fitting.

[0136] Specifically, the step of calculating the actual curvature value of the edge fitting curve in the embodiment of the present invention includes:

[0137] Based on the continuous pixel points of the edge fitting curve, the slope between adjacent pixel points is calculated;

[0138] Take some of these slopes as actual curvature values.

[0139] Specifically, the step of elastically transforming the fitted retinal image according to the radian difference value includes:

[0140] Determine a plurality of deformation parameters corresponding to each pixel point at the edge of the retina based on the plurality of curvature difference values;

[0141] Applying the deformation parameters to the pixel points corresponding to the retinal edge to locally deform the retinal edge;

[0142] The deformation parameters are gradually adjusted in an iterative manner until the actual bending curvature value matches the preset bending curvature value.

[0143] Specifically, an embodiment of the present invention determines a number of deformation parameters corresponding to each pixel point at the edge of the retina based on a number of the curvature difference values, wherein any of the deformation parameters = |the preset curvature value - the actual curvature value|.

[0144] Specifically, the step of gradually adjusting the deformation parameters in an iterative manner in the embodiment of the present invention includes:

[0145] Calculate the current deformation parameters according to any actual bending curvature value and the preset bending curvature value, and set the upper limit of the number of iterations and the convergence threshold;

[0146] Apply the current deformation parameters to each pixel and calculate the position of the retinal edge after deformation;

[0147] Evaluate the degree of match between the deformed retinal edge and the preset curvature value. If the convergence threshold is not reached, adjust the current deformation parameters according to the curvature difference value.

[0148] The above steps are repeated until the upper limit of the number of iterations is reached or the degree of matching between the deformed retinal edge and the preset curvature value meets the convergence condition.

[0149] Specifically, the upper limit of the number of iterations in the embodiment of the present invention is 10 times, and the convergence threshold is 0.1.

[0150] Specifically, the step of performing pixel-level filling on the elastically transformed image by using an interpolation algorithm includes:

[0151] Determine the missing pixel region in the elastically transformed image;

[0152] According to the pixel values ​​around the missing pixel area, the pixel value of the missing pixel is calculated by an interpolation algorithm;

[0153] The calculated pixel values ​​are filled into the missing pixel area to complete the pixel-level filling of the image.

[0154] Specifically, the interpolation algorithm described in the embodiment of the present invention can adopt any one of a variety of interpolation methods such as linear interpolation, bilinear interpolation, cubic spline interpolation, etc., and selects a suitable interpolation algorithm for pixel-level filling according to specific image quality and reconstruction requirements.

[0155] See also Figure 4 As shown, the step of drawing a fundus three-dimensional structure image based on the reconstructed retinal image and the initial choroidal image includes:

[0156] Step S510, identifying a first key point and a second key point in the reconstructed retinal image and the initial choroidal image respectively;

[0157] Step S520, matching the first key point and the second key point, and determining an image matching point based on the matching result;

[0158] Step S530, detecting a first target contour in the reconstructed retinal image, and detecting a second target contour in the initial choroidal image;

[0159] Step S540, calculating a first minimum bounding rectangle based on the first object outline, and calculating a second minimum bounding rectangle based on the second object outline;

[0160] Step S550, defining four correction vertex positions of the initial choroidal image according to the historical fundus three-dimensional structure image characteristics;

[0161] Step S560, calculating the perspective transformation matrix based on the corrected vertex positions and the image matching points, performing perspective transformation on the initial choroidal image based on the perspective transformation matrix, so as to align the initial choroidal image with the reconstructed retinal image, and drawing a fundus three-dimensional structure image based on the aligned initial choroidal image and the reconstructed retinal image.

[0162] Specifically, the embodiment of the present invention can detect key points in two images by using a SIFT (Scale Invariant Feature Transform) algorithm.

[0163] Specifically, the embodiment of the present invention matches the first key point and the second key point by using a FLANN (Fast Nearest Neighbor Library) matcher to match the key points of the two images.

[0164] Specifically, the step of fitting the initial OCTA image with the fundus three-dimensional structure image with the optic nerve head as the center includes:

[0165] Identify the position of the nerve papilla in the initial OCTA image based on a feature detection algorithm to obtain a first position, and identify the position of the nerve papilla in the fundus three-dimensional structure image based on a feature detection algorithm to obtain a second position;

[0166] Acquire a two-dimensional projection surface of the three-dimensional fundus structure image, identify four vertex positions of the two-dimensional projection surface, and calculate a first spacing value between the second position and the four vertex positions;

[0167] Identify the edge contour of the initial OCTA image, select a third minimum circumscribed rectangle based on the edge contour, identify four vertex positions of the third minimum circumscribed rectangle, and calculate a second spacing value between the first position and the four vertex positions;

[0168] Aligning the first position and the second position based on an image transformation technology, and rotating the initial OCTA image so that the first spacing value is the same as the second spacing value;

[0169] The curvature in the initial OCTA image is aligned with the curvature in the fundus three-dimensional structure image to obtain a final fundus three-dimensional image.

[0170] Specifically, in the embodiment of the present invention, the initial OCTA image is defined as imgD, and the fundus three-dimensional structure image is defined as imgC;

[0171] Read in and resize images imgC and imgD;

[0172] Locate the optic nerve head position in images imgC and imgD, the four vertices on the image enface surface, and the distance between the optic nerve head and the four vertices on the image enface surface;

[0173] Align the positions of the optic nerve head in images imgC and imgD;

[0174] Rotate the imgD image so that the distances between the optic nerve head and the four vertices on the enface surface of the imgD image and the imgC image are the same, so as to achieve the matching of the positions of the four vertices on the enface surface of the imgD image and the imgC image;

[0175] The OpenCV algorithm is used to adjust the curvature of the imgD image so that imgD fits perfectly with imgC.

[0176] Specifically, the step of defining the positions of the four correction vertices of the initial choroidal image according to the historical fundus three-dimensional structure image characteristics includes:

[0177] Identify historical choroidal edge contours in historical fundus three-dimensional structural images;

[0178] The mean edge width was calculated based on the historical choroidal edge contour;

[0179] Two correction vertex positions are defined outside and inside the historical choroidal edge contour based on the average edge width, so that the distance between the two inner correction vertices and the distance between the two outer correction vertices are both in a preset proportion to the average edge width.

[0180] Specifically, the embodiments of the present invention more accurately determine the shape and size of the choroid by identifying the choroidal edge contour in the historical fundus three-dimensional structure image and calculating the average edge width. The correction vertex position defined based on the average edge width can ensure that the correction process matches the actual morphology of the choroid, thereby improving the precision and accuracy of the correction. Through reasonable vertex position definition, it is ensured that the reconstructed choroidal image is closer to the actual situation, reducing image distortion and errors caused by position deviation.

[0181] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0182] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A three-dimensional image processing method for fundus examination, characterized in that: include: Acquire an initial OCT image and an initial OCTA image of the target to be detected; Processing the initial OCT image based on threshold segmentation, and extracting an initial retinal image and an initial choroidal image based on the processing result; Identify the image signal corresponding to the initial retinal image, obtain the retinal image signal, perform algorithm fitting on the retinal image signal through a multi-scale context aggregation network to obtain a fitted image signal, and obtain a fitted retinal image based on the fitted image signal; Reconstructing the fitted retinal image based on the retinal curvature in the fitted retinal image and a preset curvature value to obtain a reconstructed retinal image; Drawing a fundus three-dimensional structure image based on the reconstructed retinal image and the initial choroidal image; Identify the optic nerve head in the initial OCTA image and the fundus three-dimensional structure image, and fit the initial OCTA image and the fundus three-dimensional structure image with the optic nerve head as the center to obtain a final fundus three-dimensional image; The step of reconstructing the fitted retinal image based on the retinal curvature in the fitted retinal image and a preset curvature value comprises: Identifying the retinal edge in the fitted retinal image, calculating the curvature of the retinal edge, and obtaining a number of actual curvature values; Comparing the actual bending radian values ​​with the preset bending radian values ​​to obtain a plurality of radian difference values; elastically transforming the fitting retinal image according to the plurality of curvature difference values ​​so that the actual curvature value of the fitting retinal image matches the preset curvature value; The elastically transformed image is filled at the pixel level by an interpolation algorithm to obtain the reconstructed retinal image.

2. The three-dimensional image processing method for fundus examination according to claim 1, characterized in that: The step of processing the initial OCT image based on threshold segmentation comprises: Dividing the initial OCT image into a plurality of grids by preset pixel blocks; Identify the brightness values ​​corresponding to the grids, and obtain actual brightness values; Determine the average of several historical brightness values ​​corresponding to the historical retinal image as the retinal brightness threshold, and determine the average of several historical brightness values ​​corresponding to the historical choroidal image as the choroid brightness threshold; Mark the actual brightness value with the smallest difference with the retinal brightness threshold value among the plurality of actual brightness values ​​to obtain a first marked brightness value, and mark the actual brightness value with the smallest difference with the choroidal brightness threshold among the plurality of actual brightness values ​​to obtain a second marked brightness value; A first central area is determined based on the first marker brightness value, and a second central area is determined based on the second marker brightness value.

3. The three-dimensional image processing method for fundus examination according to claim 2, characterized in that: The step of extracting the initial retinal image and the initial choroidal image based on the processing result comprises: Determining a retinal brightness range based on a historical maximum value and a historical minimum value of a plurality of historical brightness values ​​corresponding to the historical retinal image; Determine a choroid brightness range based on a historical maximum value and a historical minimum value of a plurality of historical brightness values ​​corresponding to the historical choroid image; determining an initial retinal region based on the first central region and the retinal brightness range; determining an initial choroid region based on the second central region and the choroid brightness range; Extracting continuous areas in the initial retinal area to obtain the initial retinal image; Continuous regions in the initial choroid region are extracted to obtain the initial choroid image.

4. The three-dimensional image processing method for fundus examination according to claim 3, characterized in that: The step of performing algorithm fitting on the retinal digital signal through the multi-scale context aggregation network comprises: Establish operator fitting network; Acquire a number of historical retinal images and their corresponding historical image signals, and divide them into a training data set and a test data set; Preprocessing the training data set based on the auxiliary function, performing training based on the preprocessed training data set and the multi-scale CAN layer to obtain an initial operator fitting network; Constructing a test operator fitting network based on the test data set and the initial operator fitting network; Algorithmic fitting of retinal image signals through a multi-scale context aggregation network; Inputting the retinal image signal into the test operator fitting network to obtain a fitting feature vector; The fitting feature vector is converted into the fitting image signal, and image reconstruction is performed based on the fitting image signal to obtain a fitting retinal image.

5. The three-dimensional image processing method for fundus examination according to claim 4, characterized in that: The step of calculating the curvature of the retinal edge comprises: Acquire an edge pixel point set based on the retinal edge; Fitting the edge pixel point set based on a curve fitting algorithm to obtain an edge fitting curve; The actual curvature values ​​corresponding to the points on the edge fitting curve are calculated to obtain a plurality of actual curvature values, and the plurality of actual curvature values ​​are used as the plurality of actual bending radian values.

6. The three-dimensional image processing method for fundus examination according to claim 5, characterized in that: The step of elastically transforming the fitted retinal image according to the radian difference value comprises: Determine a plurality of deformation parameters corresponding to each pixel point at the edge of the retina based on the plurality of curvature difference values; Applying the deformation parameters to the pixel points corresponding to the retinal edge to locally deform the retinal edge; The deformation parameters are gradually adjusted in an iterative manner until the actual bending curvature value matches the preset bending curvature value.

7. The three-dimensional image processing method for fundus examination according to claim 6, characterized in that: The step of drawing a fundus three-dimensional structure image based on the reconstructed retinal image and the initial choroidal image comprises: identifying a first key point and a second key point in the reconstructed retinal image and the initial choroidal image, respectively; Matching the first key point and the second key point, and determining an image matching point based on the matching result; Detecting a first target contour in the reconstructed retinal image and detecting a second target contour in the initial choroidal image; Calculate a first minimum bounding rectangle based on the first object outline, and calculate a second minimum bounding rectangle based on the second object outline; Defining four correction vertex positions of the initial choroidal image according to the historical fundus three-dimensional structure image characteristics; A perspective transformation matrix is ​​calculated based on the corrected vertex positions and the image matching points, and the initial choroidal image is perspectively transformed based on the perspective transformation matrix to align the initial choroidal image with the reconstructed retinal image, so as to draw a three-dimensional fundus structure image based on the aligned initial choroidal image and the reconstructed retinal image.

8. The three-dimensional image processing method for fundus examination according to claim 7, characterized in that: The step of defining the positions of the four correction vertices of the initial choroidal image according to the historical fundus three-dimensional structure image characteristics comprises: Identify historical choroidal edge contours in historical fundus three-dimensional structural images; The mean edge width was calculated based on the historical choroidal edge contour; Two correction vertex positions are defined outside and inside the historical choroidal edge contour based on the average edge width, so that the distance between the two inner correction vertices and the distance between the two outer correction vertices are both in a preset proportion to the average edge width.

9. The three-dimensional image processing method for fundus examination according to claim 8, characterized in that: The step of fitting the initial OCTA image with the fundus three-dimensional structure image with the optic nerve head as the center comprises: Identify the position of the nerve papilla in the initial OCTA image based on a feature detection algorithm to obtain a first position, and identify the position of the nerve papilla in the fundus three-dimensional structure image based on a feature detection algorithm to obtain a second position; Acquire a two-dimensional projection surface of the three-dimensional fundus structure image, identify four vertex positions of the two-dimensional projection surface, and calculate a first spacing value between the second position and the four vertex positions; Identify the edge contour of the initial OCTA image, select a third minimum circumscribed rectangle based on the edge contour, identify four vertex positions of the third minimum circumscribed rectangle, and calculate a second spacing value between the first position and the four vertex positions; Aligning the first position and the second position based on an image transformation technology, and rotating the initial OCTA image so that the first spacing value is the same as the second spacing value; The curvature in the initial OCTA image is aligned with the curvature in the fundus three-dimensional structure image to obtain a final fundus three-dimensional image.

Images