Automatic positioning method for feature points of fundus optical coherence tomography image

Abstract

The invention relates to the technical field of retina analysis, and discloses a fundus optical coherence tomography image feature point automatic positioning method, which comprises the following steps: acquiring optical coherence tomography images of a fundus disease patient at different disease course stages, constructing an image database containing a training set and a verification set, and determining the feature points of the fundus optical coherence tomography image on the basis of the image database. And carrying out feature layer segmentation on the image by adopting a U-Net convolutional neural network. And determining a blood vessel shadow position in the fundus optical coherence tomography image and acquiring a boundary of the blood vessel shadow position based on the pigment epithelium gray scale curve. And determining the position of a seed point and the size of a subarea by combining an important layer boundary and a vessel shadow boundary, and assisting digital image correlation to realize retina full-field deformation measurement. According to the method, through deep fusion of deep learning and an image processing technology, the problem of automatic selection of the seed points in the fundus OCT image is solved, the efficiency, precision and reliability of retinal deformation measurement are remarkably improved, and a powerful technical tool is provided for mechanism research and clinical diagnosis and treatment of fundus diseases.

Description

Technical Field

[0001] The present invention relates to the technical field of retinal analysis, and particularly to an automatic positioning method for feature points of fundus optical coherence tomography images. Background Art

[0002] Fundus diseases such as retinal vein occlusion, age-related macular degeneration, and diabetic retinopathy seriously threaten the vision of middle-aged and elderly people. Optical coherence tomography technology can provide clear retinal tomographic images similar to tissue sections for ophthalmologists and has been widely used in the diagnosis of fundus diseases. Currently, doctors mainly evaluate the patient's condition based on geometric parameters (such as macular thickness, volume, etc.) or structural changes (such as disordered inner retinal structure, hyper-reflective foci, etc.) that can be directly measured in fundus optical coherence tomography images. Therefore, most research on fundus optical coherence tomography image processing methods focuses on the following two aspects: one is the precise image segmentation and geometric size calculation of disease-related imaging markers; the other is the evaluation of the condition or prediction of the treatment effect based on imaging markers using artificial intelligence technology.

[0003] However, during the development of fundus diseases, the retina often undergoes complex deformation behaviors. For example, during macular edema, the retina undergoes complex full-field deformation under the action of various factors such as being squeezed by fluid accumulation and being pulled by Müller cells, which may cause irreversible damage to the overall structure and function of the retina. Existing imaging indicators cannot describe this complex physical process. Artificial intelligence-assisted doctors in diagnosis or determining treatment plans often lack interpretability and rarely improve doctors' understanding of the pathogenesis of the disease. Therefore, there is an urgent need to develop a method for measuring the full-field deformation of the retina during the development of fundus diseases based on fundus optical coherence tomography images.

[0004] Due to its advantages such as non-contact, full-field measurement, and high precision, digital image correlation technology has been widely used in the measurement of deformation fields in the fields of biomechanics and medicine. However, due to problems such as the inability to prepare speckle markers, poor image quality, and large deformations in fundus optical coherence tomography images, it is often necessary to select pixel points with large gray-level gradients and invariant gray levels in the reference image as seed points to assist digital image correlation in realizing the full-field deformation measurement of the retina in fundus diseases. The manual selection of seed points will reduce the efficiency of deformation measurement. Therefore, the present invention proposes an automatic positioning method for feature points of fundus optical coherence tomography images to quickly provide suitable seed points for subsequent analysis of retinal deformation using digital image correlation. Therefore, the present invention proposes an automatic positioning method for feature points of fundus optical coherence tomography images to solve the above problems. Summary of the Invention

[0005] An automatic positioning method for feature points of fundus optical coherence tomography images proposed by the present invention solves the problems in the background technology.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] An automatic positioning method for feature points of fundus optical coherence tomography images includes the following steps:

[0008] S1. Collect optical coherence tomography images of patients with fundus diseases at different disease stages, and construct an image database including a training set and a validation set;

[0009] S2. Based on the image database, use a U-Net convolutional neural network to perform feature layer segmentation on the optical coherence tomography images. The segmentation targets include the layer boundaries between the vitreous body and the nerve fiber layer, the nerve fiber layer and the ganglion cell layer, the inner plexiform layer and the inner nuclear layer, the outer plexiform layer and the outer nuclear layer, the myoid area and the ellipsoid area, and the pigment epithelium layer and the choroid layer;

[0010] S3. Use the Canny algorithm to extract the boundaries of the segmented feature layers;

[0011] S4. Extract the gray curve at the junction of the pigment epithelium layer and the choroid layer, perform smoothing processing using a sliding window mean filter, and determine the position of the blood vessel shadow boundary based on the gray discontinuity;

[0012] S5. Calculate the intersection points between the layer boundaries extracted in step S3 and the blood vessel shadow boundary determined in step S4, and select the intersection points as the seed points for digital image correlation analysis.

[0013] Optionally, the construction of the image database includes:

[0014] Adopt a radial scanning mode to collect optical coherence tomography images of 18 cross-sections centered on the macula;

[0015] Divide the images into a training set and a validation set according to a ratio of 9:1, ensuring that the training set and the validation set contain images of different disease types and disease stages.

[0016] Optionally, the training process of the U-Net convolutional neural network includes:

[0017] Label the segmentation targets into five categories: the nerve fiber layer, the ganglion cell layer to the inner plexiform layer, the inner nuclear layer to the outer plexiform layer, the outer nuclear layer to the myoid area, and the ellipsoid area to the pigment epithelium layer;

[0018] Use the Adam optimizer to train the model, with an initial learning rate of 0.001, and the learning rate decays to 0.9 times the original value every 5 training cycles;

[0019] Evaluate the model performance by the Dice coefficient, accuracy, and recall rate of the validation set.

[0020] Optionally, the determination method of the grayscale discontinuity is as follows:

[0021] Set the sliding window length to 7 pixels, and calculate the difference in the sum of grayscale values in the 3-pixel regions at both ends of the window.

[0022] When the difference exceeds the threshold of 60, determine that this position is the intersection of the blood vessel shadow boundary and the pigment epithelium layer boundary.

[0023] Optionally, the application of the seed points includes:

[0024] Select a sub-region containing at least two different feature layers, and the size of the sub-region is 81 pixels × 81 pixels.

[0025] Based on the seed points, use the digital image correlation algorithm to calculate the full-field retinal deformation field of optical coherence tomography images at different times.

[0026] Optionally, the method further includes:

[0027] Determine the blood vessel shadow boundary by the vertical projection method, and use the intersection of the vertical line and the feature layer boundary as the seed point.

[0028] The beneficial effects of the present invention are:

[0029] Automatically segment each feature layer of the retina using a deep learning model, avoiding the cumbersome process of manual annotation, accurately locating the intersection of the blood vessel shadow boundary and the interlayer boundary, and greatly shortening the time for selecting seed points.

[0030] The segmentation targets cover the key interfaces of the vitreous - nerve fiber layer and the ellipsoid zone - pigment epithelium layer, ensuring that the sub-region contains at least two different feature layers, reducing the noise interference of a single layer. The blood vessel shadow remains vertical during deformation, and its intersection with the interlayer boundary is used as the seed point, effectively reducing the influence of the image decorrelation effect.

[0031] Combine the Canny algorithm to extract the interlayer boundary after segmentation, suppress noise while retaining structural details, use the Adam optimizer in the U-Net training, and verify with multiple metrics such as the Dice coefficient and recall rate to ensure the high precision of the segmentation model.

[0032] Realize the continuous calculation of the full-field retinal deformation field, reveal the biomechanical mechanism of disease development, and the automatic method eliminates the subjective differences of manual operations.

[0033] Through the deep integration of deep learning and image processing technologies, the present invention solves the problem of automatic selection of seed points in fundus OCT images, significantly improving the efficiency, accuracy, and reliability of retinal deformation measurement, and providing a powerful technical tool for the mechanism research and clinical diagnosis and treatment of fundus diseases. Description of the Drawings

[0034] Figure 1 is the optical coherence tomography image acquisition method.

[0035] Figure 2 is the segmentation result of the U-Net network.

[0036] Figure 3 is a schematic diagram of the intersection points of the feature layer boundary and the blood vessel shadow.

[0037] Figure 4 is a schematic diagram of seed point selection.

[0038] Figure 5 is a schematic diagram of the A-Scan direction deformation field calculated by digital image correlation. Detailed Embodiment

[0039] The following describes in detail the specific embodiments of the present invention with reference to the accompanying drawings. In this embodiment, the fundus optical coherence tomography images of patients with retinal vein occlusion are taken as an example to illustrate the implementation steps of the feature point automatic positioning method, but the application of the present invention is not limited thereto.

[0040] Step 1. Image data acquisition and database construction

[0041] 101. Adopt the clinical radial scanning mode (as shown in Figure 1 ), with the patient's macula as the center, perform optical coherence tomography on 18 cross-sections to cover the retinal structure of the target area. The scanning parameters are set to the clinical general standard to ensure that the image resolution meets the requirements of subsequent analysis.

[0042] 102. Prospectively include 20 patients newly diagnosed with retinal vein occlusion, and continuously collect fundus optical coherence tomography images at different stages during their disease courses, obtaining a total of 400 images.

[0043] 103. Divide the 400 images into a training set (360 images) and a validation set (40 images) according to a ratio of 9:1, ensuring that the training set and the validation set contain images of different disease types and disease course stages to improve the generalization ability of the model.

[0044] Step 2. Training of the feature layer segmentation model

[0045] 201. Use the Labelme software to label the training set images, and the labeling targets are five types of feature layer interfaces:

[0046] vitreous - nerve fiber layer interface;

[0047] nerve fiber layer - ganglion cell layer interface;

[0048] inner plexiform layer - inner nuclear layer interface;

[0049] outer plexiform layer - outer nuclear layer interface;

[0050] ellipsoid zone - retinal pigment epithelium layer interface; annotation example is Figure 2 as shown.

[0051] 202. Construct a U - Net convolutional neural network model. The encoder part extracts image features through convolution and downsampling, and the decoder part restores the resolution and outputs the segmentation result through upsampling and skip connections. Automatically segment each characteristic layer of the retina (such as the nerve fiber layer, retinal pigment epithelium layer, etc.) using the deep learning model to avoid the cumbersome process of manual annotation.

[0052] 203. Training parameter settings:

[0053] Optimizer: Adam, initial learning rate 0.001;

[0054] Learning rate decay strategy: Decay to 0.9 times the original value every 5 training epochs;

[0055] Batch size: 16 images;

[0056] Training epochs: 100 epochs;

[0057] Validation metrics: Dice coefficient, accuracy, recall.

[0058] 204. After training is completed, the segmentation results of the model on the validation set are Figure 2 as shown, and the segmentation accuracy meets the requirements of subsequent analysis.

[0059] Step 3. Feature layer boundary extraction and blood vessel shadow localization

[0060] 301. Input the fundus optical coherence tomography image to be analyzed into the trained U - Net model to automatically segment each characteristic layer interface.

[0061] 302. Use the Canny algorithm to extract the inter - layer boundary after segmentation (as Figure 3 shown), the boundary is clear and continuous, reducing noise interference.

[0062] 303. Extract the gray - scale curve at the junction of the retinal pigment epithelium layer and the choroid layer, and use a sliding window mean filter with a window length of 7 pixels to smooth the gray - scale curve to effectively remove high - frequency noise.

[0063] 304. On the smoothed grayscale curve, detect grayscale discontinuity points one by one with a 7-pixel window:

[0064] Calculate the difference in the sum of grayscale values in the 3-pixel regions at both ends of the window.

[0065] If the difference exceeds the preset threshold of 60, determine that this point is the intersection of the blood vessel shadow boundary and the pigment epithelium boundary (as Figure 3 shown by the marked points). By sliding window mean filtering and grayscale discontinuity detection (threshold 60), accurately locate the intersection of the blood vessel shadow boundary and the interlayer boundary, greatly shortening the time for selecting seed points.

[0066] Step 4. Determination of seed points and calculation of deformation field

[0067] 401. Calculate the intersections of the interlayer boundaries extracted in step 3 and the determined blood vessel shadow boundaries to obtain the coordinates of the seed points (as Figure 4 shown).

[0068] 402. Centered on the seed points, select an appropriate sub-region size according to the extracted important interlayer boundary information, ensuring that the sub-region contains at least two adjacent different characteristic layers (such as the nerve fiber layer and the ganglion cell layer) to improve the anti-noise ability of the digital image correlation algorithm. In the example, a sub-region with a size of 81 pixels × 81 pixels is selected.

[0069] 403. Group the images collected at different disease stages of the same patient according to the scanning direction. Taking the first collected image as the reference image, calculate the full-field deformation field of the retina based on the seed points using the digital image correlation algorithm (as Figure 5 shown).

[0070] Through the above method, the automatic measurement of the retinal deformation field during the course of retinal vein occlusion patients has been successfully achieved. Compared with the traditional manual point selection, this method improves the efficiency of seed point selection, and the calculation results of the deformation field are consistent with the trend of retinal structure changes observed clinically, verifying the effectiveness and reliability of the method.

[0071] It should be noted that: the segmentation targets cover key interfaces such as the vitreous - nerve fiber layer and the ellipsoid zone - pigment epithelium layer, ensuring that the sub-region (81 pixels × 81 pixels) contains at least two different characteristic layers to reduce the noise interference of a single layer. The blood vessel shadow remains vertical during deformation, and its intersection with the interlayer boundary is used as the seed point, effectively reducing the influence of the image decorrelation effect. High-precision seed points can still be stably output in the case of low image quality or large retinal deformations.

[0072] In this embodiment, the Canny algorithm is combined to extract the interlayer boundary after segmentation, suppressing noise while preserving structural details. During the training of the U-Net, the Adam optimizer is adopted (initial learning rate 0.001, decaying by 0.9 times every 5 epochs), and multi-index verification such as the Dice coefficient and recall rate is combined to ensure the high precision of the segmentation model.

[0073] Furthermore, by collecting OCT images at different disease stages (such as 1 month, 3 months, etc. of macular edema), the continuous calculation of the full-field deformation field of the retina is realized by combining the DIC algorithm, revealing the biomechanical mechanism of disease development. The automatic method eliminates the subjective differences of manual operations and provides a unified standard for multi-center research. It can be extended to the deformation analysis of diseases such as diabetic retinopathy and age-related macular degeneration, assisting doctors in formulating personalized treatment plans. The method covers the entire process of data acquisition, model training, boundary extraction, and deformation calculation, and can be flexibly adapted to different OCT devices and clinical needs. Based on a conventional GPU for training the model, only ordinary computing resources are required in the inference stage, making it suitable for popularization and application in hospitals and research institutions.

[0074] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An automatic positioning method for feature points of fundus optical coherence tomography images, characterized in that It includes the following steps: S1. Collect optical coherence tomography (OCT) images of fundus disease patients at different disease stages, and construct an image database containing a training set and a validation set; S2. Based on the image database, use a U-Net convolutional neural network to perform feature layer segmentation on the OCT images. The segmentation targets include the interfaces between the vitreous body and the nerve fiber layer, the nerve fiber layer and the ganglion cell layer, the inner plexiform layer and the inner nuclear layer, the outer plexiform layer and the outer nuclear layer, the myoid area and the ellipsoid zone, and the pigment epithelium layer and the choroid layer; S3. Use the Canny algorithm to extract the boundaries of the segmented feature layers; S4. Extract the gray curve at the junction of the pigment epithelium layer and the choroid layer, perform smoothing processing using a sliding window mean filter, and determine the position of the blood vessel shadow boundary based on the gray discontinuity; S5. Calculate the intersection points between the layer boundaries extracted in step S3 and the blood vessel shadow boundary determined in step S4, and select the intersection points as the seed points for digital image correlation analysis.

2. The automatic positioning method for characteristic points of fundus optical coherence tomography images according to claim 1, wherein The construction of the image database includes: Adopt a radial scanning mode to collect OCT images of 18 cross-sections centered on the macula; Divide the images into a training set and a validation set at a ratio of 9:1 to ensure that the training set and the validation set contain images of different disease types and disease stages.

3. The automatic positioning method of characteristic points of fundus optical coherence tomography images according to claim 1, characterized in that, The training process of the U-Net convolutional neural network includes: Label the segmentation targets into five categories: the nerve fiber layer, the ganglion cell layer to the inner plexiform layer, the inner nuclear layer to the outer plexiform layer, the outer nuclear layer to the myoid area, and the ellipsoid zone to the pigment epithelium layer; Use the Adam optimizer to train the model, with an initial learning rate of 0.001, and the learning rate decays to 0.9 times the original value every 5 training epochs; Evaluate the model performance through the Dice coefficient, accuracy, and recall rate of the validation set.

4. The automatic positioning method for characteristic points of fundus optical coherence tomography images according to claim 1, characterized in that, The determination method of the gray discontinuity is: Set the sliding window length to 7 pixels, and calculate the difference in the sum of the gray values of the 3-pixel regions at the left and right ends of the window; When the difference exceeds the threshold of 60, determine that this position is the intersection point of the blood vessel shadow boundary and the pigment epithelium boundary.

5. The automatic positioning method for characteristic points of fundus optical coherence tomography images according to claim 1, characterized in that, The application of the seed points includes: Select a sub-region containing at least two different feature layers, and the size of the sub-region is 81 pixels × 81 pixels; Based on the seed points, use the digital image correlation algorithm to calculate the full-field retinal deformation field of the OCT images at different times.

6. The automatic positioning method for characteristic points of fundus optical coherence tomography images according to claim 1, characterized in that, The method also includes: Determine the blood vessel shadow boundary by the perpendicular projection method, and use the intersection point of the perpendicular line and the feature layer boundary as the seed point.

Images