Deep learning fundus blood flow imaging method based on OCT amplitude and phase information

Abstract

The invention relates to a deep learning fundus blood flow imaging method based on OCT amplitude and phase information, and belongs to the field of fundus blood flow imaging. An artificial neural network is used for fusing multi-modal information from OCT, multiple labels from OCTA are used for optimizing the network, and finally high-precision reconstruction of a blood flow image is achieved. Compared with the prior art, the method has the following outstanding advantages that firstly, phase information which is not fully utilized in a traditional method is creatively integrated, and the microvessel recognition sensitivity is remarkably improved through an amplitude-phase bimodal feature collaborative learning mechanism; secondly, a blood flow texture-blood flow position double-label supervised learning strategy is creatively provided, blood vessel position labels are introduced in a function loss stage, the network is assisted to accurately position the blood flow position, the characterization capacity of the network on capillary and other small blood vessel structures is effectively enhanced, and blood flow microcirculation visualization is achieved; and thirdly, blood flow information can be reconstructed by using single OCT scanning data.

Description

Technical Field

[0001] The present invention relates to a deep learning fundus blood flow imaging method based on OCT amplitude and phase information, belonging to the field of fundus blood flow imaging. Background Art

[0002] In recent years, as the medical community continues to deepen its research on the relationship between fundus diseases and blood flow and vascular characterization, the patient's fundus blood flow microcirculation status has become an important examination indicator for diagnosing major fundus diseases, such as diabetic retinopathy and age-related macular degeneration. This has made fundus blood flow imaging an important part of the diagnosis of ophthalmic diseases. Optical coherence tomography (OCTA) technology has become an important means of fundus blood flow imaging due to its advantages such as fast imaging speed, high accuracy and no need for exogenous contrast agents. The core of OCTA technology is to use mathematical algorithms to process a set of optical coherence tomography (OCT) images repeatedly collected at the same tissue location, extract the time-correlation characteristics of the dynamic scattering signals, establish a differentiated characterization model of static tissue and dynamic blood flow, and finally obtain a fundus blood flow image.

[0003] However, existing OCTA technology systems still face critical technical bottlenecks when imaging fundus blood flow. This is primarily due to the fact that, to obtain reliable blood flow signal characteristics, multiple OCT images must be acquired from the same tissue location during OCTA imaging of fundus blood flow. This not only increases imaging time exponentially but also significantly increases the incidence of artifacts caused by patient motion, seriously affecting imaging quality. Reducing the number of repeated OCT image acquisitions while achieving high-quality fundus blood flow images has become a research focus for OCTA technology in the field of fundus blood flow imaging.

[0004] As deep learning technology demonstrates breakthrough potential in the field of medical image reconstruction, studies have demonstrated that deep learning can be used to reconstruct blood flow images from a small number of, or even a single, OCT image. Existing deep learning solutions include end-to-end learning methods and generative adversarial network learning methods. However, the model architectures used by existing methods are often limited to processing OCT amplitude information and fail to effectively utilize the blood flow vector characteristics contained in OCT phase information, which can easily lead to reduced sensitivity in identifying capillary branches and low-speed blood flow. This incomplete information utilization severely restricts the potential for technological transformation of deep learning technology in the field of OCTA. Therefore, developing an OCTA imaging technology that fully utilizes the amplitude and phase information of OCT images is of great significance for improving OCTA image quality and reducing the incidence of motion-induced artifacts. Summary of the Invention

[0005] The technical problem solved by the present invention is: to overcome the shortcomings of the existing technology and propose a deep learning fundus blood flow imaging method based on OCT amplitude and phase information.

[0006] The technical solution of the present invention is:

[0007] A deep learning fundus blood flow imaging method based on OCT amplitude and phase information includes the following steps:

[0008] Step 1: using an ophthalmic OCT device to collect multiple original OCT spectra at the same fundus position, and processing the multiple original OCT spectra collected at the same fundus position to obtain multiple OCT complex structure images, processing the multiple OCT complex structure images to obtain blood flow texture labels, and performing image post-processing on the obtained blood flow texture labels to obtain blood flow position labels;

[0009] In step 1, the method for processing the multiple original OCT spectra collected at the same fundus position to obtain multiple OCT complex images is as follows:

[0010] Each original OCT spectrum is sequentially resampled, windowed, dispersion compensated, and Fourier transformed to obtain a corresponding OCT complex structure image;

[0011] In step 1, the method for processing the obtained multiple OCT complex structure images to obtain blood flow texture labels is as follows:

[0012] Perform principal component analysis on multiple OCT complex structure images to obtain blood flow texture labels. Alternatively, perform principal component analysis on multiple OCT complex structure images and remove tailing artifacts to obtain blood flow texture labels.

[0013] In step 1, the method for performing image post-processing on the blood flow texture label to obtain the blood flow position label is:

[0014] The blood flow texture label is processed by filtering, denoising, and threshold segmentation to obtain a binary mask. The binary mask is then Gaussian filtered to obtain a blood flow position label. Alternatively, the binary mask is Gaussian filtered to remove trailing artifacts to obtain a blood flow position label.

[0015] Step 2: Create a fundus image dataset based on the multiple OCT complex structure images, blood flow texture labels, and blood flow position labels obtained in step 1;

[0016] In step 2, the generated fundus image data set includes a blood flow texture-blood flow position true value label system and an amplitude-phase feature input system.

[0017] Among them, the blood flow texture-blood flow position true value label system includes blood flow texture labels and blood flow position labels.

[0018] The amplitude-phase feature input system includes OCT amplitude image and phase image.

[0019] The OCT amplitude map and OCT phase map are obtained through the OCT complex structure map. The OCT amplitude map is the amplitude component of the OCT complex structure map; the OCT phase map is the phase component of the OCT complex structure map;

[0020] Step 3: Create a co-learning dual-modal blood flow image generation network;

[0021] In step 3, the co-learning dual-modal blood flow image generation network is composed of a dual-mode HRNet model and a tiled CNN model;

[0022] The dual-mode HRNet model is an encoder network composed of two parallel improved high-resolution network (HRNet) models;

[0023] The two parallel improved HRNet branches receive the fundus OCT amplitude image and fundus OCT phase image in the amplitude-phase feature input system as input, respectively, and output a high-resolution feature map by learning the features of the OCT amplitude image and OCT phase image in the input system;

[0024] The improved HRNet branch is a network with parallel resolution branches and repeated fusion, consisting of three parallel convolutional streams of different resolutions. The Split Shuffle Net serves as the backbone of the parallel convolutional streams; a Cross-Channel Feature Fusion Module is used at specific locations to fuse features from convolutional streams of different resolutions, and the fused feature maps are output to the tiled CNN model; and a Pixel Shuffle Net is used as the main method for upsampling.

[0025] The tiled CNN model is a decoder network consisting of a resolution-invariant Convolutional Neural Network (CNN) backbone, a shallow feature fusion module, and a deep feature fusion module.

[0026] The tiled CNN model receives high-resolution feature maps of different depths from the dual-mode HRNet model as input, learns the features therein, and ultimately outputs fundus blood flow position images and fundus blood flow images;

[0027] The shallow feature fusion module is a feature map fusion module composed of a channel attention mechanism, a spatial attention mechanism, and a fully connected layer (FC layer).

[0028] In the shallow feature fusion module, the shallow high-resolution feature maps output by the dual-mode HRNet model are first spliced ​​in the channel dimension. Then, the key channel weights are extracted through the channel attention mechanism. The extracted key channel weights are then reinforced with important spatial position information through the spatial attention mechanism.

[0029] The Deep Feature Fusion Module is a feature map fusion module composed of a non-local operation, a cascaded residual block, and a skip connection layer.

[0030] In the deep feature fusion module, the deep high-resolution feature map output by the dual-mode HRNet model first undergoes a non-local operation to establish position correlation on a global scale. A cascaded residual connection network is then used to perform residual connections to fuse the original features of the deep high-resolution feature map with the calculated global position correlation. Finally, a skip connection layer is used to perform skip connections to preserve multi-scale feature information.

[0031] Step 4, using the fundus image dataset generated in step 2 to train the co-learned dual-modal blood flow image generation network prepared in step 3, to obtain a trained co-learned dual-modal blood flow image generation network;

[0032] In step 4, the training of the co-learned dual-modal blood flow image generation network is based on a training method of multi-label joint loss supervision, which is:

[0033] First, the blood flow position label is compared with the blood flow position feature map output by the co-learned dual-modal blood flow image generation network through the adaptive wing loss function to obtain the loss L AW ;

[0034] Secondly, the blood flow texture label is used to compare with the blood flow image output by the co-learned dual-modal blood flow image generation network through the structural similarity loss function (Structural Similarity Index Measure), and the loss L is obtained. SSIM ;

[0035] Finally, according to the loss L AW With loss L SSIM Calculate the multi-label joint loss. The multi-label joint loss function is:

[0036] L=α·L AW +β·L SSIM

[0037] Among them, α and β are preset weight parameters;

[0038] Step 5: Using an ophthalmic OCT device, collect the original OCT spectrum of the fundus of the person to be tested, process the collected original OCT spectrum of the fundus of the person to be tested, and obtain an amplitude-phase feature input system. The obtained input feature map is input into the co-learned dual-modal blood flow image generation network trained in step 4 to obtain the fundus blood flow image of the person to be tested;

[0039] In step 5, the method for processing the collected original OCT spectrum of the fundus position to be detected to obtain the amplitude-phase feature input is:

[0040] In the first step, the original OCT spectrum of the fundus to be tested is resampled, windowed, dispersion compensated, and Fourier transformed in sequence to obtain a corresponding OCT complex structure image.

[0041] The second step is to process the obtained OCT complex structure image to obtain the OCT amplitude image and OCT phase image, and then combine the obtained OCT amplitude image and OCT phase image to form an amplitude-phase feature input system;

[0042] Compared with previous deep learning-based fundus blood flow generation technology and traditional OCTA technology, the benefits of the deep learning fundus blood flow imaging method based on OCT amplitude and phase information are as follows:

[0043] (1) The present invention innovatively proposes a co-learning dual-modal blood flow image generation network suitable for fundus blood flow image generation. By constructing an amplitude-phase feature input system, it integrates the phase information that is ignored in traditional methods, so that the network can fully utilize the information of OCT complex images during the learning stage, which helps to improve the network's ability to reconstruct microvessels.

[0044] (2) The present invention innovatively proposes a blood flow texture-blood flow position dual label supervised learning strategy. While comparing the blood flow image generated by the network with the blood flow texture image as the label, it introduces the blood vessel position label calculation loss function to assist the network in accurately locating the blood flow position, effectively enhancing the network's ability to represent small blood vessel structures such as capillaries, and realizing the visualization of blood flow microcirculation.

[0045] (3) The present invention only requires a single original fundus OCT spectrum to reconstruct fundus tomographic blood flow images, avoiding the complex operation of repeated sampling of the same position by ophthalmic OCT equipment during the original spectrum acquisition stage, greatly saving data acquisition time while reducing the impact of subject movement on imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 In the example of the present invention, the blood flow texture-blood flow position true value label system is concentrated in the fundus image dataset;

[0047] Figure 2 In the embodiment of the present invention, the amplitude-phase feature input system is concentrated in the fundus image data set;

[0048] Figure 3 This is a schematic diagram of a co-learning dual-modal blood flow imaging network model produced in an example of the present invention;

[0049] FIG4( a ) is a schematic diagram of the improved HRNet model structure in an example of the present invention;

[0050] FIG4( b ) is a schematic diagram of the structure of the segmentation and shuffling module in the improved HRNet model in an example of the present invention;

[0051] FIG4( c ) is a schematic diagram of the structure of the channel feature fusion module in the improved HRNet model in an example of the present invention;

[0052] FIG4( d ) is a schematic diagram of the structure of the pixel reconstruction network in the improved HRNet model in an example of the present invention;

[0053] FIG5( a ) is a schematic diagram of a tiled CNN model structure in an example of the present invention;

[0054] FIG5( b ) is a schematic diagram of the structure of the shallow feature fusion module in the tiled CNN model in an example of the present invention;

[0055] Figure 5(c) is a schematic diagram of the structure of the deep feature fusion module in the tiled CNN model in an example of the present invention. DETAILED DESCRIPTION

[0056] The specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings and implementation cases. The following examples are used to illustrate the present invention but are not used to limit the scope of the present invention.

[0057] Example

[0058] A deep learning fundus blood flow imaging method based on OCT amplitude and phase information includes the following steps:

[0059] Step 1: Collect multiple original OCT spectra at the same fundus position using an ophthalmic OCT device, process the multiple original OCT spectra collected at the same fundus position to obtain multiple OCT complex structure images, process the multiple OCT complex structure images to obtain blood flow texture labels, and perform image post-processing on the obtained blood flow texture labels to obtain blood flow position labels;

[0060] In step 1, the method for processing the multiple original OCT spectra collected at the same fundus position to obtain multiple OCT complex structure images is as follows:

[0061] The multiple original OCT spectra collected at the same fundus location are resampled, windowed, and dispersion compensated to obtain multiple optimized OCT spectra. Fast Fourier transform operations are performed on the multiple optimized OCT spectra to obtain multiple OCT complex structure images.

[0062] In step 1, a method for processing multiple OCT complex structure images to obtain blood flow texture labels is as follows:

[0063] Principal Component Analysis (PCA) was performed on the multiple OCT complex structure images to remove static signals representing static tissue and retain only dynamic signals representing blood flow, thereby obtaining the original dynamic signal images. The original dynamic signal images were then filtered and denoised to remove speckle noise and obtain blood flow texture labels.

[0064] Blood flow texture tags such as Figure 1 As shown, the blood flow texture label describes the blood flow texture information in multiple OCT complex structure images.

[0065] In step 1, the method of performing image post-processing on the blood flow texture label to obtain the blood flow position label is:

[0066] The threshold of the obtained blood flow texture labels is calculated using the Ostu method. The obtained threshold is then used to binarize the blood flow texture labels to obtain a binary mask. The binary mask represents the degraded blood flow locations. The resulting binary mask is then smoothed using a Gaussian filter to obtain the blood flow location labels. Gaussian smoothing is used to offset the degradation of blood flow edges caused by the binarization operation.

[0067] The obtained blood flow position labels are as follows Figure 1 As shown, the blood flow position label represents the probability information of the presence of blood flow in multiple OCT complex images.

[0068] Step 2: Create a fundus image dataset based on the OCT complex structure map, blood flow texture label, and blood flow position label obtained in step 1;

[0069] In step 2, the generated fundus image data set includes a blood flow texture-blood flow position true value label system and an amplitude-phase feature input system.

[0070] The blood flow texture-blood flow position true value label map system is composed of the blood flow texture label and blood flow position label obtained in step 1;

[0071] The amplitude-phase feature input system is to process the OCT complex structure map obtained in step 1, which is composed of the OCT amplitude map and the OCT phase map. The process of processing the OCT complex structure map obtained in step 1 is as follows:

[0072] First, in the OCT complex image, for the pixel z(x,y)=Re[x(x,y)]+j·Im(z(x,y)), the calculation formula of its amplitude is defined as the following formula (1):

[0073]

[0074] Among them, A represents the amplitude map calculated from the OCT complex structure map, A(x,y) represents the amplitude at the pixel (x,y) position, Re[z(x,y)] represents the real matrix of the complex image, and Im[z(x,y)] represents the imaginary matrix of the complex image. Figure 2 As shown;

[0075] Secondly, for the pixel z(x,y)=Re[z(x,y)]+j·Im(z(x,y)), its phase is calculated using the four-quadrant inverse tangent function to ensure that the result is within the range of (-π,π). The calculation formula is defined as the following equation (2):

[0076] P(x,y)=arctan{Im[z(x,y)],Re[z(x,y)]}#(2)

[0077] Where P represents the phase map calculated from the OCT complex structure map, P(x,y) represents the phase at the pixel (x,y), and arctan(y,x) is the four-quadrant inverse tangent function. Figure 2 As shown;

[0078] Finally, the amplitude map A and the phase map P together constitute the amplitude-phase characteristic input system of the OCT complex map obtained in step 1.

[0079] Step 3: Create a co-learning dual-modal blood flow image generation network;

[0080] In step 3, the co-learned dual-modal blood flow image generation network is composed of a dual-mode HRNet model and a tiled CNN model. The schematic diagram of the co-learned dual-modal blood flow image generation network model is as follows: Figure 3 As shown;

[0081] The dual-mode HRNet model in the co-learned dual-modal blood flow image generation network is an encoder network composed of two parallel improved HRNet models. The two parallel improved HRNet branches are respectively used to receive the fundus OCT amplitude image and the fundus OCT phase image in the amplitude-phase feature input system as input, and output a high-resolution feature map by learning the features of the fundus OCT amplitude image and the fundus OCT phase image in the input system;

[0082] The improved HRNet branch is a network with parallel resolution branches and repeated fusion, consisting of three parallel convolutional streams of different resolutions. The Split Shuffle Net serves as the backbone of the parallel convolutional streams; the Cross-Channel Feature Fusion Module fuses features from convolutional streams of different resolutions; and the Pixel Shuffle Net serves as the main method for upsampling.

[0083] In the improved HRNet branch, the input feature map is first downsampled to its original size in the network. and Features are then extracted from three convolutional streams of different resolutions. During feature extraction, the network repeatedly fuses feature maps from different convolutional streams and outputs the fused high-resolution feature maps to the tiled CNN model. Finally, the feature maps extracted from the three resolution convolutional streams are fused into output feature maps at the end of the network and output to the tiled CNN model. The improved HRNet branch is shown in Figure 4(a).

[0084] The split shuffle network (Split Shuffle Net) is a cascade network composed of multiple cascaded split shuffle modules (SplitShuffle Block). The module is a lightweight residual network composed of a channel split operation (Channel SplitBlock), a point-wise convolution module (Point Wise Convolution Block, PW Conv Block), a channel-wise convolution block (Deep Wise Convolution Block, DW Conv Block), a jump connection layer and a channel shuffle module (Channel Shuffle Block, CS Block). This network reduces the complexity of calculation through group convolution and promotes information interaction between channels through shuffling channels. Compared with ordinary residual connection blocks, this module can maintain feature expression capabilities while reducing the amount of calculation. The split shuffle module is shown in Figure 4(b);

[0085] The cross-channel feature fusion module is a cross-scale feature fusion module improved from the squeeze-and-excitation network. The cross-scale feature fusion module consists of a channel attention mechanism, a full connection layer (FC layer), a jump connection layer, and a feature map scale transformation layer. Taking the fusion of low-resolution feature maps to high resolution as an example, the network first splices the channel weights output by the feature maps of different scales and resolutions after the channel attention mechanism, and then uses the features processed by multiple fully connected layers to guide the channel weights of the low-resolution feature map. Finally, the pixel reconstruction network is used as the feature map scale transformation layer to upsample the low-resolution feature map. By learning the feature map channel information of convolutional streams of different scales, the network retains the feature map information of different convolutional streams while enhancing the calibration of key channels and dynamic features, and establishes a global interaction mode. The channel feature fusion module is shown in Figure 4(c);

[0086] The pixel reorganization network (Pixel Shuffle Net) is an upsampling network composed of a point-by-point convolution module, a pixel reorganization block (PixelShuffle Block) and a convolution block (Convolution Block, Conv Block). This network uses pixel reorganization instead of transposed convolution to achieve upsampling of low-resolution feature maps, which can improve computational efficiency while avoiding the checkerboard effect. Among them, the convolution block is composed of a cascade of 3x3 convolution layers and a LeakyReLU layer. The 3x3 convolution layer suppresses the checkerboard effect by smoothing or sharpening the reorganized feature map, and the LeakyReLU layer prevents gradient disappearance or explosion by alleviating neuron "death". The pixel reorganization network is shown in Figure 4 (d);

[0087] The tiled CNN model in the co-learning dual-modal blood flow image generation network is a decoder network consisting of a scale-invariant convolutional neural network (CNN) backbone, a shallow feature fusion module (Shallow Feature Fusion Module) and a deep feature fusion module (Deep Feature Fusion Module).

[0088] In the tiled CNN model, the high-resolution feature maps output by the two parallel improved HRNet branches in the dual-mode HRNet model are fused after passing through the shallow feature fusion module or the deep feature fusion module to obtain a fused high-resolution feature map. The fused high-resolution feature map is input into the scale-invariant CNN backbone to learn its features, and the scale-invariant CNN backbone outputs the fundus blood flow position image and fundus blood flow image. The tiled CNN model is shown in Figure 5(a);

[0089] The Shallow Feature Fusion Module is a feature map fusion module composed of a channel attention mechanism, a spatial attention mechanism, and a fully connected layer (FC layer). Its core function is to fuse the shallow high-resolution feature maps output by the dual-mode HRNet model through the channel and spatial attention mechanisms. The Shallow Feature Fusion Module is shown in Figure 5(b).

[0090] In the shallow feature fusion module, the shallow high-resolution feature maps output by the dual-mode HRNet model are first spliced ​​in the channel dimension. Key channel weights are then extracted using a channel attention mechanism. The extracted key channel weights are then reinforced with important spatial position information using a spatial attention mechanism. By extracting important texture and position information from shallow features, the shallow feature fusion module avoids the loss of important information along the feature path and ensures the quality of blood flow image generation.

[0091] The Deep Feature Fusion Module is a feature map fusion module consisting of non-local operations, a cascaded residual network, and a skip connection layer. Its core function is to build long-range dependencies in the deep, high-resolution feature maps output by the dual-mode HRNet model through non-local operations and to prevent gradient vanishing using cascaded residual connections. The Deep Feature Fusion Module is shown in Figure 5(c).

[0092] In the deep feature fusion module, the deep high-resolution feature maps output by the dual-mode HRNet model first undergo non-local operations to establish positional associations on a global scale. A cascaded residual connection network is then used to fuse the original features of the deep high-resolution feature maps with the calculated global positional correlations. Finally, a skip connection layer is used to perform skip connections to preserve multi-scale feature information. By establishing contextual associations on a global scale, the deep feature fusion module effectively enhances the model's understanding of global information, helping to overcome the local receptive field limitations of traditional convolutional operations.

[0093] Step 4: Using the fundus image dataset generated in step 2, the co-learned dual-modal blood flow image generation network prepared in step 3 is trained to obtain a trained co-learned dual-modal blood flow image generation network;

[0094] In step 4, when training the co-learned dual-modal blood flow image generation network, a training method based on multi-label joint loss supervision is used, and the method is:

[0095] First, the adaptive wing loss function is used to quantify the difference between the blood flow position label and the blood flow position feature map output by the co-learned dual-modal blood flow image generation network, and the loss L is obtained. AW . Loss L AW The calculation formula is defined as:

[0096]

[0097] Among them, y is the predicted value, is the true value, C is a constant, ω, ε, θ are preset quantities used to adjust the sensitivity of the loss function to small errors and large errors;

[0098] Secondly, the structural similarity loss function is used to quantify the difference between the blood flow texture label and the blood flow image output by the co-learned dual-modal blood flow image generation network, and the loss L is obtained. SSIM . Loss L SSIM The calculation formula is defined as:

[0099]

[0100] Among them, μ is the mean, σ is the variance, and C1C2 is a constant;

[0101] Finally, according to the loss L AW With loss L SSIM Calculate the multi-label joint loss L, the multi-label joint loss function L is defined as:

[0102] L=α·L asW +β·L SSIM #(5)

[0103] Among them, α and β are preset weight parameters;

[0104] Step 5: Use an ophthalmic OCT device to collect raw OCT spectra from the fundus of the person to be examined. These raw OCT spectra are processed to obtain an amplitude-phase feature input system. This amplitude-phase feature input system is then fed into the co-learned dual-modal blood flow image generation network trained in Step 4 to generate a fundus blood flow image of the person to be examined.

[0105] In step 5, the method for processing the collected original OCT spectrum of the fundus position to be detected is:

[0106] In the first step, the original OCT spectrum of the fundus to be tested is resampled, windowed, dispersion compensated, and Fourier transformed in sequence to obtain a corresponding OCT complex structure image.

[0107] The second step is to process the acquired OCT complex structure image to obtain an OCT amplitude map and an OCT phase map. These OCT amplitude map and OCT phase map are then combined into an amplitude-phase feature input system. The method for processing the OCT complex structure image to obtain the OCT amplitude map and OCT phase map is as follows.

[0108] Use equation (1) to calculate the amplitude A(x,y) corresponding to the pixel z(x,y) = Re[z(x,y)] + j·Im(z(x,y)) in the OCT complex image obtained in the first step. By calculating the amplitudes of all pixels in the OCT complex image, an amplitude map is obtained.

[0109] Use equation (2) to calculate the phase P(x,y) corresponding to the pixel z(x,y) = Re[z(x,y)] + j·Im(z(x,y)) in the OCT complex image obtained in the first step. By calculating the phase values ​​of all pixels in the OCT complex image, a phase map is obtained;

[0110] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention. Any technician familiar with the technical field who uses this concept to make non-substantial changes to the present invention within the technical scope disclosed by the present invention shall be deemed to infringe the scope of protection of the present invention.

Claims

1. A deep learning fundus blood flow imaging method based on OCT amplitude and phase information, characterized by The following steps are involved: Step 1: using an ophthalmic OCT device to collect multiple original OCT spectra at the same fundus position, and processing the multiple original OCT spectra collected at the same fundus position to obtain multiple OCT complex structure images, processing the multiple OCT complex structure images to obtain blood flow texture labels, and performing image post-processing on the obtained blood flow texture labels to obtain blood flow position labels; Step 2: Create a fundus image dataset based on the multiple OCT complex structure images, blood flow texture labels, and blood flow position labels obtained in step 1; Step 3: Create a co-learning dual-modal blood flow image generation network; Step 4: Using the fundus image dataset generated in step 2, the co-learned dual-modal blood flow image generation network prepared in step 3 is trained to obtain a trained co-learned dual-modal blood flow image generation network; In step 5, an ophthalmic OCT device is used to collect raw OCT spectra of the fundus of the person to be examined. The collected raw OCT spectra of the fundus of the person to be examined are processed to obtain an amplitude-phase feature input system. The obtained amplitude-phase feature input system is input into the co-learned dual-modal blood flow image generation network trained in step 4 to obtain a fundus blood flow image of the person to be examined.

2. The deep learning fundus blood flow imaging method based on OCT amplitude and phase information according to claim 1 is characterized by: In step 1, the method for processing the multiple original OCT spectra collected at the same fundus position to obtain multiple OCT complex structure images is as follows: Each original OCT spectrum is sequentially subjected to resampling, windowing, dispersion compensation, and Fourier transform operations to obtain a corresponding OCT complex structure image.

3. The deep learning fundus blood flow imaging method based on OCT amplitude and phase information according to claim 1, characterized in that: In step 1, the method for processing the obtained multiple OCT complex structure images to obtain blood flow texture labels is as follows: Principal component analysis was performed on multiple OCT complex structure images to obtain blood flow texture labels.

4. The deep learning fundus blood flow imaging method based on OCT amplitude and phase information according to claim 3 is characterized by: Principal component analysis was performed on multiple OCT complex structure images to remove tailing artifacts and obtain blood flow texture labels.

5. The deep learning fundus blood flow imaging method based on OCT amplitude and phase information according to claim 3 or 4, characterized in that: In step 1, the method for performing image post-processing on the blood flow texture label to obtain the blood flow position label is: The blood flow texture label is processed by filtering and denoising and threshold segmentation in sequence to obtain a binary mask, and the obtained binary mask is subjected to Gaussian filtering to obtain a blood flow position label.

6. The deep learning fundus blood flow imaging method based on OCT amplitude and phase information according to claim 5, characterized in that: The binary mask is Gaussian filtered to remove the tailing artifacts and obtain the blood flow position label.

7. The deep learning fundus blood flow imaging method based on OCT amplitude and phase information according to claim 1, characterized in that: In step 2, the generated fundus image dataset includes a blood flow texture-blood flow position true value label system and an amplitude-phase feature input system; Among them, the blood flow texture-blood flow position true value label system includes blood flow texture labels and blood flow position labels; The amplitude-phase feature input system includes OCT amplitude image and phase image; The OCT amplitude image and the OCT phase image are obtained through the OCT complex structure image.

8. The deep learning fundus blood flow imaging method based on OCT amplitude and phase information according to claim 1, characterized in that: In step 3, the co-learning dual-modal blood flow image generation network is composed of a dual-mode HRNet model and a tiled CNN model; The dual-mode HRNet model consists of two parallel improved HRNet branches, each containing three parallel convolutional streams of different resolutions; The improved HRNet branch consists of a segmentation shuffle network, a channel feature fusion module, and a pixel reconstruction network; The dual-mode HRNet model is used to receive amplitude-phase feature input, and output a high-resolution feature map by learning the features of the OCT amplitude map and OCT phase map in the input.

9. The deep learning fundus blood flow imaging method based on OCT amplitude and phase information according to claim 8, characterized in that: The tiled CNN model consists of a convolutional neural network backbone with constant resolution, a shallow feature fusion module, and a deep feature fusion module; The tiled CNN model is used to receive the high-resolution feature map output by the dual-modality HRNet model and generate blood flow location images and blood flow images through cross-modality image feature fusion; The shallow feature fusion module fuses the shallow high-resolution feature maps output by the dual-mode HRNet model through channel and spatial attention mechanisms; The shallow feature fusion module consists of a channel attention mechanism, a spatial attention mechanism, and a fully connected layer; After the input shallow high-resolution feature maps are spliced ​​in the channel dimension, the key channel weights are extracted through the channel attention mechanism. The extracted key channel weights are then reinforced with important spatial position information through the spatial attention mechanism. The deep feature fusion module constructs long-range dependencies in the deep high-resolution feature maps output by the dual-mode HRNet model through non-local operations, and prevents network gradient disappearance through cascaded residual connections; The deep feature fusion module consists of non-local operations, cascaded residual connection networks and skip connection layers; The input deep high-resolution feature map calculates the global position correlation through non-local operations, and uses a cascaded residual connection network to fuse the original features of the deep high-resolution feature map with the calculated global position correlation. Finally, a skip connection layer is used to perform skip connections to retain multi-scale feature information.

10. The deep learning fundus blood flow imaging method based on OCT amplitude and phase information according to claim 1, characterized in that: In step 4, the training is performed based on a multi-label joint loss supervision training method, the method is: The blood flow position feature map output by the co-learned dual-modal blood flow image generation network is compared with the blood flow position label through the adaptive wing loss function to obtain the loss L AW ; The blood flow texture label is used to compare the blood flow image output by the co-learned dual-modal blood flow image generation network through the structural similarity loss function to obtain the loss L SSIM ; According to the loss L AW and loss L SSIM Calculate multi-label joint loss; Multi-label joint loss function L = α·L AW +β·L SSIM ; Among them, α and β are preset weight parameters.

Images