A physical diffraction model driven OCT image deep learning refocusing method

Abstract

A deep learning-based refocusing method for OCT images driven by a physical diffraction model is disclosed. The method includes: refocusing a sequence of OCT fast-axis images based on a physical diffraction model; constructing a self-supervised OCT refocusing dataset based on the original OCT fast-axis image sequence and the refocused OCT fast-axis image sequence, training a neural network, and building a neural network model driven by the physical diffraction model; reconstructing the refocused OCT fast-axis image sequence into a 3D model to obtain a slow-axis OCT image sequence, inputting this sequence into the neural network model driven by the physical diffraction model, and outputting a refocused slow-axis OCT image sequence, ultimately achieving 3D refocusing of OCT images. This invention does not rely on paired data, significantly reducing the complexity of dataset construction. It improves OCT enface image defocusing through self-supervision, optimizing OCT enface image quality; compared to traditional refocusing methods, it reduces the requirements for system imaging phase stability.

Description

Technical Field

[0001] This invention belongs to the field of image processing and imaging technology, and in particular relates to a physical diffraction model-driven deep learning refocusing method for OCT images. Background Technology

[0002] Optical coherence tomography (OCT) is a high-resolution, non-destructive, and rapid three-dimensional imaging technique. High resolution aids in the imaging of sample microstructures and the accurate diagnosis of early lesions. While a tightly focused Gaussian imaging beam can achieve high lateral resolution, the resolution rapidly decreases in the depth direction as it defocuses, affecting the final three-dimensional imaging results.

[0003] Therefore, resolving the inherent trade-off between lateral resolution and depth of focus in OCT has become a research hotspot in the field of OCT in recent years. Researchers have developed hardware methods such as dynamic focus shift (W. Drexler et al., "In vivo ultrahigh-resolution optical coherence tomography," Opt. Lett., vol. 24, no. 17, pp. 1221-1223, 1999.) and needle-beam imaging (J. Zhao et al., "Flexible method for generating needle-shaped beams and its application in optical coherence tomography," Optica, vol. 9, no. 8, pp. 859-867, 2022.) as well as hardware-aided computational imaging methods such as depth-of-focus extension in optical coherence tomography via multiple aperture synthesis (E. Bo et al., "Depth-of-focus extension in optical coherence tomography via multiple aperture synthesis," Optica, vol. 4, no. 7, pp. 701-706, 2017.) to extend the depth of focus of OCT. However, these methods may increase the complexity and cost of OCT systems.

[0004] In contrast, software methods offer a low-cost alternative solution. Some studies have improved lateral resolution by applying deconvolution algorithms (Y. Liu, Y. Liang, G. Mu, and X. Zhu, “Deconvolution methods for image deblurring in optical coherence tomography,” J. Opt. Soc. Am. A:, vol. 26, no. 1, pp. 72-77, 2009). Computational methods based on physical models of beam propagation have also been developed (TS.Ralston, DL.Marks, PS.Carney, and S.A.Boppart, “Interferometric synthetic aperture microscopy,” Nat. Phys., vol. 3, no. 2, pp. 129-134, 2007.) (L. Zhu et al., “Computational refocusing of Jones matrix polarization-sensitive opticalcoherence tomography and investigation of defocus-induced polarization artifacts,” Biomed. Opt. Express, vol. 13, no. 5, pp. 2975-2994, 2022.), which can obtain effective refocusing results. These computational techniques effectively utilize the phase information in the OCT signal to achieve good refocusing effects. However, they heavily rely on high phase stability during scanning, which is a challenge for in vivo imaging.

[0005] Recently, deep learning has been used to improve the lateral resolution of optical coherence tomography (OCT) (R. Wu et al., “MAS-Net OCT: a deep-learning-based speckle-free multiple aperture synthetic optical coherence tomography,” Biomed. Opt. Express, vol. 14, no. 6, pp. 2591-2607, 2023.) (Z. Yuan et al., “Digital refocusing based on deep learning in optical coherence tomography,” Biomed. Opt. Express, vol. 13, no. 5, pp. 3005-3020, 2022.). However, existing methods require a large amount of simulation or paired experimental data, which is difficult to obtain. Therefore, it is necessary to develop an algorithm that does not require a large amount of paired experimental data and does not depend on system phase stability to effectively improve the trade-off between lateral resolution and depth of focus in OCT. Summary of the Invention

[0006] The purpose of this invention is to provide a physical diffraction model-driven deep learning refocusing method for OCT images, which improves the problem of decreased OCT lateral resolution caused by beam defocusing. This method has low requirements for system phase stability.

[0007] This invention introduces an OCT digital refocusing method that combines a physical diffraction model and a neural network. This method uses a neural network and a physical diffraction model as the core to construct a self-supervised deep learning model driven by the physical diffraction model, and finally realizes unsupervised refocusing of OCT three-dimensional data. It does not depend on phase stability and has good generalization ability.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A physical diffraction model-driven deep learning refocusing method for OCT images comprises three steps:

[0010] Step 1: Refocus the OCT fast-axis image sequence based on the physical diffraction model;

[0011] The physical diffraction model refers to a physical model used to describe the propagation and diffraction process of light waves, which can be an angular spectrum model, a Fresnel diffraction model, a scalar diffraction model, etc.

[0012] The OCT fast-axis image sequence refers to the OCT B-scan image sequence generated along the OCT fast scanning direction in the OCT three-dimensional image;

[0013] The refocusing of OCT fast-axis image sequences based on physical diffraction models refers to generating OCT fast-axis image sequences from acquired OCT 3D images, and then refocusing the OCT fast-axis image sequences sequentially through physical diffraction models to obtain refocused OCT fast-axis image sequences.

[0014] Step 2: Construct an OCT refocusing self-supervised dataset based on the OCT fast-axis image sequence and the refocused OCT fast-axis image sequence, train the neural network, and build a neural network model driven by the physical diffraction model;

[0015] The input to the neural network is an OCT fast-axis image sequence. The refocused OCT fast-axis image sequence is used as a supervision label to iteratively update the weight parameters of the neural network until the training ends. The neural network model driven by the physical diffraction model is the iteratively updated neural network.

[0016] The neural network can be a network model of any structure, including but not limited to generative adversarial neural networks, fully convolutional networks, and Transformer networks.

[0017] Step 3: After refocusing the OCT fast-axis image sequence, the OCT slow-axis image sequence is obtained through 3D reconstruction. This sequence is then input into the neural network model driven by the physical diffraction model to output the refocused OCT slow-axis image sequence, thus achieving OCT 3D image refocusing.

[0018] The OCT slow-axis image sequence refers to the OCT B-scan image sequence generated along the OCT slow scanning direction after three-dimensional reconstruction.

[0019] Advantages and beneficial effects of the present invention:

[0020] 1. This invention improves OCT enface image defocusing and optimizes OCT enface image quality through a self-supervised method.

[0021] 2. This invention belongs to the unsupervised deep learning method, which does not rely on paired data and significantly reduces the complexity of dataset construction.

[0022] 3. Compared with traditional refocusing methods, this invention reduces the requirements for the phase stability of system imaging. Attached Figure Description

[0023] Figure 1 This is a flowchart of the OCT image deep learning refocusing method driven by the physical diffraction model provided by the present invention;

[0024] Figure 2 This is a schematic diagram illustrating the propagation process of light in this invention;

[0025] Figure 3 This is a schematic diagram of the generator of the present invention;

[0026] Figure 4 This is a schematic diagram of the residual block structure in the generator of the present invention;

[0027] Figure 5 This is a schematic diagram of the structure of the discriminator of the present invention;

[0028] Figure 6 These are OCT defocused enface images acquired by this invention;

[0029] Figure 7 These are OCT-focused enface images acquired by this invention;

[0030] Figure 8 This is the refocused OCT enface image output by this invention. Detailed Implementation

[0031] The realization of the objective, functional characteristics, and advantages of the present invention will be further explained below with reference to the accompanying drawings.

[0032] A physical diffraction model-driven deep learning refocusing method for OCT images is described in the attached flowchart. Figure 1 As shown, it includes:

[0033] Step 1: Refocus the OCT fast-axis image sequence based on the physical diffraction model;

[0034] The physical diffraction model refers to the physical model used to describe the propagation and diffraction process of light waves, such as the angular spectrum model, Fresnel diffraction model, and scalar diffraction model.

[0035] In this embodiment, the Fresnel diffraction model is used as the physical diffraction model for the refocused OCT image;

[0036] The Fresnel diffraction model is described in the appendix. Figure 2 The propagation process shown involves light traveling from the defocus plane S' to the focusing plane S. This propagation process can be represented by the following formula:

[0037]

[0038] Where x, y, z represent three-dimensional spatial coordinates, S(f x ,f y S'(f,z) represents the field distribution on the focusing plane. x ,f y f(z) represents the field distribution off the focal plane. x f y Let λ be the spatial frequency, λ be the center wavelength of the system, and z be the frequency. d The defocus distance is F, where F represents the Fourier transform and H represents the defocus distance.-1 The filter function is related to the spatial frequency. The Fresnel diffraction model shows that the propagation of a beam in free space can be regarded as a linear filter with finite spatial bandwidth. Therefore, the defocused OCT signal can be refocused onto the focusing plane. In the practical application of the Fresnel diffraction model, only a single dimension is processed, so formula (1) can be simplified to formula (2).

[0039] The refocusing of OCT fast-axis image sequences based on the physical diffraction model refers to generating an OCT fast-axis image sequence from the acquired OCT 3D image, inputting the OCT fast-axis image sequence into the physical diffraction model, and obtaining the refocused OCT fast-axis image.

[0040] The specific implementation method is as follows:

[0041] In this embodiment, a Fourier transform is first performed on the OCT fast-axis image along the fast-axis direction to obtain the spatial spectrum of the OCT fast-axis image. This spectrum is a two-dimensional complex array, with frequencies distributed laterally and sample depth distributed longitudinally. The OCT system used in this embodiment has a center wavelength λ = 840 nm, and the OCT fast-axis image size is 2048 × 1000 pixels × pixels, corresponding to a field of view of 2.3 mm × 3 mm. Therefore, the spatial frequency f... x The range is 333.3 μm -1 , z d Calculated based on the distance from the selected refocusing depth position to the known focusing depth position, therefore according to λ, f x , z d A two-dimensional array H can be initialized. -1 (f x ,z d ).

[0042] Then calculate F[S(x,z)H] -1 (f x ,z d The Fourier spectrum of the refocused OCT fast axis image can be obtained, and the refocused OCT fast axis image can be obtained after inverse Fourier transform.

[0043] Step 2: Construct an OCT refocusing self-supervised dataset based on the OCT fast-axis image sequence and the refocused OCT fast-axis image sequence, train the neural network, and build a neural network model driven by the physical diffraction model;

[0044] The input to the neural network is an OCT fast-axis image sequence. The refocused OCT fast-axis image sequence is used as a supervision label to iteratively update the weight parameters of the neural network until the training ends. The neural network model driven by the physical diffraction model is the iteratively updated neural network.

[0045] The neural network can be a network model of any structure, including but not limited to generative adversarial neural networks, fully convolutional networks, and Transformer networks. In this embodiment, a generative adversarial residual neural network is used, and the specific implementation method is as follows:

[0046] The generative adversarial residual neural network (GAN) consists of a generator and a discriminator. The generator is responsible for generating the refocused image, while the discriminator is responsible for determining whether the generated refocused image or the actual refocused image is more realistic, thus ensuring that the image generated by the generator achieves the optimal effect. The network structure of the generator in this embodiment is shown in the attached figure. Figure 3 As shown, it includes a convolution module, a residual feature extraction module, and a feature reconstruction module;

[0047] The convolution module consists of one convolutional layer;

[0048] The residual feature extraction module consists of 16 residual blocks (ResBlocks). The structure of each residual block is shown in the attached figure. Figure 4 As shown, it consists of three convolutional layers and three LReLU layers. In the residual block, the input feature map is added to the feature map output by the last convolutional layer to form the final output.

[0049] Appendix Figure 3 In this module, the feature reconstruction module consists of two convolutional layers and one LReLU layer, which is used to reconstruct the learned features into a refocused image.

[0050] The discriminator structure of the embodiment is shown in the attached figure. Figure 5 As shown, Conv contains eight convolutional layers, each with a 3×3 filter kernel. Except for the first layer, each convolutional layer is followed by a batch normalization (BN) layer and an LReLU layer. After the convolutional layers are two linear layers and one LReLU layer. The kernel size (ke), number (n), and stride (s) for each convolutional layer are shown in the diagram. Figure 5 It has been marked in the text.

[0051] Using a dataset consisting of OCT fast-axis image sequences and refocused OCT fast-axis image sequences, an adversarial residual neural network was trained to obtain a physics-driven neural network model. All models were optimized using the Adam algorithm, with hyperparameters α = 0, β1 = 0.9, and β2 = 0.99. The training iterations were set to 150,000. Model training and testing were performed using the deep learning framework PyTorch.

[0052] Step 3: After refocusing the OCT fast-axis image sequence, the OCT slow-axis image sequence is obtained through 3D reconstruction and input into the neural network model driven by the physical diffraction model. The refocused OCT slow-axis image sequence is output, and finally the OCT 3D image refocusing is achieved.

[0053] The OCT slow-axis image sequence refers to the OCT B-scan image sequence generated along the OCT slow scanning direction after three-dimensional reconstruction.

[0054] As attached Figure 6 Appendix Figure 7 and attached Figure 8 As shown, the digital refocusing results of the OCT enface defocused image, focused image, and defocused image of the present invention are presented. Figure 6 The OCT enface defocused image of the human ex vivo thyroid gland acquired by OCT is obtained by three-dimensional reconstruction of the OCT fast axis image sequence described in step 1. The corresponding defocus amount is 300μm. Due to the influence of beam defocus, the sample structure in the image becomes blurred and difficult to distinguish. Figure 7 The image provided is a focused OCT enface image of the same location as an isolated human thyroid gland. This image was obtained by adjusting the focusing position of the light beam in the sample using an OCT system and can be used as a reference enface image to evaluate the refocusing results of this invention. Figure 7 Compared to OCT enface defocused images, the sample structure in the middle sample is shown. Figure 6 Clearly distinguishable; Figure 8 The output of this invention Figure 6 The refocused image is obtained by three-dimensional reconstruction of the refocused OCT slow-axis image sequence described in step 3. Figure 8 The thyroid gland structure is clearly visible, and the refocused structure is distributed along both the fast and slow axes in the same direction as the reference enface image. Figure 7 The structural distributions of the samples are very close, which indicates that the refocused OCT enface image can reconstruct the defocused and blurred sample structure using the method of the present invention. This shows that the lateral resolution of the OCT image is improved, and three-dimensional refocusing is achieved.

[0055] In this invention, the OCT fast-axis image sequence is first refocused based on a physical diffraction model. Then, a self-supervised OCT refocusing dataset is constructed using the OCT fast-axis image sequence and the refocused OCT fast-axis image sequence. A neural network is trained to build a neural network model driven by the physical diffraction model. Next, the refocused OCT fast-axis image sequence is 3D reconstructed to obtain the OCT slow-axis image sequence, which is then input into the neural network model driven by the physical diffraction model. The output is the refocused OCT slow-axis image sequence, ultimately achieving OCT 3D image refocusing. This invention uses a physical diffraction model to drive a deep learning algorithm, enabling self-supervised OCT image refocusing without relying on high-resolution prior knowledge. This is an unsupervised deep learning method, significantly reducing the complexity of dataset construction. Furthermore, the method's refocusing of OCT slow-axis images does not depend on the phase stability of the OCT along the slow-axis scanning direction. Compared to traditional refocusing methods, this reduces the requirements for system imaging phase stability and offers strong portability.

[0056] The above embodiments are intended to illustrate the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it. They should not be used to limit the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A physical diffraction model-driven deep learning refocusing method for OCT images, characterized in that, Includes the following steps: Step 1: Refocus the OCT fast-axis image sequence based on the physical diffraction model; Step 2: Construct an OCT refocusing self-supervised dataset based on the OCT fast-axis image sequence and the refocused OCT fast-axis image sequence, train the neural network, and build a neural network model driven by the physical diffraction model; Step 3: After refocusing the OCT fast-axis image sequence, the OCT slow-axis image sequence is obtained through 3D reconstruction and input into the neural network model driven by the physical diffraction model. The refocused OCT slow-axis image sequence is output, and finally the OCT 3D image refocusing is achieved.

2. The physical diffraction model-driven deep learning refocusing method for OCT images according to claim 1, characterized in that, The physical diffraction model refers to a physical model used to describe the propagation and diffraction process of light waves, which can be an angular spectrum model, a Fresnel diffraction model, or a scalar diffraction model.

3. The physical diffraction model-driven deep learning refocusing method for OCT images according to claim 1, characterized in that, The OCT fast-axis image sequence refers to the OCT B-scan image sequence generated along the OCT fast scanning direction in the OCT three-dimensional image.

4. The physical diffraction model-driven deep learning refocusing method for OCT images according to claim 1, characterized in that, The refocusing of OCT fast-axis image sequences based on physical diffraction models refers to generating OCT fast-axis image sequences from acquired OCT 3D images, and then refocusing the OCT fast-axis image sequences sequentially through physical diffraction models to obtain refocused OCT fast-axis image sequences.

5. The physical diffraction model-driven deep learning refocusing method for OCT images according to claim 1, characterized in that, The input to the neural network is an OCT fast-axis image sequence. The refocused OCT fast-axis image sequence is used as a supervision label to iteratively update the weight parameters of the neural network until the training ends. The neural network model driven by the physical diffraction model is the iteratively updated neural network.

6. The physical diffraction model-driven deep learning refocusing method for OCT images according to claim 1, characterized in that, The neural network can be a network model of any structure, including but not limited to generative adversarial neural networks, fully convolutional networks, and Transformer networks.

7. The physical diffraction model-driven deep learning refocusing method for OCT images according to claim 1, characterized in that, The OCT slow-axis image sequence refers to the OCT B-scan image sequence generated along the OCT slow scanning direction after three-dimensional reconstruction.

Images