Method for improving OCT image resolution based on lightweight network

Abstract

The invention discloses a method for improving OCT image resolution based on a lightweight network. The method mainly comprises the four steps of obtaining an OCT image to be processed, constructing a training data set, constructing an image super-resolution model and improving the OCT image resolution based on the constructed image super-resolution model. In the construction of an image super-resolution model, a lightweight convolutional network is utilized to learn the mapping relation between a plurality of pairs of OCT images with different resolutions through short training time. The method is a self-supervised digital super-resolution method which is short in training time, and super-resolution of the image can be realized only by utilizing the OCT image to be processed.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing and imaging, and in particular relates to a method for improving the resolution of optical coherence tomography images based on a lightweight network. Background Art

[0002] Optical coherence tomography (OCT) is a non-contact, non-invasive imaging technology. Improved image resolution helps identify smaller structural information in samples. The longitudinal resolution of OCT is inversely proportional to the spectral width of the light source. Using a light source with a wider spectrum can achieve higher longitudinal resolution, but hardware methods represented by the use of light sources with a wider spectrum rely on complex optical designs and expensive equipment. On the other hand, OCT is a three-dimensional imaging technology that is constrained by the diffraction limit and imaging focal depth. Its lateral resolution and imaging focal depth usually require a compromise design. The use of phase filters and axicons can extend the focal depth and improve the lateral resolution in different depth directions, but it requires the design and processing of fine components or the construction of a complex imaging system.

[0003] Compared with methods that improve hardware systems, digital processing technologies such as deconvolution methods, spectral estimation methods, and spectral shaping methods provide more convenient and cheaper solutions to improve the longitudinal resolution of OCT. However, due to the lack of prior knowledge, the above-mentioned processing methods cannot break through the theoretical limit of longitudinal resolution in complex samples; and the coherent synthetic aperture method requires maintaining high phase stability during imaging.

[0004] In recent years, deep learning-based methods have achieved improvements in the resolution of OCT images (Z.Yuan et al., Axial super-resolution study for optical coherence tomography images via deep learning, IEEE Access, 2020, 8: 204941-204950; Z.Yuan, et al., Digital refocusing based on deep learning in optical coherence tomography, Biomed. Opt. Express, 2022, 13(5): 3005-3020; Z.Yuan, et al., Self super-resolution of optical coherence tomography images based on deep learning, Optics Express, 2023, 31(17): 27566-27581.). However, existing deep learning methods for OCT image super-resolution have the following problems: the need for prior knowledge of the high-resolution system, the need to collect thousands or even tens of thousands of images to construct a training dataset, the cumbersome image data collection, the large network size, the long training time, and the need for offline training. Summary of the Invention

[0005] The main purpose of this invention is to overcome the shortcomings of the above-mentioned existing technologies and provide a method for improving the resolution of OCT images based on a lightweight network. This method, based on the multi-scale self-similarity of OCT images, uses a lightweight convolutional network to achieve improved OCT image resolution while shortening training time.

[0006] Based on the imaging principles of OCT systems, this method exploits the self-similarity of image information at different scales to generate a training dataset of 20 or fewer images from a single image. Using a convolutional neural network with 50 or fewer layers, the method learns the mapping relationships between multiple sets of high-resolution and low-resolution images generated from a single image. This method quickly constructs a super-resolution model for OCT images, achieving real-time improvement in OCT image resolution. This method is a self-supervised image super-resolution method.

[0007] To achieve the above objectives, the present invention adopts the following scheme:

[0008] A lightweight network-based method for improving OCT image resolution includes four steps:

[0009] Step 1: Acquire the OCT image to be processed;

[0010] The OCT system is used to collect the interference spectrum of the sample to be tested, and each column of the A-scan interference spectrum is inverse Fourier transformed to obtain the OCT image to be processed; the collected OCT image to be processed includes a B-scan OCT image and an en face OCT image.

[0011] Step 2: Construct an OCT image super-resolution training dataset; the specific steps are:

[0012] Downsampling the acquired OCT image to be processed at different levels to obtain N low-resolution images of different levels; the OCT image to be processed and the N low-resolution images of different levels are arranged according to the resolution to form N+1 images; wherein every two images with adjacent resolutions form a pair of OCT high-resolution and OCT low-resolution images, and there are N pairs of OCT high-resolution and OCT low-resolution images for N+1 images with adjacent resolutions; wherein the ratio of the low-resolution to high-resolution values ​​in each pair of images with adjacent resolutions is constant at r;

[0013] The N OCT high-resolution and OCT low-resolution image pairs constitute an OCT image super-resolution training dataset; the value of N is a natural number between 2 and 20, preferably, the value of N is a natural number between 2 and 8;

[0014] The r value is a rational number between 1.05 and 6. Preferably, the r value is a rational number between 1.2 and 3.

[0015] For a B-scan OCT image to be processed, the method for obtaining N low longitudinal resolution images of different levels is as follows:

[0016] The A-scan interference spectrum described above is spectrally shaped as in formula (1).

[0017] S shape,N (k)=S(k)×gaussian N (k) #(1)

[0018] Where k represents the wave number, S(k) represents a series of A-scan interference spectra collected by the OCT system, and gaussian N (k) represents N Gaussian functions with different full width at half maximum (FWHM); S shape,N(k) represents an A-scan interference spectrum obtained after spectral shaping, and its full width at half maximum determines the longitudinal resolution of the B-scan OCT image; by adjusting the full width at half maximum of the Gaussian function, A-scan interference spectra with different spectral shaping are obtained, and then B-scan OCT images with different longitudinal resolutions are obtained; the specific steps can be one of the following two:

[0019] (1) For the B-scan OCT image to be processed, each column of the A-scan interference spectrum is multiplied by the N Gaussian functions with different half-maximum full widths to obtain N sets of shaped spectra S shape,N (k) performing an inverse Fourier transform on each of the spectra at different numbers of points, where the number of points of the inverse Fourier transform is determined by the resolution ratio r between each pair of adjacent resolution images, the number of points used in the obtained higher-resolution image among the adjacent resolutions being r times the number of points of the lower-resolution image, and the number of vertical pixels of the obtained higher-resolution image among the adjacent resolutions being also r times the number of vertical pixels of the lower-resolution image; obtaining a total of N sets of low-resolution A-scan OCT signals of different levels, each set of A-scan OCT signals forming a low-resolution B-scan OCT image, and obtaining a total of N low-resolution B-scan OCT images of different levels corresponding to the B-scan OCT image to be processed;

[0020] (2) For the B-scan OCT image to be processed, each column of the A-scan interference spectrum is multiplied by the N Gaussian functions with different half-maximum full widths to obtain N sets of shaped spectra S shape,N (k) Perform an inverse Fourier transform of the same number of points on each spectrum to obtain a total of N sets of low-resolution A-scan OCT signals of different levels. Each set of A-scan OCT signals will form a low-resolution B-scan OCT image, and a total of N low-resolution B-scan OCT images with the same number of longitudinal pixels at different levels corresponding to the B-scan OCT image to be processed are obtained.

[0021] For the en face OCT image to be processed, a digital defocus method is used to obtain N low lateral resolution images of different levels. The specific steps are as follows:

[0022] Perform the operation (2) on the en face OCT image to be processed.

[0023]

[0024] Where x and y represent the coordinates of the two dimensions of the en face OCT image, z represents the depth position of the sample, and S(x,y,z) represents the en face OCT signal. and represent the transverse two-dimensional Fourier transform and inverse transform, respectively. are N spatial frequency filters corresponding to N defocus distances, where f x is the spatial frequency corresponding to the coordinate x, f y is the spatial frequency corresponding to the coordinate y, d N Represents the distance from the focus; S' N (x, y, z) represents the defocused en face OCT signal. Different levels of low-resolution images refer to en face OCT images at different defocus distances. The farther the distance from the focus, the lower the resolution. En face OCT images of different resolutions are obtained by adjusting the defocus distance of the spatial frequency filter. The specific steps can be one of the following two:

[0025] (1) For the en face OCT image to be processed, its original signal is processed by N spatial frequency filters with different defocus distances according to formula (2) to obtain N low-resolution images of different levels; the N low-resolution images of different levels are downsampled at different levels, and the size of the downsampled image is determined by the resolution ratio r between each pair of adjacent resolution images, and the number of pixels in the x and y dimensions of the higher resolution image among the adjacent resolutions is r times the number of pixels in the x and y dimensions of the lower resolution image; a total of N low-resolution en face OCT images of different levels corresponding to the en face OCT image to be processed are obtained;

[0026] (2) For the en face OCT image to be processed, its original signal is processed by N spatial frequency filters with different defocus distances according to formula (2) to obtain N low-resolution images of different levels. These N low-resolution images of different levels are not downsampled, and a total of N low-resolution en face OCT images of the same size and different levels corresponding to the en face OCT image to be processed are obtained.

[0027] Step 3: Build a super-resolution model of OCT images;

[0028] Train a lightweight network to learn the mapping relationship between the N pairs of OCT high-resolution and OCT low-resolution images constructed in step 2, and build a super-resolution model for OCT images;

[0029] Step 4: Realize super-resolution of OCT images;

[0030] The OCT image to be processed is input into the super-resolution model of the OCT image constructed in step 3. The output of the super-resolution model is a super-resolution OCT image, thereby improving the resolution of the OCT image to be processed.

[0031] The OCT image super-resolution training dataset constructed in the present invention includes both longitudinal and transverse super-resolution training datasets, meaning that the longitudinal and transverse resolutions of the processed OCT images can be improved separately. When improving the longitudinal and transverse resolutions separately, the values ​​of N and r can be the same or different.

[0032] Advantages and beneficial effects of the present invention:

[0033] 1. The present invention does not require any hardware device assistance and uses digital methods to improve the OCT image resolution, which can reduce the hardware cost of system development;

[0034] 2. This invention combines spectral shaping and digital defocusing methods, exploiting the self-similarity of information contained in a single image at different scales. Independent of any external training data, the image to be processed is downsampled at different levels to construct a dataset. A lightweight convolutional neural network is then used to learn the mapping relationship between multiple pairs of high-resolution and low-resolution images generated from a single image. This approach shortens training time and accelerates processing speed, enabling real-time, online self-supervised super-resolution of OCT images.

[0035] 3. The implementation of the present invention has low requirements on the hardware system for image super-resolution and has strong generalization capability of super-resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of a method for improving OCT image resolution based on a lightweight network provided by the present invention;

[0037] Figure 2 It is a schematic diagram of the network structure of the present invention;

[0038] Figure 3 This is the result of the present invention improving the longitudinal resolution of B-scan OCT images; wherein (a) is a B-scan OCT image of an orange with a longitudinal resolution of 27 microns, (b) is a B-scan OCT image of an orange with a longitudinal resolution of 18 microns, (c) is a B-scan OCT image of an orange with a longitudinal resolution of 12 microns, and (d) is a B-scan OCT image of an orange with a longitudinal resolution of 8 microns. This figure is the image to be processed, (e) is the super-resolution result of the image to be processed (d) output by the super-resolution model of the present invention, and (f) is a B-scan OCT image of an orange with a longitudinal resolution of 5.3 microns, which is used for comparison with the super-resolution image (e).

[0039] Figure 4 This is the result of the present invention improving the lateral resolution of en face OCT images; among them, (a) is an orange en face OCT image with a lateral resolution of 64 microns, (b) is an orange en face OCT image with a lateral resolution of 32 microns, and (c) is an orange en face OCT image with a lateral resolution of 16 microns. This figure is the image to be processed, (d) is the super-resolution result of the image to be processed output by the super-resolution model of the present invention, and (e) is an orange en face OCT image with a lateral resolution of 8 microns, which is used for comparison with the super-resolution image (d). DETAILED DESCRIPTION

[0040] The realization of the purpose, functional features and advantages of the present invention are further described below with reference to the accompanying drawings.

[0041] Example 1

[0042] The present invention provides a method for improving the resolution of OCT images based on a lightweight network, the flow chart of which is shown in the attached figure. Figure 1 As shown, it includes four steps:

[0043] Step 1: Acquire the OCT image to be processed;

[0044] The OCT system is used to collect the interference spectrum of the sample to be tested, and each column of the A-scan interference spectrum is inverse Fourier transformed to obtain the OCT image to be processed; the collected OCT image to be processed includes a B-scan OCT image and an en face OCT image.

[0045] Step 2: Construct an OCT image super-resolution training dataset; the specific steps are:

[0046] Downsampling the acquired OCT image to be processed at different levels to obtain N low-resolution images of different levels; the OCT image to be processed and the N low-resolution images of different levels are arranged according to the resolution to form N+1 images; wherein every two images with adjacent resolutions form a pair of OCT high-resolution and OCT low-resolution images, and there are N pairs of OCT high-resolution and OCT low-resolution images for N+1 images with adjacent resolutions; wherein the ratio of the low-resolution to high-resolution values ​​in each pair of images with adjacent resolutions is constant at r;

[0047] The N OCT high-resolution and OCT low-resolution image pairs constitute an OCT image super-resolution training dataset;

[0048] For a B-scan OCT image to be processed, the method for obtaining N low longitudinal resolution images of different levels is as follows:

[0049] The A-scan interference spectrum described above is spectrally shaped as in formula (1).

[0050] S shaape,N (k)=S(k)×Gaussian N (k) #(1)

[0051] Where k represents the wave number, S(k) represents a series of A-scan interference spectra collected by the OCT system, and gaussian N (k represents N Gaussian functions with different full width at half maximum (FWHM); S shape,N (k) represents the A-scan interference spectrum obtained after spectral shaping, and its full width at half maximum determines the longitudinal resolution of the B-scan OCT image. By adjusting the full width at half maximum of the Gaussian function, A-scan interference spectra with different spectral shaping are obtained, and then B-scan OCT images with different longitudinal resolutions are obtained. The specific method is as follows:

[0052] For the B-scan OCT image to be processed, each column of the A-scan interference spectrum is multiplied by the N Gaussian functions with different half-maximum full widths to obtain N sets of shaped spectra S shape,N (k) performing an inverse Fourier transform on each of the spectra at different numbers of points, where the number of points of the inverse Fourier transform is determined by the resolution ratio r between each pair of images of adjacent resolutions, the number of points used in the obtained higher-resolution image among the adjacent resolutions is r times the number of points of the lower-resolution image, and the number of vertical pixels of the obtained higher-resolution image among the adjacent resolutions is also r times the number of vertical pixels of the lower-resolution image; a total of N sets of low-resolution A-scan OCT signals of different levels are obtained, each set of A-scan OCT signals will form a low-resolution B-scan OCT image, and a total of N low-resolution B-scan OCT images of different levels corresponding to the B-scan OCT image to be processed are obtained.

[0053] In the present invention, the images with adjacent resolutions refer to two images with the closest resolutions, which respectively constitute the high-resolution image and the low-resolution image of the image pair in the data set.

[0054] In this example, the value of r is 1.5, and the value of N is 3. In this example, a B-scan OCT image with a longitudinal resolution of 8 microns is first obtained. Using spectral shaping, three lower-resolution B-scan OCT images with longitudinal resolutions of 12, 18, and 27 microns are obtained, resulting in a total of four B-scan OCT images with different longitudinal resolutions. These four images have longitudinal resolutions of 540, 360, 240, and 160 pixels, respectively. While the number of points changes during the inverse Fourier transform, the corresponding longitudinal imaging range remains unchanged, remaining at 0.72 mm. The lateral resolution is 1000 pixels per image, resulting in a lateral imaging range of 1.5 mm. In this embodiment, a B-scan OCT image with a longitudinal resolution of 8 microns and a B-scan OCT image with a longitudinal resolution of 12 microns constitute a high-resolution and low-resolution B-scan OCT image pair; a B-scan OCT image with a longitudinal resolution of 12 microns and a B-scan OCT image with a longitudinal resolution of 18 microns constitute a high-resolution and low-resolution B-scan OCT image pair; and a B-scan OCT image with a longitudinal resolution of 18 microns and a B-scan OCT image with a longitudinal resolution of 27 microns constitute a high-resolution and low-resolution B-scan OCT image pair. These three pairs of high-resolution and low-resolution B-scan OCT images constitute a longitudinal super-resolution training dataset for the B-scan OCT images to be processed. The longitudinal super-resolution training dataset for B-scan OCT images includes the above three image pairs. Next, images with an 8-μm longitudinal resolution were cropped into 54-by-16-pixel patches, images with a 12-μm longitudinal resolution were cropped into 36-by-16-pixel patches, images with an 18-μm longitudinal resolution were cropped into 24-by-16-pixel patches, and images with a 27-μm longitudinal resolution were cropped into 16-by-16-pixel patches. During the cropping process, patches with less feature information were removed by calculating the grayscale average of the patches. The dataset was then augmented with data rotation and other methods. Finally, the training and validation sets were split in a 9:1 ratio, with the test images being the B-scan OCT images to be processed.

[0055] Step 3: Build a super-resolution model of OCT images;

[0056] Train a lightweight network to learn the mapping relationship between the three OCT high-resolution and OCT low-resolution image pairs constructed in step 2, and construct a super-resolution model of OCT images (the constructed network model has 50 or fewer layers, preferably a lightweight network with 3-15 layers). The specific implementation method is as follows:

[0057] This example uses a convolutional neural network as a deep learning super-resolution model (image super-resolution models can also use generative adversarial neural networks, capsule networks, or graph neural networks). The network used for training is fully convolutional and can be applied to images of different sizes. The low-resolution image of the generated low-resolution and high-resolution OCT image pairs is input into the fully convolutional network to obtain a predicted high-resolution image. The network weight parameters are updated by returning the loss results. Multiple training iterations are performed until the convolutional neural network converges.

[0058] The convolutional neural network structure used in the embodiment is as shown in the attached figure. Figure 2 As shown, considering the simple mapping relationship between high-resolution and low-resolution OCT image pairs obtained from a single image, that is, the high similarity between the high-resolution and low-resolution OCT image pairs, this embodiment selects a small and simple convolutional neural network for training. Except for the last layer, each convolutional layer is followed by an LReLU layer, consisting of a total of 8 convolutional layers and 7 LReLU layers. The convolution kernel size of each convolutional layer is 3×3, and each hidden layer has 64 channels.

[0059] Using the constructed super-resolution OCT image dataset, a convolutional neural network was trained to obtain a super-resolution OCT image model. The model was optimized using the Adam algorithm, with hyperparameters α = 0, β1 = 0.9, and β2 = 0.99. The number of training iterations was set to 15,000. The convolutional neural network model was trained and tested using the deep learning framework Pytorch, and training took 1 to 2 minutes.

[0060] Step 4: Implement super-resolution of OCT images; the specific implementation method is as follows:

[0061] The B-scan OCT image to be processed, with a longitudinal resolution of 8 microns, is input into the OCT image super-resolution model constructed in step 3. The super-resolution model outputs a super-resolved B-scan OCT image, thereby improving the OCT image resolution. Because training only takes 1-2 minutes, the method of the present invention can achieve super-resolution of OCT images in real time and online.

[0062] The present invention can improve the resolution by a factor of r, where r is calculated by calculating the ratio of the low-resolution to high-resolution values ​​in the super-resolution OCT image dataset. In this embodiment, r is set to 1.5, meaning that the ratio of each pair of low-resolution to high-resolution values ​​in the super-resolution OCT image dataset is 1.5.

[0063] As attached Figure 3The figures show the results of the present invention in improving the longitudinal resolution of B-scan OCT images. (a) is a B-scan OCT image of an orange with a longitudinal resolution of 27 microns; (b) is a B-scan OCT image of an orange with a longitudinal resolution of 18 microns; (c) is a B-scan OCT image of an orange with a longitudinal resolution of 12 microns; (d) is a B-scan OCT image of an orange with a longitudinal resolution of 8 microns, which is the image to be processed; (e) is the super-resolution result of the image to be processed output by the super-resolution model of the present invention; (f) is a B-scan OCT image of an orange with a longitudinal resolution of 5.3 microns as a comparison of the super-resolution effect. From the comparison of Figures (e) and (d), it can be seen that the tissue information of the orange in the super-resolution image of Figure (c) is clearer and more detailed; at the same time, Figure (e) is closer to the tissue information of the orange in Figure (f) used as the effect comparison, indicating that the longitudinal resolution of the OCT image has been improved.

[0064] In the method for improving OCT image resolution based on a lightweight network of the present invention, in step 2 of constructing an OCT image super-resolution training dataset, the following method can also be used to obtain B-scan OCT images of different longitudinal resolutions:

[0065] For the B-scan OCT image to be processed, each column of the A-scan interference spectrum is multiplied by the N Gaussian functions with different half-maximum full widths to obtain N sets of shaped spectra S shape,N (k) Performing an inverse Fourier transform on each of the spectra at the same number of points, thereby obtaining N sets of low-resolution A-scan OCT signals of varying levels. Each set of A-scan OCT signals forms a low-resolution B-scan OCT image, resulting in a total of N low-resolution B-scan OCT images of varying levels corresponding to the B-scan OCT image to be processed, each with the same number of longitudinal pixels. The remaining steps and methods are the same as those of Example 1.

[0066] Example 2

[0067] The present invention provides a method for improving the resolution of OCT images based on a lightweight network, the flow chart of which is shown in the attached figure. Figure 1 As shown, the method includes four steps. In step 2, during the process of constructing the OCT image super-resolution training dataset, for the enface OCT image to be processed, a digital defocus method is used to obtain N low-lateral resolution images of different levels. The specific steps are as follows:

[0068] Perform the operation (2) on the en face OCT image to be processed.

[0069]

[0070] Where x and y represent the coordinates of the two dimensions of the en face OCT image, z represents the depth position of the sample, and S(x,y,z) represents the en face OCT signal. and represent the transverse two-dimensional Fourier transform and inverse transform, respectively. are N spatial frequency filters corresponding to N defocus distances, where f x is the spatial frequency corresponding to the coordinate x, f y is the spatial frequency corresponding to the coordinate y, d N Represents the distance from the focus; S' N (x, y, z) represents the defocused en face OCT signal. Different levels of low-resolution images refer to en face OCT images at different defocus distances. The farther away from the focus, the lower the resolution. En face OCT images of different resolutions are obtained by adjusting the defocus distance of the spatial frequency filter. The specific method is as follows:

[0071] For the en face OCT image to be processed, its original signal is processed by N spatial frequency filters with different defocus distances according to formula (2) to obtain N low-resolution images of different levels. These N low-resolution images of different levels are not downsampled, and a total of N low-resolution en face OCT images of the same size and different levels corresponding to the en face OCT image to be processed are obtained.

[0072] In this embodiment, the value of r is 2, and the value of N is 2. First, an en face OCT image with a lateral resolution of 16 microns is obtained, and the imaging range of both the x and y dimensions is 0.75 mm. Two low-resolution en face OCT images of 32 microns and 64 microns corresponding to the en face OCT image to be processed are obtained using a digital defocus method, thereby obtaining a total of three en face OCT images with different lateral resolutions. The number of pixels in the x and y dimensions of these three images is 248 pixels. In the present invention, the images with adjacent resolutions refer to the two images with the closest resolutions, which respectively constitute the high-resolution image and low-resolution image of the image pair in the dataset. In this embodiment, an en face OCT image with a lateral resolution of 16 microns and an en face OCT image with a lateral resolution of 32 microns constitute a pair of high-resolution and low-resolution en face OCT image pairs, and an en face OCT image with a lateral resolution of 32 microns and an en face OCT image with a lateral resolution of 64 microns constitute a pair of high-resolution and low-resolution en face OCT image pairs. These two pairs of high-resolution and low-resolution en face OCT image pairs together constitute a lateral super-resolution training dataset for the en face OCT image to be processed. The lateral super-resolution training dataset of the en face OCT image contains the above two image pairs. The image is then cropped into small image blocks of 32 pixels × 32 pixels. During the cropping process, the image blocks with less feature information are removed by calculating the grayscale average of the small image blocks. Then, data expansion such as rotation is performed to generate a dataset. Finally, the training set and validation set are allocated in a 9:1 ratio, and the test image is the en face OCT image to be processed.

[0073] In step 4: realizing super-resolution of OCT images, the OCT images to be processed are enface OCT images with a lateral resolution of 16 microns. Figure 4The figures show the results of the present invention in improving the lateral resolution of en face OCT images. (a) is an en face OCT image of an orange with a lateral resolution of 64 microns; (b) is an en face OCT image of an orange with a lateral resolution of 32 microns; (c) is an en face OCT image of an orange with a lateral resolution of 16 microns, which is the image to be processed; (d) is the super-resolution result of the image to be processed output by the super-resolution model of the present invention; (e) is an en face OCT image of an orange with a lateral resolution of 8 microns as a comparison of the super-resolution effect. From the comparison of Figures (c) and (d), it can be seen that the tissue information of the orange in the super-resolution image of Figure (d) is clearer and more detailed. At the same time, Figure (d) is closer to the tissue information of the orange in Figure (e) used as the comparison, and the noise is also reduced, indicating that the lateral resolution of the OCT image has been improved.

[0074] The remaining steps and methods are the same as in Example 1.

[0075] In the method for improving OCT image resolution based on a lightweight network of the present invention, in step 2 of constructing an OCT image super-resolution training dataset, the following method can also be used to obtain en face OCT images of different resolutions:

[0076] For the en face OCT image to be processed, its original signal is processed by N spatial frequency filters with different defocus distances according to formula (2) to obtain N low-resolution images of different levels; the N low-resolution images of different levels are downsampled to different levels, and the size of the downsampled image is determined by the resolution ratio r between each pair of adjacent resolution images, and the number of pixels in the x and y dimensions of the higher resolution image in the obtained adjacent resolution is r times the number of pixels in the x and y dimensions of the lower resolution image; a total of N low-resolution en face OCT images of different levels corresponding to the en face OCT image to be processed are obtained. The remaining steps and methods are the same as those in Example 2.

[0077] During implementation, the present invention first obtains a sample interference signal using an OCT system to obtain an OCT image to be processed. Multiple low-resolution OCT images of varying levels are then obtained using an inverse Fourier transform, spectral shaping, or digital defocusing methods. An OCT image super-resolution training dataset is constructed using the image to be processed and these low-resolution images of varying levels. A lightweight network is then used to learn the mapping relationship between image pairs in the OCT image super-resolution training dataset to construct an OCT image super-resolution model. Finally, the resolution of the OCT image to be processed is improved based on the OCT image super-resolution model. Considering that the relationship between low-resolution and high-resolution OCT image pairs obtained from a single image is relatively small in diversity, i.e., the information contained in a single image has a high degree of self-similarity at different scales, the present invention uses a lightweight convolutional neural network to achieve super-resolution of OCT images while reducing the amount of computation. This achieves super-resolution results using a shorter training time without increasing the hardware complexity of the OCT system, resulting in high speed and strong generalization.

[0078] The above embodiments are for illustrating the technical concept and features of the present invention. Their purpose is to enable people familiar with this technology to understand the content of the present invention and implement it. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made based on the essence of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for improving OCT image resolution based on a lightweight network, characterized in that: Here are the steps: Step 1: Use an optical coherence tomography (OCT) system to collect the interference spectrum of the sample to be tested, perform inverse Fourier transform on each column of the A-scan interference spectrum, and obtain the OCT image to be processed; Step 2: Construct an OCT image super-resolution training dataset. The specific steps are as follows: Downsampling the acquired OCT image to be processed at different levels to obtain N low-resolution images of different levels; the OCT image to be processed and the N low-resolution images of different levels are arranged according to the resolution to form N+1 images; wherein every two images with adjacent resolutions form a pair of OCT high-resolution and OCT low-resolution images, and there are N pairs of OCT high-resolution and OCT low-resolution images for N+1 images with adjacent resolutions; wherein the ratio of the low-resolution to high-resolution values ​​in each pair of images with adjacent resolutions is constant at r; The N OCT high-resolution and OCT low-resolution image pairs constitute an OCT image super-resolution training dataset; Step 3: Build a super-resolution model for OCT images: Train a lightweight network to learn the mapping relationship between the N pairs of OCT high-resolution and OCT low-resolution images constructed in step 2, and build a super-resolution model for OCT images; Step 4: Super-resolution of OCT images: The OCT image to be processed is input into the super-resolution model of the OCT image constructed in step 3. The output of the super-resolution model is a super-resolution OCT image, thereby improving the resolution of the OCT image to be processed.

2. The method for improving OCT image resolution based on a lightweight network according to claim 1, characterized in that: The OCT images to be processed acquired in step 1 include B-scan OCT images and en face OCT images.

3. The method for improving OCT image resolution based on a lightweight network according to claim 1, characterized in that: The constructed OCT image super-resolution training data set includes longitudinal and transverse super-resolution training data sets, which can respectively improve the longitudinal resolution and transverse resolution of the OCT image to be processed.

4. The method for improving OCT image resolution based on a lightweight network according to claim 2, characterized in that: For a B-scan OCT image to be processed, the method for obtaining N low longitudinal resolution images of different levels is as follows: The A-scan interference spectrum described in step 1 is spectrally shaped as in formula (1). S shape,N (k)=S(k)×Gaussian N (k) #(1) Where k represents the wave number, S(k) represents a series of A-scan interference spectra collected by the OCT system, and Gausian N (k) represents N Gaussian functions with different full width at half maximum (FWHM); S shape,N (k) represents an A-scan interference spectrum obtained after spectral shaping, and its full width at half maximum determines the longitudinal resolution of the B-scan OCT image; by adjusting the full width at half maximum of the Gaussian function, A-scan interference spectra with different spectral shaping are obtained, and then B-scan OCT images with different longitudinal resolutions are obtained; the specific steps can be one of the following two: (1) For the B-scan OCT image to be processed, each column of the A-scan interference spectrum is multiplied by the N Gaussian functions with different half-maximum full widths to obtain N sets of shaped spectra S shape,N (k) performing an inverse Fourier transform on each of the spectra at a different number of points, where the number of points of the inverse Fourier transform is determined by the resolution ratio r between each pair of adjacent resolution images, the number of points used in the obtained higher-resolution image among the adjacent resolutions being r times the number of points of the lower-resolution image, and the number of vertical pixels of the obtained higher-resolution image among the adjacent resolutions being also r times the number of vertical pixels of the lower-resolution image; obtaining a total of N sets of low-resolution A-scan OCT signals of different levels, each set of A-scan OCT signals forming a low-resolution B-scan OCT image, and obtaining a total of N low-resolution B-scan OCT images of different levels corresponding to the B-scan OCT image to be processed; (2) For the B-scan OCT image to be processed, each column of the A-scan interference spectrum is multiplied by the N Gaussian functions with different half-maximum full widths to obtain N sets of shaped spectra S shape,N (k) Perform an inverse Fourier transform of the same number of points on each spectrum to obtain a total of N sets of low-resolution A-scan OCT signals of different levels. Each set of A-scan OCT signals will form a low-resolution B-scan OCT image, and a total of N low-resolution B-scan OCT images with the same number of longitudinal pixels at different levels corresponding to the B-scan OCT image to be processed are obtained.

5. The method for improving OCT image resolution based on a lightweight network according to claim 2, characterized in that: For the en face OCT image to be processed, a digital defocus method is used to obtain N low lateral resolution images of different levels. The specific steps are as follows: Perform the operation (2) on the en face OCT image to be processed. Where x and y represent the coordinates of the two dimensions of the en face OCT image, z represents the depth position of the sample, and S(x,y,z) represents the en face OCT signal. and represent the transverse two-dimensional Fourier transform and inverse transform, respectively. are N spatial frequency filters corresponding to N defocus distances, where f x is the spatial frequency corresponding to the coordinate x, f y is the spatial frequency corresponding to the coordinate y, d N Represents the distance from the focus; S' N (x, y, z) represents the defocused en face OCT signal. Different levels of low-resolution images refer to en face OCT images at different defocus distances. The farther the distance from the focus, the lower the resolution. En face OCT images of different resolutions are obtained by adjusting the defocus distance of the spatial frequency filter. The specific steps can be one of the following two: (1) For the en face OCT image to be processed, its original signal is processed by N spatial frequency filters with different defocus distances according to formula (2) to obtain N low-resolution images of different levels; the N low-resolution images of different levels are downsampled at different levels, and the size of the downsampled image is determined by the resolution ratio r between each pair of adjacent resolution images, and the number of pixels in the x and y dimensions of the higher resolution image among the adjacent resolutions is r times the number of pixels in the x and y dimensions of the lower resolution image; a total of N low-resolution en face OCT images of different levels corresponding to the en face OCT image to be processed are obtained; (2) For the en face OCT image to be processed, its original signal is processed by N spatial frequency filters with different defocus distances according to formula (2) to obtain N low-resolution images of different levels. These N low-resolution images of different levels are not downsampled, and a total of N low-resolution en face OCT images of the same size and different levels corresponding to the en face OCT image to be processed are obtained.

6. The method for improving OCT image resolution based on a lightweight network according to claim 1, characterized in that: The constructed network model is a lightweight network with less than or equal to 50 layers, preferably, the number of layers is between 3 and 15 layers.

7. The method for improving OCT image resolution based on a lightweight network according to any one of claims 1 to 6, characterized in that: The value of N is a natural number between 2 and 20.

8. The method for improving OCT image resolution based on a lightweight network according to claim 7, characterized in that: The value of N is a natural number between 2 and 8.

9. The method for improving OCT image resolution based on a lightweight network according to any one of claims 1 to 6, characterized in that: The r value is a rational number between 1.05 and 6.

10. The method for improving OCT image resolution based on a lightweight network according to claim 9, characterized in that: The value of r is a rational number between 1.2 and 3.

Images