Optical coherence tomography image speckle denoising method, device and electronic equipment

Abstract

The present application provides a method, device, electronic device and storage medium for optical coherence tomography image speckle denoising, the method comprising: obtaining an initial optical coherence layer image output by an optical coherence tomography system; inputting the initial optical coherence tomography image into a pre-trained speckle denoising deep learning network model, and performing speckle denoising on the initial optical coherence tomography image by the speckle denoising deep learning network model to obtain a target optical coherence tomography image without speckle noise. The pre-trained speckle denoising deep learning network model can automatically extract speckle noise features, effectively eliminate speckle noise, and obtain a high-quality optical coherence layer image.

Description

Technical Field

[0001] The present application relates to the technical field of optical imaging denoising methods, and in particular to an optical coherence tomography image speckle denoising method, device, electronic device and storage medium. Background Art

[0002] Optical Coherence Tomography (OCT) can achieve powerful non-invasive, high-resolution three-dimensional imaging of biological tissues and materials, and has been widely used in biomedicine and other fields. However, OCT relies on the coherence of backscattered light to distinguish the morphology of biological tissues. This coherence inevitably produces speckle noise, resulting in a decrease in spatial resolution. Moreover, this speckle noise can mask the subtle but very important microstructures in the tissue, thus having a negative impact on clinical diagnosis and reducing the accuracy of diagnosis.

[0003] In the prior art, methods for removing speckle noise are divided into hardware methods and software methods. The hardware system in the hardware method has high design complexity and requires repeated scanning to reduce the imaging time resolution. The images processed by the software method are often too smooth and lose important detail features. Summary of the invention

[0004] The purpose of the present application is to address the deficiencies in the above-mentioned prior art and to provide a method, device, electronic device and storage medium for optical coherence tomography image speckle denoising, which can automatically extract speckle noise characteristics, effectively eliminate speckle noise and obtain high-quality optical coherence tomography images.

[0005] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for denoising speckle in an optical coherence tomography image, the method comprising:

[0007] Acquiring an initial optical coherence tomography image output by an optical coherence tomography system, wherein the initial optical coherence tomography image contains speckle noise;

[0008] The initial optical coherence tomography image is input into a pre-trained deep learning network model for speckle denoising, and the deep learning network model for speckle denoising performs speckle denoising on the initial optical coherence tomography image to obtain a target optical coherence tomography image without speckle noise.

[0009] Optionally, the speckle denoising deep learning network model includes: an encoder and a decoder connected in sequence;

[0010] The step of inputting the initial optical coherence tomography image into a pre-trained speckle denoising deep learning network model, and performing speckle denoising on the initial optical coherence tomography image by the speckle denoising deep learning network model to obtain a target optical coherence tomography image without speckle noise comprises:

[0011] The initial optical coherence tomography image is input into the encoder of the speckle denoising deep learning network model, the encoder performs down-sampling processing on the initial optical coherence tomography image, and the result of the down-sampling processing is sent to the decoder of the speckle denoising deep learning network model, and the decoder performs up-sampling processing on the result of the down-sampling processing to obtain the target optical coherence tomography image.

[0012] Optionally, before inputting the initial optical coherence tomography image into a pre-trained speckle denoising deep learning network model, and performing speckle denoising on the initial optical coherence tomography image by the speckle denoising deep learning network model, the method further comprises:

[0013] Constructing a sample data set, wherein the sample data set includes a plurality of image pairs, each image pair includes a reference image and a noise image, wherein the reference image does not contain speckle noise, and the noise image contains speckle noise;

[0014] Constructing an initial model, the initial model comprising: a generator and a discriminator, the generator comprising an encoder and a decoder connected in sequence;

[0015] The initial model is trained based on the sample data set to obtain the speckle denoising deep learning network model.

[0016] Optionally, constructing a sample data set includes:

[0017] Scanning any sample object using a speckle modulation optical coherence tomography system to obtain a plurality of initial images corresponding to the sample object, wherein a movable optical diffuse reflector is included between a first lens and a second lens in the speckle modulation optical coherence tomography system;

[0018] Performing image averaging processing on the multiple initial images to obtain a reference image corresponding to the sample object;

[0019] Using each of the initial images as a noise image in the image pair, and combining them with the reference image to form an image pair;

[0020] The image pairs corresponding to the sample objects are combined to obtain the sample data set.

[0021] Optionally, the training the initial model based on the sample data set to obtain the speckle denoising deep learning network model includes:

[0022] A. Inputting the noise image in each image pair of the sample data set into the generator, the encoder and the decoder in the generator sequentially perform downsampling and upsampling on the noise image, and inputting the processed image into the discriminator, and the discriminator discriminates the true probabilities of the processed image and the reference image;

[0023] B. determining first loss information of the generator according to the processed image and the reference image;

[0024] C. determining second loss information of the discriminator according to the processed image output by the discriminator and the true probability of the reference image;

[0025] D. Correcting the network parameters of the generator and the network parameters of the discriminator according to the first loss information and the second loss information, respectively, to obtain a corrected generator and a corrected discriminator;

[0026] Steps AD are iteratively performed until the first loss information of the generator and the second loss information of the discriminator respectively meet preset conditions, and the generator is used as the speckle denoising deep learning network model.

[0027] Optionally, determining the first loss information of the generator according to the processed image and the reference image includes:

[0028] According to the processed image and the reference image, respectively calculating the pixel mean square loss and the perceptual loss of the generator;

[0029] The first loss information is obtained according to the pixel mean square loss and the perceptual loss.

[0030] Optionally, determining the second loss information of the discriminator according to the true probability and the false probability of the processed image and the reference image output by the discriminator includes:

[0031] The second loss information of the discriminator is calculated according to the difference between the true probability of the processed image and the true probability of the reference image.

[0032] In a second aspect, an embodiment of the present application further provides an optical coherence tomography image speckle denoising device, the device comprising:

[0033] An acquisition module, used for acquiring an initial optical coherence tomography image output by the optical coherence tomography system, wherein the initial optical coherence tomography image contains speckle noise;

[0034] a denoising module, configured to input the initial optical coherence tomography image into a pre-trained speckle denoising deep learning network model, and perform speckle denoising on the initial optical coherence tomography image by the speckle denoising deep learning network model to obtain a target optical coherence tomography image without speckle noise.

[0035] Optionally, the speckle denoising deep learning network model includes: an encoder and a decoder connected in sequence;

[0036] The denoising module is specifically used for:

[0037] The initial optical coherence tomography image is input into the encoder of the speckle denoising deep learning network model, the encoder performs down-sampling processing on the initial optical coherence tomography image, and the result of the down-sampling processing is sent to the decoder of the speckle denoising deep learning network model, and the decoder performs up-sampling processing on the result of the down-sampling processing to obtain the target optical coherence tomography image.

[0038] Optionally, the denoising module is specifically used for:

[0039] Constructing a sample data set, wherein the sample data set includes a plurality of image pairs, each image pair includes a reference image and a noise image, wherein the reference image does not contain speckle noise, and the noise image contains speckle noise;

[0040] Constructing an initial model, the initial model comprising: a generator and a discriminator, the generator comprising an encoder and a decoder connected in sequence;

[0041] The initial model is trained based on the sample data set to obtain the speckle denoising deep learning network model.

[0042] Optionally, the denoising module is specifically used for:

[0043] Scanning any sample object using a speckle modulation optical coherence tomography system to obtain a plurality of initial images corresponding to the sample object, wherein a movable optical diffuse reflector is included between a first lens and a second lens in the speckle modulation optical coherence tomography system;

[0044] Performing image averaging processing on the multiple initial images to obtain a reference image corresponding to the sample object;

[0045] Using each of the initial images as a noise image in the image pair, and combining them with the reference image to form an image pair;

[0046] The image pairs corresponding to the sample objects are combined to obtain the sample data set.

[0047] Optionally, the denoising module is specifically used for:

[0048] A. Inputting the noise image in each image pair of the sample data set into the generator, the encoder and the decoder in the generator sequentially perform downsampling and upsampling on the noise image, and inputting the processed image into the discriminator, and the discriminator discriminates the true probabilities of the processed image and the reference image;

[0049] B. determining first loss information of the generator according to the processed image and the reference image;

[0050] C. determining second loss information of the discriminator according to the processed image output by the discriminator and the true probability of the reference image;

[0051] D. Correcting the network parameters of the generator and the network parameters of the discriminator according to the first loss information and the second loss information, respectively, to obtain a corrected generator and a corrected discriminator;

[0052] Steps AD are iteratively performed until the first loss information of the generator and the second loss information of the discriminator respectively meet preset conditions, and the generator is used as the speckle denoising deep learning network model.

[0053] Optionally, the denoising module is specifically used for:

[0054] According to the processed image and the reference image, respectively calculating the pixel mean square loss and the perceptual loss of the generator;

[0055] The first loss information is obtained according to the pixel mean square loss and the perceptual loss.

[0056] Optionally, the denoising module is specifically used for:

[0057] The second loss information of the discriminator is calculated according to the difference between the true probability of the processed image and the true probability of the reference image.

[0058] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, and when an application is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the steps of the optical coherence tomography image speckle denoising method described in the first aspect above.

[0059] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and the computer program is read and executes the steps of the optical coherence tomography image speckle denoising method described in the first aspect above.

[0060] The beneficial effects of this application are:

[0061] The present application provides an optical coherence tomography image speckle denoising method, device, electronic device and storage medium, which obtains an initial optical coherence layer image output by an optical coherence tomography system; inputs the initial optical coherence tomography image into a pre-trained speckle denoising deep learning network model, and performs speckle denoising on the initial optical coherence tomography image by the speckle denoising deep learning network model to obtain a target optical coherence tomography image without speckle noise. The pre-trained speckle denoising deep learning network model can automatically extract speckle noise features, effectively eliminate speckle noise, and obtain a high-quality optical coherence layer image. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 A schematic diagram of an application scenario of an optical coherence tomography image speckle denoising method provided in an embodiment of the present application;

[0064] Figure 2 A schematic diagram of a process of a method for denoising speckle in an optical coherence tomography image provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of a flow chart of another optical coherence tomography image speckle denoising method provided in an embodiment of the present application;

[0066] Figure 4 A schematic diagram of the structure of the initial model provided in the embodiment of the present application;

[0067] Figure 5 A flowchart of a method for constructing a sample data set provided in an embodiment of the present application;

[0068] Figure 6 A schematic diagram of the structure of a speckle modulated optical coherence tomography system provided in an embodiment of the present application;

[0069] Figure 7A schematic diagram of a flow chart of a method for initial model training provided in an embodiment of the present application;

[0070] Figure 8 A schematic diagram of the system structure for initial model training provided in an embodiment of the present application;

[0071] Fig. 9 A schematic diagram of a device for an optical coherence tomography image speckle denoising method provided in an embodiment of the present application;

[0072] Fig.10 A structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0074] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0075] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0076] Figure 1 A schematic diagram of an application scenario of an optical coherence tomography image speckle denoising method provided in an embodiment of the present application is a schematic diagram of a structure, such as Figure 1As shown, the present application is applicable to the scenario of obtaining an OCT image of any sample. The sample may be, for example, a biological sample, etc. The scenario involves an optical coherence tomography (OCT) system and an electronic device. First, the OCT system is used to scan any sample to output an OCT image with speckle noise, and then the OCT image with speckle noise is input into the above-mentioned electronic device, and the electronic device performs speckle denoising on the OCT image using the trained speckle denoising deep learning network model, and finally obtains an OCT image without speckle noise.

[0077] Figure 2 A schematic diagram of a process flow of an optical coherence tomography image speckle denoising method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method is applied to the above-mentioned electronic device, which can be, for example, a terminal device with computing and display capabilities such as a desktop computer or a notebook computer, or a server. The method includes:

[0078] S101, acquiring an initial optical coherence layer image output by an optical coherence tomography system.

[0079] Optionally, the OCT system may include a light source, a coupler, a reference mirror, a sample arm, a reference arm, and a spectrometer. The specific principle is: the system may use a broadband light source as the light source, for example, a broadband light source with a central wavelength of 850nm and a half-maximum full width of 165nm may be used. The light emitted by the broadband light source passes through the coupler through the sample arm and the reference arm to irradiate the sample and the reference mirror respectively. The reflected light in the two optical paths converges in the coupler to generate an interference signal under certain conditions, and the interference signal is output to the spectrometer. The spectrometer transmits the detected interference signal to the computer, and the computer performs grayscale imaging and outputs an initial OCT image. The initial OCT image has speckle noise.

[0080] S102, inputting the initial optical coherence tomography image into a pre-trained deep learning network model for speckle denoising, and using the deep learning network model for speckle denoising to perform speckle denoising on the initial optical coherence tomography image to obtain a target optical coherence tomography image without speckle noise.

[0081] Optionally, the speckle denoising deep learning network model is trained according to the method provided in the present application, and the trained speckle denoising deep learning network model can be combined with the OCT system in the above step S101, and the initial OCT image acquired in the above step S101 can be input into the trained speckle denoising deep learning network model, and the trained speckle denoising deep learning network model is used to perform speckle denoising on the initial OCT image to obtain a target OCT image without speckle noise.

[0082] In this embodiment, an initial optical coherence layer image output by an optical coherence tomography system is obtained; the initial optical coherence tomography image is input into a pre-trained speckle denoising deep learning network model, and the speckle denoising deep learning network model performs speckle denoising on the initial optical coherence tomography image to obtain a target optical coherence tomography image without speckle noise. The pre-trained speckle denoising deep learning network model can automatically extract speckle noise features, effectively eliminate speckle noise, and obtain a high-quality optical coherence layer image.

[0083] Optionally, the above-mentioned speckle denoising deep learning network model may include an encoder and a decoder connected in sequence.

[0084] Among them, the encoder may include a convolution layer, a pooling layer and a dense connection block, the decoder may include a deconvolution layer, a convolution layer and a dense connection block, the layers can be connected through dense connection blocks, the image output by the encoder can be input to the decoder for decoding, and finally the target image is output by the decoder.

[0085] Optionally, step S102 of inputting the initial optical coherence tomography image into a pre-trained speckle denoising deep learning network model, and performing speckle denoising on the initial optical coherence tomography image by the speckle denoising deep learning network model to obtain a target optical coherence tomography image without speckle noise may include:

[0086] Optionally, the noisy initial OCT image is input to the encoder of the speckle denoising deep learning network model, and the encoder downsamples the initial optical coherence tomography image, and sends the downsampled result to the decoder of the speckle denoising deep learning network model, and the decoder upsamples the downsampled result to obtain the target OCT image. Specifically, the initial OCT image is input to the encoder of the speckle denoising deep learning network model, and the convolution layer in the encoder is used to upsample the downsampled result. 、 The densely connected blocks and pooling layers are used for downsampling to extract image features. The feature images extracted by downsampling are sent to the encoder. The image size is restored through the convolutional layers, densely connected blocks and deconvolutional layers in the decoder. The image size is consistent with the original OCT image size. The image after feature extraction is then output and used as the target OCT image, which is a noise-free OCT image after speckle denoising.

[0087] The following embodiments illustrate the training process of the above-mentioned speckle denoising deep learning network model.

[0088] Figure 3 A schematic diagram of a process flow of another optical coherence tomography image speckle denoising method provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, in the above step S102, the initial optical coherence tomography image is input into the pre-trained speckle denoising deep learning network model. Before the speckle denoising deep learning network model performs speckle denoising on the initial optical coherence tomography image, the process of training the speckle denoising deep learning network model may include:

[0089] S201: Build a sample data set.

[0090] Optionally, the sample data set may include a plurality of image pairs, each image pair includes a reference image and a noise image, wherein the reference image does not contain speckle noise, and the noise image contains speckle noise.

[0091] S202: construct an initial model.

[0092] Optionally, the initial model may include a generator and a discriminator, wherein the generator may include an encoder and a decoder connected in sequence, and a noise-free OCT image may be generated by the encoder and the decoder in the generator.

[0093] Exemplarily, the generator can be a U-Net structure consisting of an encoder and a decoder constructed by a densely connected network.

[0094] Optionally, the discriminator may include convolution and activation functions, etc. The discriminator may discriminate the difference between the image and the real image, thereby discriminating whether the image is a real image or a fake image, and outputting a true probability and a false probability.

[0095] Optionally, the generator and the discriminator can communicate with each other, and the image generated by the generator can be sent to the discriminator, which determines whether the generated image is a real image or a fake image, so that the generator can be trained based on the judgment result of the discriminator. Figure 4 A schematic diagram of the structure of the initial model provided in the embodiment of the present application, such as Figure 4 As shown, the input and output on the left are the input and output of the generator, and the structure for distinguishing true from false on the right is a schematic diagram of the discriminator structure.

[0096] S203: Train the initial model based on the sample data set to obtain a speckle denoising deep learning network.

[0097] Optionally, the initial model may be trained using a preset method using multiple image pairs in the sample data set in S201. Specifically, the generator and the discriminator in the initial model are trained using a preset method so that the generator can generate a speckle-free OCT image that is close enough to the real image, so that the discriminator cannot distinguish between the image generated by the generator and the real image. At the same time, the trained discriminator can identify the difference between the image generated by the generator and the real image as much as possible, that is, the generator can generate high-quality images, and the discriminator cannot distinguish between the true and false images. The trained initial model is used as a speckle denoising deep learning network.

[0098] Figure 5 A flow chart of a method for constructing a sample data set provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the above step S201 constructs a sample data set, which may include:

[0099] S301 , scanning any sample using a speckle modulation optical coherence tomography system to obtain a plurality of initial images corresponding to the sample object.

[0100] Figure 6 A schematic diagram of the structure of a speckle modulation optical coherence tomography system provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the speckle modulation optical coherence tomography system includes a spectrometer, a fiber coupler, a collimating lens, a first lens, a second lens, a third lens, a fourth lens, an objective lens, glass, a two-dimensional scanning galvanometer, a first polarization controller, a second polarization controller, a first mirror, a second mirror, and a movable optical diffuse reflector.

[0101] Optionally, the speckle modulation optical coherence tomography system is further provided with a movable optical diffuse reflector relative to the optical coherence tomography system. Figure 6 A movable optical diffuse reflector is included between the first lens and the second lens in the embodiment. When scanning the sample object, an initial image can be acquired by moving the optical diffuse reflector. For example, the initial image can be acquired by moving the optical diffuse reflector up and down.

[0102] Exemplarily, an initial image can be obtained by moving the optical diffuse reflector once for scanning, and multiple initial images can be obtained by continuously moving the optical diffuse reflector multiple times for scanning. The multiple initial images obtained by moving the optical diffuse reflector for scanning are multiple images with speckle noise, and the speckle noises of the multiple initial images are irrelevant. According to the feature that the speckle noises of the multiple initial images are irrelevant, the reference image obtained according to the processing method of step S302 can be a reference image without speckle noise.

[0103] Optionally, a plurality of sample objects may be scanned respectively using a speckle modulation optical coherence tomography system to obtain a plurality of initial images corresponding to each sample, wherein the sample objects may include vegetables, fruits, meat, etc.

[0104] Exemplarily, the plurality of initial images may be, for example, 100 initial images.

[0105] S302 , performing image averaging processing on a plurality of initial images respectively to obtain a reference image corresponding to the sample object.

[0106] Optionally, the multiple initial images corresponding to each sample are averaged respectively. For example, if a sample object is scanned in S301 to obtain 100 initial images, the 100 images are averaged, and the image after the average calculation is used as the reference image corresponding to the sample object. Specifically, the pixel values ​​of the multiple initial images can be averaged, and the image corresponding to the pixel value after the average calculation is used as the reference image corresponding to the sample object, and the reference image is a reference image without speckle noise.

[0107] S303 , taking each initial image as a noise image in an image pair, and combining each initial image with a reference image to form an image pair.

[0108] Optionally, for each sample object, select one of the corresponding multiple initial images and combine it with the reference image corresponding to the sample object to obtain an image pair. By using this method, combine other initial images with the reference image respectively to obtain multiple image pairs, and each image pair includes a noise image and a reference image.

[0109] Exemplarily, if there are 100 initial images corresponding to a sample object, then the sample object may have 100 image pairs.

[0110] S304: Combine the image pairs corresponding to the sample objects to obtain a sample data set.

[0111] Among them, the sample object can be one sample object or multiple sample objects. In this case, the data set composed of all image pairs corresponding to all sample objects can be used as the sample data set, and the sample data sets with preset proportions can be used as the training sample data set and the test sample data set respectively. For example, 80% of the sample data set can be used as the training sample data set, and 20% of the sample data set can be used as the test sample data set.

[0112] For example, if there are 1000 sample objects, then each sample object can obtain 100 image pairs corresponding to each sample object through the above S301 - S303 , and the sample data set includes 1000×100 image pairs.

[0113] In this embodiment, the noisy image and the reference image obtained by the modified OCT system can make the obtained reference image closer to the real image and make the trained initial model more optimized.

[0114] Figure 7 A flow chart of the method for initial model training provided in the embodiment of the present application is as follows: Figure 7 As shown, the above step S203 trains the initial model based on the sample data set to obtain a speckle denoising deep learning network, including:

[0115] S401. Input the noise image in each image pair of the sample data set into the generator, and the encoder and decoder in the generator sequentially downsample and upsample the input noise image, and input the processed image into the discriminator, which discriminates the true probability of the processed image and the reference image.

[0116] Optionally, before the noise image in each image pair in the sample data set is input into the generator, a normalization method can be used to normalize the training sample data set and the test sample data set in the sample data set. Specifically, the noise image in each image pair and the corresponding reference image are selected for normalization. Specifically, the pixel values ​​of the noise image and the reference image in each image pair can be normalized, and the grayscale image of 256 brightness levels in the image can be selected through appropriate thresholds to obtain an image that can still reflect the overall and local features of the image. Multi-level values ​​of pixels are no longer involved in image processing, making the image simpler in subsequent processing.

[0117] Optionally, the noise image in each image pair in the normalized training sample data set is input into the generator, the encoder in the generator downsamples the input noise image, and the decoder upsamples the downsampled noise image to obtain an image after downsampling and upsampling. The generator inputs the output processed image into the discriminator, and the discriminator inputs a reference image corresponding to the noise image. The discriminator determines the difference between the processed image and the corresponding reference image, thereby determining whether the processed image is a true image or a false image, and outputs the true probability of the processed image and the reference image.

[0118] S402. Determine first loss information of the generator according to the processed image and the reference image.

[0119] Optionally, the first loss information of the generator may be determined by using a preset formula according to the processed image and the reference image, wherein the first loss information may include pixel mean square loss and perceptual loss.

[0120] S403: Determine second loss information of the discriminator according to the processed image output by the discriminator and the true probability of the reference image.

[0121] Optionally, a preset formula may be used according to the processed image output by the discriminator and the true probability of the reference image to determine the second loss information of the discriminator.

[0122] S404. Correct the network parameters of the generator and the network parameters of the discriminator according to the first loss information and the second loss information, respectively, to obtain a corrected generator and a corrected discriminator.

[0123] Among them, the network parameters of the generator may include convolution parameters, deconvolution parameters, pooling parameters, etc., and the network parameters of the discriminator may include convolution parameters, activation parameters, etc.

[0124] Optionally, the above steps S401-S404 are iteratively performed. Specifically, an image pair is iteratively selected from the training sample data set, and the above steps S401-S404 are respectively performed, and the network parameters of the generator and the discriminator are continuously corrected until the first loss information of the generator and the second loss information of the discriminator respectively meet the preset conditions, and the corrected generator and discriminator are tested using the test data set. After the test passes, the final corrected generator is used as the speckle denoising deep learning network model.

[0125] Exemplarily, the preset condition may be that the first loss information and the second loss information tend to be stable.

[0126] Figure 8 A schematic diagram of the system structure of the initial model training provided in the embodiment of the present application, such as Figure 8 As shown, the training system includes sample data set preparation and initial model training.

[0127] The specific implementation method for preparing the sample data set has been described in detail in the above specific implementation manner and will not be repeated here.

[0128] Optionally, the noise image in the training sample data set is forward propagated into the initial model for training and the processed image is output. The loss between the processed image and the reference image is calculated. The loss value is back-propagated and the network parameters of the generator and discriminator in the initial model are optimized in the direction of gradient descent. At the same time, the trained initial model is tested using the test sample data set.

[0129] In this embodiment, by using the sample data set to iteratively correct the initial model, the corrected initial model can remove the speckle noise while also being able to analyze the tiny but important microstructures masked by the speckle noise.

[0130] Optionally, the above S402 determines the first loss information of the generator according to the processed image and the reference image, including:

[0131] Optionally, the pixel mean square loss and perceptual loss of the generator are calculated based on the processed image and the reference image, respectively.

[0132] Optionally, the pixel mean square loss can be calculated according to formula (1).

[0133]

[0134] Among them, L 2 represents pixel mean square loss, x represents the processed image, y represents the reference image, G(X i ) represents the i-th pixel value of the processed image, y i represents the i-th pixel value of the reference image, n represents the total number of image pixels, and the mean square error of the pixel values ​​of the processed image and the reference image is calculated according to the above formula (1) to obtain the pixel mean square loss.

[0135] Optionally, the perceptual loss can be calculated according to formula (2).

[0136]

[0137] Among them, L VGG represents the perceptual loss, x represents the processed image, y represents the reference image, VGG 19 (y i ) represents the i-th pixel value of the reference image extracted by the VGG-19 network feature extraction, VGG 19 (G(x i )) represents the i-th pixel value of the processed image after feature extraction by the VGG-19 network, w represents the width of the image, h represents the height of the image, and d represents the number of channels of the image. As can be seen from the above, the size of the processed image is consistent with that of the reference image.

[0138] Optionally, the content loss can be calculated using formula (III) according to the pixel mean square loss and perceptual loss calculated above.

[0139] L content =α×L 2 +β×L VGG Formula (III)

[0140] Among them, L content Indicates content loss, L 2 represents the pixel mean square loss, L VGG represents the perceptual loss, α and β represent the coefficients of pixel mean square loss and perceptual loss, and illustratively, α may be 1, and β may be 0.1, for example.

[0141] Optionally, the first loss information includes the calculated pixel mean square loss, perceptual loss and content loss, and gradient optimization is performed on the network parameters of the generator according to the calculated content loss.

[0142] Optionally, the above step S403 determines the second loss information of the discriminator according to the processed image output by the discriminator and the true probability of the reference image, including:

[0143] Optionally, the second loss information of the discriminator is calculated according to the difference between the true probability of the processed image and the true probability of the reference image. Specifically, it can be calculated according to formula (IV).

[0144]

[0145] Among them, L discriminator represents the discriminator loss, x represents the processed image, y represents the reference image, D(y i ) represents the probability value of the i-th pixel of the reference image being judged as true, D[G(x i )] represents the probability value of the i-th pixel of the processed image being judged as true, 1-D[G(x i )] represents the probability value of the i-th pixel of the processed image being judged as false. The discriminator loss calculated using the cross entropy function is the second loss information. The network parameters of the discriminator are gradient optimized according to the calculated discriminator loss.

[0146] This embodiment uses the first loss information and the second loss information to continuously optimize the network parameters of the generator and the network parameters of the discriminator, so that the difference between the image generated by the generator and the real image can be minimized, and the discriminator can maximize the ability to distinguish the authenticity of the image.

[0147] Fig. 9 A schematic diagram of a device for an optical coherence tomography image speckle denoising method provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the device comprises:

[0148] An acquisition module 501 is used to acquire an initial optical coherence tomography image output by an optical coherence tomography system, wherein the initial optical coherence tomography image contains speckle noise;

[0149] The denoising module 502 is configured to input the initial optical coherence tomography image into a pre-trained speckle denoising deep learning network model, and perform speckle denoising on the initial optical coherence tomography image by the speckle denoising deep learning network model to obtain a target optical coherence tomography image without speckle noise.

[0150] Optionally, the denoising module 502 is specifically used for:

[0151] The initial optical coherence tomography image is input into the encoder of the speckle denoising deep learning network model, the encoder performs down-sampling processing on the initial optical coherence tomography image, and the result of the down-sampling processing is sent to the decoder of the speckle denoising deep learning network model, and the decoder performs up-sampling processing on the result of the down-sampling processing to obtain the target optical coherence tomography image.

[0152] Optionally, the denoising module 502 is specifically used for:

[0153] Constructing a sample data set, wherein the sample data set includes a plurality of image pairs, each image pair includes a reference image and a noise image, wherein the reference image does not contain speckle noise, and the noise image contains speckle noise;

[0154] Constructing an initial model, the initial model comprising: a generator and a discriminator, the generator comprising an encoder and a decoder connected in sequence;

[0155] The initial model is trained based on the sample data set to obtain the speckle denoising deep learning network model.

[0156] Optionally, the denoising module 502 is specifically used for:

[0157] Scanning any sample object using a speckle modulation optical coherence tomography system to obtain a plurality of initial images corresponding to the sample object, wherein a movable optical diffuse reflector is included between a first lens and a second lens in the speckle modulation optical coherence tomography system;

[0158] Performing image averaging processing on the multiple initial images to obtain a reference image corresponding to the sample object;

[0159] Using each of the initial images as a noise image in the image pair, and combining them with the reference image to form an image pair;

[0160] The image pairs corresponding to the sample objects are combined to obtain the sample data set.

[0161] Optionally, the denoising module 502 is specifically used for:

[0162] A. Inputting the noise image in each image pair of the sample data set into the generator, the encoder and the decoder in the generator sequentially perform downsampling and upsampling on the noise image, and inputting the processed image into the discriminator, and the discriminator discriminates the true probabilities of the processed image and the reference image;

[0163] B. determining first loss information of the generator according to the processed image and the reference image;

[0164] C. determining second loss information of the discriminator according to the processed image output by the discriminator and the true probability of the reference image;

[0165] D. Correcting the network parameters of the generator and the network parameters of the discriminator according to the first loss information and the second loss information, respectively, to obtain a corrected generator and a corrected discriminator;

[0166] Steps AD are iteratively performed until the first loss information of the generator and the second loss information of the discriminator respectively meet preset conditions, and the generator is used as the speckle denoising deep learning network model.

[0167] Optionally, the denoising module 502 is specifically used for:

[0168] According to the processed image and the reference image, respectively calculating the pixel mean square loss and the perceptual loss of the generator;

[0169] The first loss information is obtained according to the pixel mean square loss and the perceptual loss.

[0170] Optionally, the denoising module 502 is specifically used for:

[0171] The second loss information of the discriminator is calculated according to the difference between the true probability of the processed image and the true probability of the reference image.

[0172] Fig.10 FIG. 6 is a structural block diagram of an electronic device 600 provided in an embodiment of the present application. Fig.10 As shown, the electronic device may include: a processor 601 and a memory 602 .

[0173] Optionally, a bus 603 may also be included, wherein the memory 602 is used to store machine-readable instructions executable by the processor 601. When the electronic device 600 is running, the processor 601 communicates with the memory 602 through the bus 603. When the machine-readable instructions are executed by the processor 601, the method steps in the above method embodiment are performed.

[0174] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method steps in the embodiment of the above-mentioned optical coherence tomography image speckle denoising method are executed.

[0175] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0176] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program code.

[0177] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A method for denoising speckle in optical coherence tomography images, characterized in that: The method comprises: Acquiring an initial optical coherence tomography image output by an optical coherence tomography system, wherein the initial optical coherence tomography image contains speckle noise; Inputting the initial optical coherence tomography image into a pre-trained speckle denoising deep learning network model, and using the speckle denoising deep learning network model to perform speckle denoising on the initial optical coherence tomography image to obtain a target optical coherence tomography image without speckle noise; Before inputting the initial optical coherence tomography image into a pre-trained deep learning network model for speckle denoising, and performing speckle denoising on the initial optical coherence tomography image by the deep learning network model for speckle denoising, the method further comprises: Constructing a sample data set, wherein the sample data set includes a plurality of image pairs, each image pair includes a reference image and a noise image, wherein the reference image does not contain speckle noise, and the noise image contains speckle noise; Constructing an initial model, the initial model comprising: a generator and a discriminator, the generator comprising an encoder and a decoder connected in sequence; Training the initial model based on the sample data set to obtain the speckle denoising deep learning network model; The constructing of the sample data set includes: A speckle modulation optical coherence tomography system is used to scan any sample object to obtain a plurality of initial images corresponding to the sample object, wherein a movable optical diffuse reflector is included between a first lens and a second lens in the speckle modulation optical coherence tomography system, each of the initial images is an image with speckle noise, and the speckle noise of each of the initial images is irrelevant to each other; Performing image averaging processing on the multiple initial images to obtain a reference image corresponding to the sample object; Using each of the initial images as a noise image in the image pair, and combining them with the reference image to form an image pair; The image pairs corresponding to the sample objects are combined to obtain the sample data set.

2. The optical coherence tomography image speckle denoising method according to claim 1, characterized in that: The speckle denoising deep learning network model comprises: an encoder and a decoder connected in sequence; The step of inputting the initial optical coherence tomography image into a pre-trained speckle denoising deep learning network model, and performing speckle denoising on the initial optical coherence tomography image by the speckle denoising deep learning network model to obtain a target optical coherence tomography image without speckle noise comprises: The initial optical coherence tomography image is input into the encoder of the speckle denoising deep learning network model, the encoder performs down-sampling processing on the initial optical coherence tomography image, and the result of the down-sampling processing is sent to the decoder of the speckle denoising deep learning network model, and the decoder performs up-sampling processing on the result of the down-sampling processing to obtain the target optical coherence tomography image.

3. The optical coherence tomography image speckle denoising method according to claim 1, characterized in that: The training of the initial model based on the sample data set to obtain the speckle denoising deep learning network model includes: A. Inputting the noise image in each image pair of the sample data set into the generator, the encoder and the decoder in the generator sequentially perform downsampling and upsampling on the noise image, and inputting the processed image into the discriminator, and the discriminator discriminates the true probabilities of the processed image and the reference image; B. determining first loss information of the generator according to the processed image and the reference image; C. determining second loss information of the discriminator according to the processed image output by the discriminator and the true probability of the reference image; D. Correcting the network parameters of the generator and the network parameters of the discriminator according to the first loss information and the second loss information, respectively, to obtain a corrected generator and a corrected discriminator; Steps AD are iteratively performed until the first loss information of the generator and the second loss information of the discriminator respectively meet preset conditions, and the generator is used as the speckle denoising deep learning network model.

4. The optical coherence tomography image speckle denoising method according to claim 3, characterized in that: The determining the first loss information of the generator according to the processed image and the reference image comprises: According to the processed image and the reference image, respectively calculating the pixel mean square loss and the perceptual loss of the generator; The first loss information is obtained according to the pixel mean square loss and the perceptual loss.

5. The optical coherence tomography image speckle denoising method according to claim 3, characterized in that: The step of determining the second loss information of the discriminator according to the processed image output by the discriminator and the true probability and the false probability of the reference image comprises: The second loss information of the discriminator is calculated according to the difference between the true probability of the processed image and the true probability of the reference image.

6. An optical coherence tomography image speckle denoising device, characterized in that: include: An acquisition module, used for acquiring an initial optical coherence tomography image output by the optical coherence tomography system, wherein the initial optical coherence tomography image contains speckle noise; a denoising module, configured to input the initial optical coherence tomography image into a pre-trained speckle denoising deep learning network model, and perform speckle denoising on the initial optical coherence tomography image by the speckle denoising deep learning network model to obtain a target optical coherence tomography image without speckle noise; The denoising module specifically includes: Constructing a sample data set, wherein the sample data set includes a plurality of image pairs, each image pair includes a reference image and a noise image, wherein the reference image does not contain speckle noise, and the noise image contains speckle noise; Constructing an initial model, the initial model comprising: a generator and a discriminator, the generator comprising an encoder and a decoder connected in sequence; The initial model is trained based on the sample data set to obtain the speckle denoising deep learning network model, The constructing of the sample data set includes: A speckle modulation optical coherence tomography system is used to scan any sample object to obtain a plurality of initial images corresponding to the sample object, wherein a movable optical diffuse reflector is included between a first lens and a second lens in the speckle modulation optical coherence tomography system, each of the initial images is an image with speckle noise, and the speckle noise of each of the initial images is irrelevant to each other; Performing image averaging processing on the multiple initial images to obtain a reference image corresponding to the sample object; Using each of the initial images as a noise image in the image pair, and combining them with the reference image to form an image pair; The image pairs corresponding to the sample objects are combined to obtain the sample data set.

7. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, the steps of the optical coherence tomography image speckle denoising method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the optical coherence tomography image speckle denoising method according to any one of claims 1 to 5 are executed.

Images