A bright field microscope flat field map real-time prediction method based on neural network

Abstract

This invention relates to a real-time prediction method for bright-field microscope plan images based on neural networks. The method includes acquiring images of bright-field microscopes from multiple scenes and their corresponding plan images; obtaining the corresponding brightness-corrected images according to the plan correction formula; designing and constructing a neural network architecture to directly predict the plan images of the microscope images; constructing a loss function to train the neural network architecture for predicting the plan images; and training the neural network architecture for predicting the plan images of the microscope images: using the acquired images as a training set, the designed loss function is used to train the neural network architecture to train the corresponding model; finally, brightness non-uniformity correction is applied to the microscope images. This invention, through the design of an efficient neural network architecture, directly predicts the corresponding plan image from the input image, enabling real-time and rapid correction of brightness non-uniformity in microscope images. It is unaffected by the number of images, the contents of the images, or the color, size, and density of objects in the images.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for real-time prediction of bright-field microscope plan images based on neural networks. Background Technology

[0002] Bright-field microscopes are the most commonly used type of microscope, playing a vital role, especially in the medical field. However, due to factors such as hardware precision and optical path system, the brightness of the acquired microscopic images is uneven, which affects the quantification of important information such as the color and texture of the target object in the image. Therefore, brightness unevenness correction in bright-field microscopes is particularly important.

[0003] Brightness non-uniformity correction in bright-field microscopy requires the use of a flat-field correction formula. This formula requires inputting the corresponding flat-field and dark-field images. In bright-field microscopy, the dark-field image is often ignored. Therefore, obtaining the flat-field image is crucial for completing the brightness non-uniformity correction of bright-field micrographs.

[0004] Currently, traditional computer vision techniques are often used to directly predict and simulate the corresponding flat-field image by acquiring information from multiple images. For example, the 2014 paper "CIDRE: An illumination correction method for optical microscopy", the 2017 paper "A BaSic tool for background and shading correction of optical microscopy images", and the 2020 paper "Simple ShadingCorrection Method for Brightfield Whole Slide Imageing" all proposed methods for predicting and simulating flat-field images using multiple images with traditional computer vision techniques. This method mainly has the following three problems: First, it is greatly affected by the number of images. Generally, the more images, the better the result, and the result is not ideal when the number of images is small. Second, it is easily affected by factors such as the color, size, and density distribution of the target object in the image. The true flat-field image should be unrelated to the target object. Third, it requires acquiring multiple images to simulate the flat-field image, and then performing brightness non-uniformity correction on the acquired images one by one. It cannot perform brightness non-uniformity correction on the currently acquired image in real time. Summary of the Invention

[0005] The technical problem to be solved by this invention is to overcome the above-mentioned defects of the prior art and provide a real-time prediction method for bright-field microscopy plan images based on neural networks. This real-time prediction method for bright-field microscopy plan images based on neural networks uses deep learning technology to overcome the problem of current traditional computer vision techniques that rely on multiple images to predict plan images. By designing an efficient neural network architecture, it directly inputs an image to predict the corresponding plan image, thereby enabling real-time and rapid correction of brightness inhomogeneities in microscopic images.

[0006] This invention is achieved through the following technical solution:

[0007] A method for real-time prediction of bright-field microscopy plan-field images based on neural networks includes the following steps:

[0008] S1: Acquire images of multiple scenes using a bright-field microscope and their corresponding plan-field images;

[0009] S2: Obtain the corresponding brightness-corrected image according to the flat field correction formula;

[0010] S3: Design and construct a neural network architecture for directly predicting flat-field maps of microscopic images;

[0011] S4: The loss function for constructing the neural network architecture for training and predicting flat-field plots;

[0012] Assuming the input image is The flat-field images corresponding to the multi-scene bright-field microscope images acquired according to S1 are as follows: According to S2, the result after standard brightness non-uniformity correction is as follows: Set the input image The flat field diagram predicted according to the neural network architecture designed by S3 is as follows: Calculate the corresponding result after brightness non-uniformity correction. The loss function is constructed as follows:

[0013] ,

[0014] in Represents the absolute error loss. Set as and The absolute error loss, Set as and The absolute error loss, Represents multi-level structural similarity loss. Set as and Multi-level structural similarity loss, Set as and Multi-level structural similarity loss, , , and They are respectively , , and The weight parameters are all greater than 100. , Representing a flat field diagram The average value in the channel dimension. The representative flat field diagram is The average value across the channel dimension Representative input image Using the flat field diagram The result after correction according to the flat field correction formula;

[0015] S5: Training the neural network architecture for predicting flat-field micrographs: Using the images acquired in S1 as the training set, the neural network architecture in S3 is trained using the loss function designed in S4, and the corresponding model is trained.

[0016] S6: Brightness non-uniformity correction of microscopic images: Based on the model trained in S5, the corresponding flat field image can be predicted for the input image, and then the brightness non-uniformity correction of the input image is completed by using the flat field correction formula.

[0017] As an optimization, step S1 is as follows: Prepare specimen slides for various types of scene samples, observe each sample slide using a bright-field microscope, and acquire images, as follows:

[0018] S11: Adjust the microscope's field of view, adjusting the corresponding focus, suitable light source brightness, and camera parameters to achieve clear observation of the current field of view and capture images under the current conditions;

[0019] S12: Fix the current focus, light source brightness and corresponding parameters, and acquire the corresponding flat field image;

[0020] S13: Repeat the above data collection process sequentially, and set the data collection... For color microscopic images and their corresponding plan-field plots, set as ,in For the first time collected A color micrograph, It's due to uneven brightness. for The corresponding flat-field images all have a resolution of [missing information]. That is, high is , width is .

[0021] As an optimization, the specific steps of S2 are as follows:

[0022] For the images acquired by S1, Brightness non-uniformity correction is performed, and the corrected result is set according to the flat field correction formula. Image after brightness non-uniformity correction The Channel 1 Line 1 Column pixel values

[0023] ,

[0024] in Representative image The Channel 1 Line 1 Column pixel values, Represents a flat field image The Channel 1 Line 1 Column pixel values, Set as The The average value of each channel, i.e. ,in ,and It is an integer. Representing the blue B channel, Represents the green G channel. Represents the red R channel. , , , , All are integers. Represents a flat field image Average value in channel flavor.

[0025] As an optimization, the neural network architecture includes three modules: a downsampling module, a Fourier convolution module, and a Fourier upsampling module.

[0026] As an optimization, the downsampling module in S3 includes a kernel size of And the step size is The convolutional layers and activation function layers.

[0027] As an optimization, the Fourier convolution module in S3 includes sequentially connected real-valued fast Fourier transform layers and a first kernel size of... And the step size is The convolutional layer, the first activation function layer, the real fast inverse Fourier transform layer, and the second kernel size are And the step size is The convolutional layer and the second activation function layer.

[0028] As an optimization, the Fourier upsampling module in S3 includes a Fourier convolutional layer and a deconvolutional layer.

[0029] As an optimization, the specific process of designing the entire network architecture based on the three modules—downsampling module, Fourier convolution module, and Fourier upsampling module—in S3 is as follows:

[0030] S31: For the input color image Its shape is After three consecutive downsampling operations, the results were obtained respectively. , and ,Right now After downsampling operation ,in The shape is , After downsampling operation ,in The shape is , After downsampling operation ,in The shape is ,and , and The number of channels is ;

[0031] S32: Will Passing through the core with a size of And the step size is The convolution operation and the LeakyReLU activation function operation are used to obtain... Its shape is ;

[0032] S33: After one Fourier convolution module FFT-Conv-IFFT operation After performing a Fourier upsampling module FFT-Conv-IFFT-Up operation, the results of the two operations are added together to obtain the final result. Its shape is ;

[0033] S34: After one Fourier convolution module FFT-Conv-IFFT operation After performing a Fourier upsampling module FFT-Conv-IFFT-Up operation, the results of the two operations are added together to obtain the final result. Its shape is ;

[0034] S35: Original Image After one Fourier convolution module FFT-Conv-IFFT operation After performing a Fourier upsampling module FFT-Conv-IFFT-Up operation, the results of the two operations are added together to obtain the final result. Its shape is ;

[0035] S36: After the core size is And the step size is The convolution operation and the LeakyReLU activation function operation are used to obtain... Its shape is ;

[0036] S37: After the core size is And the step size is The convolution operation and sigmoid activation function operation yield the output of the entire network architecture. , That is, the input color image The corresponding flat field diagram obtained through the entire neural network architecture;

[0037] in The number of channels representing the features after the downsampling process. For high, For width, , , , , All are integers.

[0038] The beneficial effects of this invention are:

[0039] This invention provides a real-time prediction method for bright-field microscope planar images based on neural networks. The method includes acquiring images of bright-field microscopes from multiple scenes and their corresponding planar images; obtaining the corresponding brightness-corrected images according to the planar correction formula; designing and constructing a neural network architecture for directly predicting the planar images of the microscope images; constructing a loss function for training the neural network architecture for predicting the planar images; and training the neural network architecture for predicting the planar images of the microscope images. The acquired images are used as a training set, and the designed loss function is used to train the neural network architecture to train the corresponding model. Finally, brightness non-uniformity correction is applied to the microscope images. This invention uses deep learning technology to overcome the problems of current traditional computer vision techniques that use multiple images to predict planar images. By designing and constructing an efficient neural network architecture for directly predicting the planar images of microscope images, it directly predicts the corresponding planar images from the input images, enabling real-time and rapid correction of brightness non-uniformity in microscope images. According to the neural network model for training and predicting the planar images of microscope images based on this invention, the corresponding planar images are predicted from the input images. Compared with traditional computer vision techniques that use multiple images to predict planar images, this method is no longer affected by the number of images, is independent of the contents of the images, and is not affected by the color, size, or density distribution of objects in the images. Attached Figure Description

[0040] The following section, with reference to the accompanying figures, further explains a method for real-time prediction of bright-field microscope plan images based on neural networks:

[0041] Figure 1 This is a flowchart illustrating a method for real-time prediction of bright-field microscope plan images based on neural networks, according to the present invention.

[0042] Figure 2 This is an example of a color microscopic image and its corresponding flat field image acquired by a method for real-time prediction of bright-field microscopy flat field images based on neural networks according to the present invention.

[0043] Figure 3 for Figure 2 The image in the image is an example of the result after flat-field correction;

[0044] Figure 4 This is a schematic diagram of the downsampling module Down in the real-time prediction method for bright-field microscope plan-field images based on neural networks of the present invention.

[0045] Figure 5 This is a schematic diagram of the Fourier convolution module FFT-Conv-IFFT in the real-time prediction method of bright-field microscope flat-field images based on neural networks of the present invention.

[0046] Figure 6 This is a schematic diagram of the Fourier upsampling module FFT-Conv-IFFT-Up of a real-time prediction method for bright-field microscope plan images based on neural networks according to the present invention.

[0047] Figure 7 This is a schematic diagram of the neural network architecture for predicting the flat field image of a microscopic image, which is a real-time prediction method for bright field microscopy flat field images based on neural networks according to the present invention.

[0048] Figure 8 This is a schematic diagram illustrating how a method for real-time prediction of a bright-field microscope plan view based on a neural network, according to the present invention, corrects for brightness non-uniformity using a plan view predicted by a neural network. Detailed Implementation

[0049] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be noted that the following embodiments are for illustrative purposes only and do not limit the scope of the application. Similarly, the following embodiments are only some, not all, embodiments of the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0050] Please see Figures 1-8 , Figure 1 This is a flowchart illustrating a method for real-time prediction of bright-field microscope plan images based on neural networks, according to the present invention. Figure 2 This is an example of a color microscopic image and its corresponding flat field image acquired by a method for real-time prediction of bright-field microscopy flat field images based on neural networks according to the present invention. Figure 3 for Figure 2 The image in the image is an example of the result after flat-field correction; Figure 4 This is a schematic diagram of the downsampling module Down in the real-time prediction method for bright-field microscope plan-field images based on neural networks of the present invention. Figure 5 This is a schematic diagram of the Fourier convolution module FFT-Conv-IFFT in the real-time prediction method of bright-field microscope flat-field images based on neural networks of the present invention. Figure 6 This is a schematic diagram of the Fourier upsampling module FFT-Conv-IFFT-Up of a real-time prediction method for bright-field microscope plan images based on neural networks according to the present invention. Figure 7 This is a schematic diagram of the neural network architecture for predicting the flat field image of a microscopic image, which is a real-time prediction method for bright field microscopy flat field images based on neural networks according to the present invention. Figure 8 This is a schematic diagram illustrating how a method for real-time prediction of a bright-field microscope plan view based on a neural network, according to the present invention, corrects for brightness non-uniformity using a plan view predicted by a neural network.

[0051] A method for real-time prediction of bright-field microscopy plan-field images based on neural networks includes the following steps:

[0052] Step 1: Acquire images and corresponding plan-field images of various scenes using a bright-field microscope; prepare specimen slides for various types of scenes, observe each slide using a bright-field microscope, and acquire images. The specific process is as follows:

[0053] 1. Adjust the microscope's field of view, including the focal point, light source brightness, and camera parameters, to achieve clear observation and image acquisition under current conditions.

[0054] 2. Fix the current focus, light source brightness, and corresponding parameters, and acquire the corresponding flat field image;

[0055] 3. Repeat the above data collection process sequentially, setting the data to be collected. For color microscopic images and their corresponding plan-field plots, set as ,in For the first time collected A color micrograph, It's due to uneven brightness. for The corresponding flat-field images all have a resolution of [missing information]. That is, high is , width is Example images are as follows Figure 2 As shown, the subgraph ( ) is a type of color image collected for a particular scene, sub-image ( ) is a subgraph ( The corresponding flat plot, subplot ( ) is another type of color image collected, sub-image ( ) is a subgraph ( The corresponding flat plot; Representative image The Channel 1 Line 1 Column pixel values, Represents a flat field image The Channel 1 Line 1 Column pixel values, Representing the blue B channel, Represents the green G channel. Represents the red R channel, in which , , , , , All are integers.

[0056] Step 2: Obtain the corresponding brightness-corrected image according to the flat field correction formula; for the image pairs acquired in Step 1, [the following steps are performed]. Brightness non-uniformity correction is performed, and the corrected result is set according to the flat field correction formula. Example images are as follows Figure 3 As shown, the subgraph ( )yes Figure 2 A schematic diagram of the result of flat-field correction for an image pair of a certain type of scene, sub-image ( )yes Figure 2 Another type of scene in the image is shown in the diagram after flat-field correction; the image after brightness non-uniformity correction. The Channel 1 Line 1 Column pixel values ,in , , , , , All are integers, where Set as The The average value of each channel, i.e. ,in ,and It is an integer.

[0057] Step 3: Design and construct a neural network architecture for directly predicting flat-field images from microscopic images; this network architecture mainly consists of three modules: as follows... Figure 4 The downsampling module Down, as shown, Figure 5 The Fourier convolution module FFT-Conv-IFFT and as shown Figure 6 The Fourier upsampling module FFT-Conv-IFFT-Up is shown; the specific details of the sequential operations of the downsampling module Down are as follows: kernel size is... And the step size is The convolution operation and LeakyReLU activation function operation; the specific details of the sequential operation of the Fourier convolution module FFT-Conv-IFFT are as follows: after one real fast Fourier transform, then after a kernel size of... And the step size is The convolution operation and LeakyReLU activation function operation are then performed, followed by a real fast Fourier inverse transform, and then a kernel size of... And the step size is The convolution operation and LeakyReLU activation function operation; the specific details of the sequential operation of the Fourier upsampling module FFT-Conv-IFFT-Up are as follows: after one Fourier convolution module FFT-Conv-IFFT operation, and then after one deconvolution ConvTranspose operation, the upsampling is completed.

[0058] The entire network architecture designed based on the above three modules is as follows: Figure 7 As shown, the specific process is as follows: For the input color image Its shape is After three consecutive downsampling operations, the results are obtained respectively. , and ,Right now Obtained after downsampling operation ,in The shape is , Obtained after downsampling operation ,in The shape is , The shape is ,and , and The number of channels is .Will Passing through the core with a size of And the step size is The convolution operation and the LeakyReLU activation function operation are used to obtain... Its shape is . After one FFT-Conv-IFFT operation After performing an FFT-Conv-IFFT-Up operation, the results of the two operations are added together to obtain the final result. Its shape is . After one FFT-Conv-IFFT operation After performing an FFT-Conv-IFFT-Up operation, the results of the two operations are added together to obtain the final result. Its shape is Original image After one FFT-Conv-IFFT operation After performing an FFT-Conv-IFFT-Up operation, the results of the two operations are added together to obtain the final result. Its shape is . After the core size is And the step size is The convolution operation and the LeakyReLU activation function operation are used to obtain... The size is . After the core size is And the step size is The convolution operation and sigmoid activation function operation yield the output of the entire network architecture. . That is, the input color image The corresponding flat-field image obtained after the entire neural network architecture. In this embodiment, the following is set... , , .

[0059] Step 4: Construct the loss function for training the neural network that predicts the flat field image; assuming the input image is... The corresponding flat field map collected according to the gold standard operation in step one is as follows: According to step two, the result after standard brightness non-uniformity correction is as follows: Assume the flat-field image predicted by the neural network architecture designed in step three is... .set up The flat-field plot predicted by the neural network architecture designed in step three is: Calculate the corresponding result after brightness non-uniformity correction. The loss function is constructed as follows:

[0060] ,

[0061] in Represents the absolute error loss. Set as and The absolute error loss, Set as and The absolute error loss. Represents multi-level structural similarity loss. Set as and Multi-level structural similarity loss, Set as and Multi-level structural similarity loss. , , and They are respectively , , and The weight parameters are all greater than 100. In this embodiment, it is set , , , .

[0062] Step 5: Train the neural network architecture for predicting flat-field images from microscopic images; using the images acquired in Step 1 as the training set, train the neural network from Step 3 using the loss function designed in Step 4 to train the corresponding model. In this embodiment, an NVIDIA RTX 4090 graphics card is used for training, and the training parameter epoch is set to... It uses the Adam optimizer.

[0063] Step Six: Brightness Uniformity Correction of the Microscopic Image; Based on the model trained in Step Five, the corresponding flat-field image can be predicted for the input image. Then, the flat-field correction formula is used to complete the brightness uniformity correction of the input image. The correction process is as follows: Figure 8 As shown, for the input microscopic image Its resolution is The model trained in step five predicts the corresponding flat field map. Then, the corrected result is obtained using the flat field correction formula. On Intel's 11th generation CPUs, the time to calibrate a single image is 0.09 seconds, which meets the requirements for real-time calibration.

[0064] Unlike existing technologies, this invention provides a real-time prediction method for bright-field microscope plan images based on neural networks. The method includes acquiring images of bright-field microscopes from multiple scenes and their corresponding plan images; obtaining the corresponding brightness-corrected images according to the plan image correction formula; designing and constructing a neural network architecture to directly predict the plan image of the microscope image; constructing a loss function to train the neural network architecture for predicting the plan image; and training the neural network architecture by using the acquired images as a training set and utilizing the designed loss function to train the corresponding model. Finally, the brightness non-uniformity of the microscope image is corrected. This invention uses deep learning technology to overcome the problems of current traditional computer vision methods that use multiple images to predict plan images. By designing an efficient neural network architecture, it directly predicts the corresponding plan image from the input image, enabling real-time and rapid correction of brightness non-uniformity in microscope images. This method is unaffected by the number of images, the contents of the images, or the color, size, and density of objects in the images.

[0065] The foregoing description illustrates the main features, basic principles, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments or examples described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the above embodiments or examples should be considered exemplary and not restrictive. The scope of the present invention is defined by the appended claims rather than the foregoing description, and therefore all changes falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. Any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the technical principles of the present invention should fall within the patent protection scope of the present invention.

Claims

1. A method for real-time prediction of bright-field microscope plan-field images based on neural networks, characterized in that, Includes the following steps: S1: Acquire images of multiple scenes using a bright-field microscope and their corresponding plan-field plots; S2: Obtain the corresponding brightness-corrected image according to the flat field correction formula; S3: Design and construct a neural network architecture for directly predicting flat-field maps of microscopic images; S4: The loss function for constructing the neural network architecture for training and predicting flat-field plots; Assuming the input image is The flat-field images corresponding to the multi-scene bright-field microscope images acquired according to S1 are as follows: According to S2, the result after standard brightness non-uniformity correction is as follows: Set the input image The flat field diagram predicted according to the neural network architecture designed by S3 is as follows: Calculate the corresponding result after brightness non-uniformity correction. The loss function is constructed as follows: ， in Represents the absolute error loss. Set as and The absolute error loss, Set as and The absolute error loss, Represents multi-level structural similarity loss. Set as and Multi-level structural similarity loss, Set as and Multi-level structural similarity loss, , , and They are respectively , , and The weight parameters are all greater than 100. , Representing a flat field diagram The average value in the channel dimension. The representative flat field diagram is The average value across the channel dimension Representative input image Using the flat field diagram The result after correction according to the flat field correction formula; S5: Training the neural network architecture for predicting flat-field micrographs: Using the images acquired in S1 as the training set, the neural network architecture in S3 is trained using the loss function designed in S4, and the corresponding model is trained. S6: Brightness non-uniformity correction of microscopic images: Based on the model trained in S5, the corresponding flat field image can be predicted for the input image, and then the brightness non-uniformity correction of the input image is completed by using the flat field correction formula.

2. The method for real-time prediction of bright-field microscope plan-field images based on neural networks as described in claim 1, characterized in that, The S1 step is as follows: Prepare specimen slides for various scene samples. For each sample slide, observe it using a bright-field microscope and acquire images, as follows: S11: Adjust the microscope's field of view, adjusting the corresponding focus, suitable light source brightness, and camera parameters to achieve clear observation of the current field of view and capture images under the current conditions; S12: Fix the current focus, light source brightness and corresponding parameters, and acquire the corresponding flat field image; S13: Repeat the above data collection process sequentially, and set the data collection... For color microscopic images and their corresponding plan-field plots, set as ,in For the first time collected A color micrograph, It's due to uneven brightness. for The corresponding flat-field images all have a resolution of [missing information]. That is, high is , width is .

3. The method for real-time prediction of bright-field microscope plan-field images based on neural networks as described in claim 2, characterized in that, The specific steps of S2 are as follows: For the images acquired by S1, Brightness non-uniformity correction is performed, and the corrected result is set according to the flat field correction formula. Image after brightness non-uniformity correction The Channel 1 Line 1 Column pixel values ， in Representative image The Channel 1 Line 1 Column pixel values, Represents a flat field image The Channel 1 Line 1 Column pixel values, Set as The The average value of each channel, i.e. ,in ,and It is an integer. Representing the blue B channel, Represents the green G channel. Represents the red R channel. , , , , All are integers. Represents a flat field image Average value in channel flavor.

4. The method for real-time prediction of bright-field microscope plan-field images based on neural networks as described in claim 1, characterized in that: The neural network architecture includes three modules: a downsampling module, a Fourier convolution module, and a Fourier upsampling module.

5. The method for real-time prediction of bright-field microscope plan-field images based on neural networks as described in claim 4, characterized in that: The downsampling module in S3 includes a kernel size of... And the step size is The convolutional layers and activation function layers.

6. The method for real-time prediction of bright-field microscope plan-field images based on neural networks as described in claim 5, characterized in that: The Fourier convolution module in S3 includes sequentially connected real fast Fourier transform layers, and a first kernel size of... And the step size is The convolutional layer, the first activation function layer, the real fast inverse Fourier transform layer, and the second kernel size are... And the step size is The convolutional layer and the second activation function layer.

7. The method for real-time prediction of bright-field microscope plan-field images based on neural networks as described in claim 6, characterized in that: The Fourier upsampling module in S3 includes a Fourier convolutional layer and a deconvolutional layer.

8. The method for real-time prediction of bright-field microscope plan-field images based on neural networks as described in claim 7, characterized in that, The specific process of designing the entire network architecture based on the three modules—downsampling module, Fourier convolution module, and Fourier upsampling module—in S3 is as follows: S31: For the input color image Its shape is After three consecutive downsampling operations, the results were obtained respectively. , and ,Right now After downsampling operation ,in The shape is , After downsampling operation ,in The shape is , After downsampling operation ,in The shape is ,and , and The number of channels is ; S32: Will Passing through the core with a size of And the step size is The convolution operation and the LeakyReLU activation function operation are used to obtain... Its shape is ; S33: After one Fourier convolution module FFT-Conv-IFFT operation After performing a Fourier upsampling module FFT-Conv-IFFT-Up operation, the results of the two operations are added together to obtain the final result. Its shape is ; S34: After one Fourier convolution module FFT-Conv-IFFT operation After performing a Fourier upsampling module FFT-Conv-IFFT-Up operation, the results of the two operations are added together to obtain the final result. Its shape is ; S35: Original Image After one Fourier convolution module FFT-Conv-IFFT operation After performing a Fourier upsampling module FFT-Conv-IFFT-Up operation, the results of the two operations are added together to obtain the final result. Its shape is ; S36: After the core size is And the step size is The convolution operation and the LeakyReLU activation function operation are used to obtain... Its shape is ; S37: After the core size is And the step size is The convolution operation and sigmoid activation function operation yield the output of the entire network architecture. , That is, the input color image The corresponding flat field diagram obtained through the entire neural network architecture; in The number of channels representing the features after the downsampling process. For high, For width, , , , , All are integers.

Images