Image denoising model training, denoising method and device, and storage medium
By introducing parallel convolutional layers and residual structures into the image denoising model, the model training process is optimized, solving the problems of long image denoising processing time and insufficient detail preservation in existing technologies, and achieving efficient and accurate image denoising results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG HUARAY TECH CO LTD
- Filing Date
- 2022-06-09
- Publication Date
- 2026-06-16
AI Technical Summary
Existing technologies for image denoising using deep learning struggle to retain sufficient detail or edge information while reducing noise, and the processing time is also long.
An image denoising model is adopted, which includes two parallel convolutional layers and a residual structure. By preprocessing and training the noisy image dataset, the predicted noise points of the image denoising model are compared with the real noise points to optimize the model, thereby reducing the number of network parameters and improving robustness.
This approach reduces image denoising processing time while improving the accuracy and efficiency of the image denoising model, enabling faster image denoising tasks.
Smart Images
Figure CN115170812B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image denoising technology, and in particular to an image denoising model training, denoising method and device, and storage medium. Background Technology
[0002] Images are one of the mediums through which humans convey information, but during the transmission or recording of images, they are inevitably affected by various noises, impacting their visual quality. Therefore, research on image denoising techniques has significant theoretical and practical value. Traditional image denoising methods include mean filtering, Gaussian filtering, median filtering, bilateral filtering, and the BM3D algorithm. With the development of artificial intelligence technology, deep learning has achieved significant breakthroughs in image denoising tasks. Compared with traditional image denoising methods, how to utilize deep learning to provide an image denoising method that reduces image noise while preserving sufficient details or edge information, and reduces processing time / increases network speed, is a problem that urgently needs to be solved. Summary of the Invention
[0003] This application provides an image denoising model training, denoising method and device, and storage medium, and provides a new image denoising model to achieve image denoising processing, obtain denoised images, and reduce image denoising processing time.
[0004] To address the aforementioned technical problems, this application provides a model training method comprising: acquiring a noisy image dataset, inputting the images to be trained in the noisy image dataset into an image denoising model for prediction, and then training the image denoising model using the predicted noise points of the image denoising model and the real noise points in the images to be trained, so as to obtain the final image denoising model.
[0005] The image denoising model includes at least two convolutional layers connected in parallel.
[0006] The image denoising model includes a residual structure, which includes at least two convolutional layers in parallel; one convolutional layer is a 1*3 convolutional layer and the other is a 3*1 convolutional layer.
[0007] The image denoising model includes a first residual structure, a second residual structure, a third residual structure, a fourth residual structure, and a fifth residual structure.
[0008] The process of inputting the training image from the noisy image dataset into the image denoising model for prediction includes: inputting the training image into a first residual structure; downsampling the output features of the first residual structure and then inputting them into a second residual structure; downsampling the output features of the second residual structure and then inputting them into a third residual structure; inputting the downsampled output features of the first residual structure, the output features of the second residual structure, and the upsampled output features of the third residual structure into a fourth residual structure; inputting the upsampled output features of the fourth residual structure, the output features of the first residual structure, and the upsampled output features of the second residual structure into a fifth residual structure; and using the output features of the fifth residual structure to predict the noise in the training image.
[0009] After obtaining the noisy image dataset, the process includes: image preprocessing of all training images in the noisy image dataset.
[0010] Image preprocessing includes image stitching, image compositing, image flipping, and / or image scaling.
[0011] The preprocessing of all training images in the noisy image dataset includes: dividing the training images into several training sub-images; and reassembling the several training sub-images into a new training image according to a preset order or a random order.
[0012] The model training method further includes: obtaining the predicted noise points output by the image denoising model during model training; denoising the image to be trained based on the predicted noise points to obtain a trained denoised image; obtaining the signal-to-noise ratio of the trained denoised image; and completing the training of the image denoising model when the signal-to-noise ratio is higher than or equal to a preset signal-to-noise ratio threshold.
[0013] The process of obtaining a noisy image dataset includes: obtaining the signal-to-noise ratio (SNR) of all training images in the noisy image dataset; and deleting training images with an SNR higher than or equal to a preset SNR threshold from the noisy image dataset.
[0014] To address the aforementioned technical problems, another technical solution adopted in this application is to provide an image denoising method, which includes: acquiring an image to be denoised, inputting the image to be denoised into a pre-trained image denoising model, and then performing denoising processing on the image to be denoised based on the noise prediction result of the image denoising model to obtain a denoised image.
[0015] The image denoising model is obtained by the model training method described above.
[0016] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide an image denoising device, which includes a memory and a processor. The memory is used to store program data, and the processor is used to execute the program data to implement the model training method or image denoising method as described above.
[0017] To address the aforementioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium that stores program data. When the program data is executed by a processor, it is used to implement the model training method or image denoising method described above.
[0018] The beneficial effects of this application are as follows: Unlike existing technologies, the model training method provided in this application involves inputting the training image from the acquired noisy image dataset into the image denoising model for prediction. Then, the predicted noise points of the image denoising model and the real noise points in the training image are used to train the image denoising model to obtain the final image denoising model. It is worth noting that, in one embodiment, the image denoising model includes at least two parallel convolutional layers. Using parallel convolutional layers reduces the number of network parameters and improves the robustness of the image denoising model to different image noise detections, thereby training an accurate and efficient image denoising model. Furthermore, this image denoising model can not only complete the image denoising task but also reduce the image denoising processing time. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:
[0020] Figure 1 This is a flowchart illustrating the first embodiment of the image denoising model training method provided in this application;
[0021] Figure 2 This is a flowchart illustrating the second embodiment of the image denoising model training method provided in this application;
[0022] Figure 3 This is a schematic diagram of the residual structure in the image denoising model provided in this application;
[0023] Figure 4 This is a schematic diagram of the image denoising model provided in this application;
[0024] Figure 5 This is a flowchart illustrating the third embodiment of the image denoising model training method provided in this application;
[0025] Figure 6 This is a schematic diagram of the structure of the first embodiment of image stitching provided in this application;
[0026] Figure 7 This is a schematic diagram of the structure of the second embodiment of image stitching provided in this application;
[0027] Figure 8 This is a flowchart illustrating the fourth embodiment of the image denoising model training method provided in this application;
[0028] Figure 9 This is a schematic flowchart of the first embodiment of the image noise reduction method provided in this application;
[0029] Figure 10 This is a schematic diagram of the structure of an embodiment of the image denoising device provided in this application;
[0030] Figure 11 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation
[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0032] See Figure 1 , Figure 1 This is a flowchart of the first embodiment of the image denoising model training method provided in this application. The method includes: Step 11: Obtaining a noisy image dataset.
[0033] Alternatively, the noisy image dataset can be obtained from existing image databases, such as RENOIR, Nam, DND, PolyU, and SIDD.
[0034] Among them, RENOIR captured 120 low-light scenes, including indoor and outdoor scenes, with approximately 4 images per scene, including 2 noisy images and 3 low-noise images; Nam captured 11 scenes, with more than 500 .jpeg images captured for these 11 scenes, mostly featuring similar objects and textures; DND captured more than 50 scenes, including indoor and outdoor scenes; PolyU captured more than 40 scenes, including indoor scenes with normal lighting and low-light scenes, and outdoor scenes with normal lighting, with each scene captured 500 times consecutively; the SIDD dataset was captured using 5 cameras under 4 camera parameters for 10 scenes, with 200 scene instances captured, and each scene captured 150 images consecutively, of which 160 scene instances were used as the training set and 40 scene instances were used as the test set.
[0035] Step 12: Input the images to be trained in the noisy image dataset into the image denoising model for prediction; wherein the image denoising model includes at least two convolutional layers in parallel.
[0036] Specifically, in one embodiment, the image noise denoising model uses an Encoder-Decoder model.
[0037] Understandably, the Encoder-Decoder model, also known as the encoder-decoder model, was first applied to the seq2seq problem in natural language processing (generating one output sequence from one input sequence). With technological advancements, the Encoder-Decoder model has become extremely flexible. The encoder is not limited to sequences; it can handle any text, speech, image, and video data. The Encoder and Decoder can be implemented using various modules such as CNN, RNN, and LSTM. Therefore, to be precise, the Encoder-Decoder is not a model, but rather a framework whose purpose is to downsample and encode arbitrary data and then upsample and decrypt it for output.
[0038] It is worth noting that the image denoising model includes a residual structure. In one embodiment, the residual structure includes at least two parallel convolutional layers, that is, a 3*3 convolutional layer is decomposed into two parallel convolutional layers, 1*3 and 3*1. In other embodiments, the two parallel convolutional layers can be other parallel decomposition methods, such as decomposing 3*3 into parallel 1*3 and 3*1, wherein 1*3 and / or 3*1 are concatenated with 1*1. Step 13: Using the predicted noise points of the image denoising model and the real noise points in the image to be trained, the image denoising model is trained to obtain the final image denoising model.
[0039] Specifically, by predicting noise points in the image to be trained and comparing them with the actual noise points in the image to be trained, the predicted noise points are continuously corrected to obtain an image denoising model.
[0040] Unlike existing technologies, the image denoising model training method of this application provides a new image denoising model that can denoise the input image to be denoised to obtain a denoised image. Since the image denoising model used includes at least two parallel convolutional layers, it can achieve the beneficial effects of improving network speed and reducing denoising processing time.
[0041] The following describes the process of inputting the images to be trained from the noisy image dataset into the image denoising model for prediction.
[0042] See Figure 2 , Figure 2 This is a flowchart illustrating a second embodiment of the image denoising model training method provided in this application. The image denoising model includes a first residual structure, a second residual structure, a third residual structure, a fourth residual structure, and a fifth residual structure. The method includes:
[0043] Step 21: Input the image to be trained into the first residual structure, and after downsampling the output features of the first residual structure, input them into the second residual structure.
[0044] Understandably, the first residual structure performs convolution processing on the input training image to obtain a first processing result. Further, the first processing result is downsampled to obtain a first downsampled structure, and the first downsampled result is used as the input of the second residual structure.
[0045] Specifically, the second residual structure in =f downsampling (First residual structure) out ).
[0046] Where the subscript 'in' represents the input of the residual structure, and the subscript 'out' represents the output of the residual structure, f upsampling Indicates upsampling, f upsampling This indicates downsampling.
[0047] Step 22: After downsampling the output features of the second residual structure, input them into the third residual structure.
[0048] Understandably, the second residual structure performs convolution processing on the first downsampling result obtained from the first residual structure to obtain a second processing result. Further, the second processing result is downsampled to obtain a second downsampling result, and the second downsampling result is used as the input of the third residual structure.
[0049] Specifically, the third residual structurein =f downsampling (Second residual structure) out ).
[0050] Step 23: Input the downsampled output features of the first residual structure, the output features of the second residual structure, and the upsampled output features of the third residual structure into the fourth residual structure.
[0051] Understandably, the third residual structure performs convolution processing on the second downsampling result obtained from the second residual structure to obtain the third processing result. Further, the third processing result is upsampled to obtain the third upsampling result, and the first downsampling result, the second processing result, and the third upsampling result are used as inputs to the fourth residual result.
[0052] Specifically, the fourth residual result in =f upsampling (Third residual structure) out )+Second residual structure out +f downsampling (First residual structure) out ).
[0053] Step 24: Input the upsampled output features of the fourth residual structure, the output features of the first residual structure, and the upsampled output features of the second residual structure into the fifth residual structure.
[0054] Understandably, the fourth residual structure performs convolution processing on the third upsampling result obtained from the third residual structure to obtain the fourth processing result. Further, the fourth processing result is upsampled to obtain the fourth upsampling result, and the first processing result, the second upsampling result, and the fourth upsampling result are used as inputs to the fifth residual result.
[0055] Specifically, the fifth residual structure in =f upsampling (Fourth residual structure) out )+First Residual Structure out +f upsampling (Second residual structure) out ).
[0056] Step 25: Use the output features of the fifth residual structure to predict the noise in the training image.
[0057] Specifically, the first, second, third, fourth, and fifth residual structures are all bottleneck layers. In one embodiment, the internal structure of the first, second, third, fourth, and fifth residual structures includes a 1x1 convolutional layer, a 3x1 and 1x3 convolutional layer in parallel, and a 1x1 convolutional layer. The first 1x1 convolutional layer is used for dimensionality reduction (reducing the dimension of high-value features), reducing the number of parameters and thus reducing the computational cost; the second 1x1 convolutional layer is used for dimensionality increase (increasing the dimension of high-value features); the intermediate 3x3 convolutional layer is decomposed into parallel 3x1 and 1x3 convolutional layers, further reducing computational cost and accelerating network speed. In other embodiments, the first, second, third, fourth, and fifth residual structures contain more than three convolutional layers, but each residual structure includes at least two parallel convolutional layers.
[0058] Specifically, please refer to Figure 3 , Figure 3 This is a schematic diagram of the residual structure in the image denoising model of this application.
[0059] Unlike existing technologies, the first residual structure, second residual structure, third residual structure, fourth residual structure and fifth residual structure of this application all include at least parallel convolutional layers. By using parallel convolutional layers, the number of network parameters is reduced and the robustness of the image denoising model to different image noise detection is improved, thereby training an accurate and efficient image denoising model.
[0060] See Figure 3 , Figure 3 This is a schematic diagram of the residual structure in the image denoising model provided in this application. The residual structure includes three convolutional layers: the first layer is a 1*1 convolutional layer, the second layer is a 3*1 and a 1*3 convolutional layer connected in parallel, and the third layer is a 1*1 convolutional layer.
[0061] The first layer, a 1x1 convolutional layer, is used for dimensionality reduction, which reduces the number of parameters and thus the computational cost. The third layer, a 1x1 convolutional layer, is used for dimensionality increase. The second layer, a 3x3 convolutional layer, is decomposed into parallel 3x1 and 1x3 convolutional layers, which further reduces the computational cost and speeds up the network.
[0062] See Figure 4 , Figure 4 This is a schematic diagram of the structure of the image denoising model provided in this application. The image denoising model 40 includes a first residual structure 401, a second residual structure 402, a third residual structure 403, a fourth residual structure 404, and a fifth residual structure 405.
[0063] Specifically, the first residual structure 401 is connected to the second residual structure 402, the fourth residual structure 404, and the fifth residual structure 405; the second residual structure 402 is connected to the first residual structure 401, the third residual structure 403, the fourth residual structure 404, and the fifth residual structure 405; the third residual structure 403 is connected to the second residual structure 402 and the fourth residual structure 404; the fourth residual structure 404 is connected to the first residual structure 401, the second residual structure 402, the third residual structure 403, and the fifth residual structure 405; and the fifth residual structure 405 is connected to the first residual structure 401, the second residual structure 402, and the fourth residual structure 404.
[0064] Specifically, the input to the first residual structure 401 is the image to be trained, and the output of the first residual structure 401 is downsampled and used as the input to the second residual structure 402, the fourth residual structure 404, and the fifth residual structure 405; the output of the second residual structure 402 is downsampled and used as the input to the third residual structure 403 and the fourth residual structure 404, and the output of the second residual structure 402 is upsampled and used as the input to the fifth residual structure 405; the output of the third residual structure 403 is upsampled and used as the input to the fourth residual structure 404; and the output of the fourth residual structure 404 is upsampled and used as the input to the fifth residual structure 405.
[0065] In this way, each residual structure includes at least two parallel convolutional layers, which can reduce the computational cost of network parameters and improve the robustness of the image denoising model to different image noise detection, thereby training an accurate and efficient image denoising model.
[0066] See Figure 5 , Figure 5 This is a flowchart illustrating the third embodiment of the image denoising model training method provided in this application. The method includes:
[0067] Step 51: Obtain the noisy image dataset.
[0068] Step 52: Perform image preprocessing on all training images in the noisy image dataset; wherein the image preprocessing includes image stitching, image synthesis, image flipping, and / or image scaling.
[0069] Optionally, preprocessing of the images to be trained may include image stitching, image synthesis, image flipping, and / or image scaling.
[0070] Understandably, image stitching of training images includes cropping and stitching a single training image into a new image different from the original; stitching together several acquired training images into a single training image; image synthesis of training images, transforming a multispectral black-and-white image into a color image through multispectral color synthesis; image flipping of training images, including flipping vertically and horizontally; image scaling of training images, i.e., enlarging and reducing training images; and image translation of training images, changing the positions of different images, etc.
[0071] Understandably, preprocessing for training can be just one of the preprocessing operations mentioned above, or it can be a combination of the above preprocessing operations. For example, cropping the image to be trained and randomly stitching it together to form a new image to be trained, and then enlarging / reducing the size of the new image to be trained.
[0072] Please refer to details. Figure 6 and Figure 7 , Figure 6 and Figure 7 These are all examples of image stitching in preprocessing.
[0073] Step 53: Input the images to be trained in the noisy image dataset into the image denoising model for prediction; wherein the image denoising model includes at least two convolutional layers in parallel.
[0074] Specifically, in one embodiment, the two convolutional layers connected in parallel are 3*1 and 1*3, while in other embodiments, they can be 5*1 and 1*5, 4*1 and 1*4, etc.
[0075] Step 54: Train the image denoising model using the predicted noise points of the image denoising model and the real noise points in the image to be trained, so as to obtain the final image denoising model.
[0076] Understandably, the predicted noise points are compared with the real noise points in the image to be trained, and the predicted noise points of the image denoising model are continuously corrected until a complete image denoising model containing all predicted noise points is obtained. At this point, the training of the image denoising model is complete.
[0077] Unlike existing technologies, the model training method in this application can not only complete the image denoising task, but also reduce the amount of parameter calculation and improve network speed.
[0078] The following section will illustrate two methods for image stitching in preprocessing.
[0079] See Figure 6 , Figure 6This is a schematic diagram of the structure of the first embodiment of image stitching provided in this application. The image stitching method involves dividing the image to be trained into nine equal parts, obtaining nine smaller images, then labeling these nine smaller images sequentially, and finally shuffling the order of these nine smaller images to obtain a stitched image with a different order than the original. In one embodiment, the original image to be trained is divided into nine equal parts, labeled sequentially according to the numbers 1-9, and then shuffled and reassembled to obtain a new image with a different order than the original.
[0080] See Figure 7 , Figure 7 This is a schematic diagram of the structure of the second embodiment of image stitching provided in this application. The image stitching method involves dividing the image to be trained into four equal parts to obtain four smaller images. These four smaller images are then numbered sequentially. Finally, the order of these four smaller images is shuffled and rearranged to obtain a stitched image with a different order than the original. In one embodiment, the original image to be trained is divided into four equal parts, numbered according to the alphabetical order of AD, and then shuffled and rearranged to obtain a new image with a different order than the original.
[0081] See Figure 8 , Figure 8 This is a flowchart illustrating the fourth embodiment of the image denoising model training method provided in this application. The method includes:
[0082] Step 81: Obtain the noisy image dataset.
[0083] The noisy image dataset includes (not shown in the figure):
[0084] S1: Obtain all training images and their signal-to-noise ratios in the noisy image dataset.
[0085] S2: Remove training images with a signal-to-noise ratio higher than or equal to a preset signal-to-noise ratio threshold from the noisy image dataset.
[0086] Step 82: Divide the images to be trained in the noisy image dataset into several sub-images to be trained.
[0087] Optionally, the image to be trained is cropped into several sub-images to be trained. The cropping method includes regular cropping and irregular cropping. That is, the image to be trained can be equally divided into sub-images of the same size, or it can be irregularly and unequally divided into sub-images of different sizes.
[0088] Step 83: Reassemble the several sub-images to be trained into a new image to be trained according to a preset order or a random order.
[0089] For example, several training sub-images are labeled in their original order, and then the sorting of the labels is shuffled and reordered to obtain new training images that are inconsistent with the original order.
[0090] Step 84: Obtain the predicted noise points output by the image denoising model during model training.
[0091] Understandably, during the training process, image denoising models continuously correct the predicted noise points.
[0092] Step 85: Denoise the training image based on the predicted noise points to obtain a denoised training image.
[0093] Understandably, the predicted noise points of the image denoising model are compared with the real noise points of the image to be trained to obtain a training denoised image, wherein the training denoised image is not necessarily a completely denoised image.
[0094] Step 86: Obtain the signal-to-noise ratio of the trained denoised image.
[0095] Understandably, image signal-to-noise ratio (SNR) is usually an approximate estimate of the SNR, that is, the ratio of the variance of the signal to the variance of the noise. Peak signal-to-noise ratio (PSNR), on the other hand, is a widely used objective standard for evaluating image quality. It is calculated by calculating the gray-level change error of each pixel in the image before and after denoising, summing the squares, and then averaging them to obtain the mean-square error (MSE) of the two images before and after denoising. The peak signal-to-noise ratio is then obtained based on the mean-square error.
[0096] Wherein, the mean square error (MSE) is:
[0097]
[0098] Where K and L are the length and width of the denoised image, n represents the nth pixel of the image, and f n and g n These represent the pixel values of n points before and after noise reduction, respectively.
[0099] The formula for calculating Peak Signal-to-Noise Ratio (PSNR) is:
[0100]
[0101] Among them, 2 8 -1 represents the maximum grayscale value of the denoised image. Generally, the grayscale bit depth of an image is 8 bits.
[0102] Step 87: When the signal-to-noise ratio is higher than or equal to the preset signal-to-noise ratio threshold, the training of the image denoising model is completed.
[0103] Through the above methods, this application can train an image denoising model to obtain a mature image denoising model.
[0104] See Figure 9 , Figure 9 This is a flowchart illustrating the first embodiment of the image denoising method provided in this application. The method includes:
[0105] Step 91: Obtain the image to be denoised.
[0106] Optionally, the image to be denoised can be obtained from an existing image database, such as RENOIR, Nam, DND, PolyU, and SIDD, or from datasets such as PASCAL VOC, ImageNet, MS-COCO, Open Images, and DOTA.
[0107] Step 92: Input the image to be denoised into the pre-trained image denoising model.
[0108] The image denoising model is obtained using the model training method described above.
[0109] Step 93: Based on the noise prediction results of the image denoising model, perform denoising processing on the image to be denoised to obtain a denoised image.
[0110] Understandably, the image denoising model continuously corrects the predicted noise points during the training phase to obtain the noise prediction result. When the image denoising model performs denoising operation on the image to be denoised, it compares the noise prediction result with the image to be denoised and removes the noise points found in the comparison to achieve denoising processing and obtain a denoised image.
[0111] Unlike existing technologies, the image denoising method of this application can denoise noisy images to obtain denoised images. The residual structure of the image denoising model used includes at least two parallel convolutional layers. When using this denoising model to perform denoising operations on the image to be denoised, the amount of computation for image denoising can be reduced, the processing time can be reduced, and the network speed can be accelerated.
[0112] See Figure 10 , Figure 10 This is a schematic diagram of an embodiment of the image denoising device provided in this application. The image denoising device 100 includes a memory 1001 and a processor 1002. The memory 1001 is used to store program data, and the processor 1002 is used to execute the program data to implement the following method:
[0113] A noisy image dataset is acquired, and the images to be trained in the noisy image dataset are input into an image denoising model for prediction. The image denoising model includes at least two parallel convolutional layers. Then, the predicted noise points of the image denoising model and the real noise points in the images to be trained are used to train the image denoising model to obtain the final image denoising model; or
[0114] The image to be denoised is acquired and input into a pre-trained image denoising model. Then, based on the noise prediction results of the image denoising model, the image to be denoised is processed to obtain a denoised image.
[0115] The processor can be called a CPU (Central Processing Unit). The processor may be an integrated circuit chip, or it may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0116] See Figure 11 , Figure 11 This is a schematic diagram of an embodiment of a computer-readable storage medium provided in this application. The computer-readable storage medium 110 stores program data 1101. When the program data 1101 is executed by the processor, it is used to implement the method described above, which will not be repeated here.
[0117] The storage media used in this application include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), or optical discs.
[0118] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A model training method, characterized in that, The method includes: Obtain a dataset of noisy images; Image preprocessing is performed on all training images in the noisy image dataset; wherein, the image preprocessing includes image stitching, image synthesis, image flipping, image scaling, and / or image translation; the image stitching includes: regularly or irregularly cropping the training images to obtain several training sub-images, and re-stitching the several training sub-images into a new training image according to a preset order or a random order; the image synthesis includes: synthesizing multiple training images to transform a multispectral black and white image into a color image through multispectral image color synthesis; the image flipping includes: flipping the training images, including flipping them vertically, horizontally, and so on; the image scaling includes: enlarging or reducing the training images; the image translation includes: translating the training images to change the position of the middle of different training images; The images to be trained in the noisy image dataset are input into the image denoising model for prediction. The image denoising model is trained using the predicted noise points of the image denoising model and the real noise points in the image to be trained, so as to obtain the final image denoising model. The image denoising model includes residual structures, which are a first residual structure, a second residual structure, a third residual structure, a fourth residual structure, and a fifth residual structure. Each of these residual structures—the first, second, third, fourth, and fifth—is a bottleneck layer. Each of these residual structures sequentially includes three convolutional layers: a 1x1 convolutional layer, a 1x3 and a 3x1 convolutional layer in parallel, and a 1x1 convolutional layer. The 1x1 convolutional layer in the first layer is used for dimensionality reduction, and the 1x1 convolutional layer in the third layer is used for dimensionality increase. The step of inputting the images to be trained from the noisy image dataset into the image denoising model for prediction includes: The image to be trained is input into the first residual structure, and the output features of the first residual structure are downsampled and then input into the second residual structure. After downsampling the output features of the second residual structure, the input is fed into the third residual structure; The downsampled result of the output feature of the first residual structure, the output feature of the second residual structure, and the upsampled result of the output feature of the third residual structure are input into the fourth residual structure; The upsampled output features of the fourth residual structure, the output features of the first residual structure, and the upsampled output features of the second residual structure are input into the fifth residual structure. The noise in the training image is predicted using the output features of the fifth residual structure.
2. The method according to claim 1, characterized in that, The method further includes: During model training, the predicted noise points output by the image denoising model are obtained; The training image is denoised based on the predicted noise points to obtain a denoised training image. Obtain the signal-to-noise ratio of the trained denoised image; When the signal-to-noise ratio is higher than or equal to a preset signal-to-noise ratio threshold, the training of the image denoising model is completed.
3. The method according to claim 2, characterized in that, The acquisition of the noisy image dataset includes: Obtain the signal-to-noise ratio of all training images in the noisy image dataset; Images to be trained with a signal-to-noise ratio higher than or equal to the preset signal-to-noise ratio threshold are removed from the noisy image dataset.
4. An image noise reduction method, characterized in that, The method includes: Obtain the image to be denoised; The image to be denoised is input into a pre-trained image denoising model; wherein the image denoising model is obtained by the model training method according to any one of claims 1-3; The image to be denoised is denoised based on the noise prediction results of the image denoising model to obtain a denoised image.
5. An image denoising device, characterized in that, The image denoising device includes a memory and a processor. The memory is used to store program data, and the processor is used to execute the program data to implement the model training method as described in any one of claims 1-3 or the image denoising method as described in claim 4.
6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program data, which, when executed by a processor, is used to perform the model training method as described in any one of claims 1-3 or the image denoising method as described in claim 4.